Wednesday, 3 October 2012

Who's accountable for this spreadsheet? Ministers, civil servants and mistakes

A junior civil servant makes a coding error on a complicated spreadsheet.  Unfortunately, this has direct implications for a hotly debated political issue.  When the error is discovered, the Secretary of State apologises and issues an embarrassing correction.  Not surprisingly, his opposition counterpart is outraged:
"This is an extraordinary development. It really does call into into question the competence of ministers and of the government as a whole"

This was the Department for Work and Pensions, 2007.  The Secretary of State was Peter Hain and his shadow was Chris Grayling, now Secretary of State for Justice. The incorrect classification of a number of Labour Force Survey variables meant that answers we'd given to Parliamentary Questions about the number of foreign nationals in employment - then as now a difficult political issue - were wrong.  I remember it well; I was the Chief Economist, and if the checking and quality control processes didn't work - and here, very exceptionally, they didn't - I had at least a share of the accountability.

One big difference between this episode and the West Coast Main Line saga (apart from the fact that by any measure it was by some orders of magnitude less important!) was that the error was not discovered because someone else spotted it, or a court case.  We discovered it internally, as a result of cross-checking the detail of our numbers with some ONS data, and fessed up immediately.    As soon as I and other senior managers were told, we decided immediately that a formal correction was required; the Secretary of State would have to write to Chris Grayling and put the correct figures on the record.

In my view, Peter Hain behaved admirably. He agreed that a formal correction and apology was required; and commendably - given that he knew that the result would be considerable personal and political embarrassment for something that was absolutely not his fault - showed absolutely no sign of anger or resentment at me or my senior colleagues, still less at the unfortunate junior official involved.  Nor did he at any time try to blame civil servants. His letter to the Speaker and his Opposition counterparts said simply:
"I apologise for having to make this revision."
In contrast, the Secretary of State for Transport has explicitly refused to apologise, either personally or on behalf of the government or his Ministerial predecessors:  
"I'm not going to apologise for the terrible mistake that has been made by the Department."
And apparently the Prime Minister thinks none of his Ministers have done anything wrong: 
"The prime minister believes that the errors were a serious technical blunder by civil servants and that transport ministers Justine Greening and Theresa Villiers could not have been expected to spot the mistakes amid pages of numbers."
Well, leaving out the standard political hypocrisy, and the less than graceful attempt to imply that it's all the civil servants' fault, there is considerable force to the basic argument here.  Secretaries of State shouldn't be checking spreadsheets.  If DFT civil servants got their modelling wrong, and internal checks failed to pick this up, then it is DFT senior management - not Ministers, or the junior staff who made the actual mistakes - who are directly responsible.  

But this doesn't mean Ministers didn't do anything wrong or have nothing to apologise for.  Ultimately, they took the decision to award the franchise to First Group.  This decision was hardly uncontroversial or low profile - even leaving aside Virgin's sour grapes, many informed commentators clearly thought that First Group's revenue projections were hopelessly optimistic.  

So Ministers had months in the run up to the franchise award in August, and two months since, to require DFT senior management to explain to them - not with pages of numbers, but with convincing analysis - why this view, now apparently vindicated, was wrong.  No remotely competent Minister would accept the explanation "That's what the model says" on an issue like this.   So either they didn't ask the right questions, or they were incapable of understanding that they were getting the wrong answers.  Neither interpretation reflects well. 



7 comments:

  1. There seems to be a patter of antagonism between the Coalition ministers and their departments (Transport, Brodie Clark at the Home Office). As a manager, hanging mt staff out to dry doesn't seem strategic, particularly when they will be there for a long time and I'm likely to go in a bit.

    ReplyDelete
  2. Well, but how does one explain the issue (re final paragraph above) without, if not pages of numbers, some numbers based on calculations ministers should not be expected to check?

    Isn't it not so much whether First Group's projections were optimistic, as whether the government was adequately protected if they were optimistic?

    If ministers were taken through the principles that had been applied to minimise risk, but in fact they had not been applied properly through erroneous assumptions in the modelling, could ministers reasonably be expected to know the numbers they were given were wrong?

    ReplyDelete
    Replies
    1. Fair points: we won't know the details of this case until the reviews report, if then.

      But in general I think officials should be able to explain analysis, and Ministers understand it, without having to check numbers. For instance, the Stern Review conclusions are sensitive to choice of discount rates. Ministers shouldn't be expected to understand the detailed calculations. But they should be able to understand, and come to a view, on whether the Stern Review assumptions - on how much we care about our great-times-ten grandchildren, for example - are sensible and defensible.

      Delete
    2. Agreed (i) that ministers should at some level understand the analysis and take a view on key assumptions; (ii) we don't know the details of this case.

      Is this scenario unlikely: that ministers were told
      * our calculations discount future revenues and allow for optimism bias
      * the extent to which they do is consistent with past practice, extrapolated for a longer franchise period, and with what we set out in the tender (possibly citing some numbers)
      * external commentators pose reasonable questions about First Group's projections but we've allowed for the uncertainty so that we're protected to a reasonable level and risk lies largely with them (learning from past mistakes)
      * Virgin have questioned our results but we've checked and are satisfied they are right?

      If this is plausible, and if DfT's calculations were wrong, should we have expected ministers to probe further (in what way?) or is the implication of your headline (ministers are accountable) a touch too definite?

      Delete
  3. This episode makes one wonder about the coalition's longer term ambition to make civil servants more accountable for delivery, which must inevitably reduce senior officials' willingness to go along with (what they regard as) flawed policies. See www.civilservant.org.uk/wcml.shtml if interested in this issue.

    ReplyDelete
  4. I think we should wait and see whether this was the result of a data entry error or whether flawed assumptions were made which led to the particular results. If the latter, we may get into the grey area of middle management doing what senior management wanted. I was interested to see a TV interview with the then SofS in which she said that external consultants had been involved - perhaps we will find out what exactly for.

    ReplyDelete
  5. Interestingly, everyone in the rail industry seems to have been aware of something odd happening for weeks ahead. People were blogging about the process taking a very long time, and openly speculating that it would run out of time and be handed over to the government.

    Roger Ford of Modern Railways had flagged that the payments profile was hugely backloaded in his regular e-mail bulletin, and pointed out that this had blown up before.

    ReplyDelete