1 January, 2006: Ask a stupid question...

[ Home page | Web log ]

So, Happy New Year to all my readers. And with the pleasantries concluded, it's back the frankly tedious subject of the cost of Parliamentary questions. Following my Freedom of Information request, a nice chap in the Treasury has sent me a bunch of information on the cost estimates which the Treasury prepares on Parliamentary Questions; the summary goes like this--

So, what happens if we correct for the indexation which has been applied to the estimates? This:

Plot of cost of written answers, corrected for indexation

-- i.e., a perfect straight line (as expected), but still no evidence for any fixed costs, which is bizarre given that the the 1991 survey certainly found some!

The error here is that the quoted average costs are nonsense, because the indexation takes no account of the variation in the number of questions asked in each year; that means that the quoted annual total costs are also rubbish. Specifically, suppose that the costs in 1991 followed the model T = F + nd (total cost equals fixed cost plus number of questions times direct cost) and therefore the average cost A = F/n + d. In a later year the costs follow T' = F' + n'd' and A' = F'/n' + d'. However, the quoted average cost is not an estimate of A' but rather is iA = iF/n + id, where i is the indexation constant (1.59 by 2003); the difference between these two estimates, and therefore the correction to be applied to the quoted average costs, is iF(n - n')/nn'.

Taking this into account, and using the quoted marginal cost as the estimate of iF/n, gives a result as expected; the fixed cost of answering written questions was about half a million quid per Parliamentary session in 1991. And so we can correct the figures given in Hansard by Stephen Timms a year ago, as follows:

Session Claimed average cost Corrected average cost
2003-04 138 96
2002-03 135 94
2001-02 129 86
2000-01 123 120
1999-00 121 91
1998-99 115 90
1997-98 112 79
1996-97 107 100
1995-96 105 80
1994-95 99 71

-- i.e., the Treasury estimates of average cost per question are typically around 30% over the true average costs, even if you accept that the methodology by which they're calculated is sensible. As it happens, it isn't; two obvious problems stand out. Firstly, fixed and direct costs are unlikely to suffer the same rate of inflation. Secondly, the 1991 study implicitly assumed that the fixed costs of answering written questions over all departments was simply proportional to the total number of questions asked of all departments, which isn't obviously sensible. And note, (emphasis mine)

5.1. Average costs per PQ varied widely across departments. This partly reflected the style of the department concerned (ranging from ``fast throughput of large numbers'' -- up to ``in-depth consideration of potentially important subjects with due consideration''). [...] Consequently, the survey is felt to have provided a wide and representative mix. Its value is in the totality of the mix and, by agreement, separate figures are not being published for each department.

So, in summary, I think the quoted average cost of answering a written question is something of a nonsense. However, the estimate of the direct (``marginal'') cost is relatively sensible, since it was based on a measurement of staff time expended. Happily it is the latter which is used to set policy (on the maximum cost of answering a question before it may be refused on grounds of disproportionate cost).

Of course, we only have one estimate of this figure, from 1991, based on answers to about 1,600 questions, and with no estimate of its variability or accuracy. And whether typical answers still take around 2 hours of staff time per question answered remains accurate today, almost fifteen years on, is less clear. For instance, you might expect that increasing use of IT will have made answering some sorts of questions much easier.

Anyway, I'm not really sure what useful conclusion to draw from this, beyond that the Freedom of Information Act really works, and that no government statistics should be trusted until you've seen the methodology used to compile them. Task for next week: find someting more interesting to write about.

(Update: I should have mentioned that I haven't put up a copy of the stuff I got from the Treasury because of the Crown Copyright in official documents nonsense: I haven't gone to the trouble of transcribing it, and I'm not allowed to put the scanned copies on the web. If you really want a copy you could (a) ask the Treasury yourself -- the document you want is from 1997 1999, is titled ``Quinqennial Review of the Costs of Answering PQs'', and you want the appendices, which include the 1991 report, too; or (b) ask me, but that depends on my having a moment to type the thing in, or hassling somebody with some OCR software.)

Copyright (c) 2006 Chris Lightfoot; available under a Creative Commons License.