Micromanagement as a substitute for masterly inaction

Elsewhere, I have said that you should always look at the data.  That is not always the case.

When I worked for Norfolk Children’s Services, we reported monthly on the number of Looked After Children – i.e. children staying with foster carers. We looked at the total number, children leaving foster care, children entering foster care, the demographics and all that. “Benchmarking” had shown that the number of Looked After Children was “too high”. I’m sceptical – can you really compare the aggregate of such complex lives between one local authority and another? And surely, if a child needs foster care they need foster care and that should be that.

The Powers That Be decided that the number that was “too high” should come down. To this end, we were asked to report weekly with details of every child that had entered and left care. Why? What could possibly be achieved by this? The highly-paid head honchos were agonising over data that should have been the concern of the social workers involved and their managers. THEIR concern was and should be the children, but now they had weekly numbers to be implicitly judged upon. But if a child needs foster care he needs foster care. End of. Should the social workers decline to put children into care, or take them out too early in order to bring the figure down?

The Powers That Be wanted to do the right thing. But is it within the power or remit of a Local Authority to affect the number of children who are or should be in care? Surely that is far too wide and deep a societal issue for a council to solve. But when Something Must Be Done, that answer is unacceptable. If nothing can really be done, then the appearance of action (to deceive oneself as much as others) is necessary. When no meaningful action can be taken, only meaningless action is possible. Hence, an ineffectual but well-meaning micro-management.

Don’t be so sure of yourself

Try as I might, most statistics is beyond my understanding.  However, I do know one thing that put’s me ahead of most people on this front:

Don’t be so sure of yourself

The spurious precision of a test statistic is a great comfort – you are trying to understand something complicated, and LOOK!  A number, which tells us everything!  Except it doesn’t.  It tells us something about a parameter of model which may or may not approximate your data.  The data may be misleading (intentionally or unintentionally on the part of the person who generated the dataset), or the tests chosen specifically to produce the magical “statistical significance”.

When you read an article about how such-and-such has been “proved” or “shown”, ignore the spurious confidence and head for the data.  What AREN’T you told?  How much data was collected to draw this conclusion?  On this final point, I can’t put it better than this excerpt from a contaminated land email list I’m on:

For samples with an inhomogeneous matrix, (most made ground in this country) the single result taken from the 0.5 kg sample submitted, generated on a 5g sample that is actually analysed is therefore questionable. In fact the statistics of taking a 0.5kg sample on a 10m x 10m grid at 0.5m depths is the same as going into Leeds city centre, randomly tapping a person on the shoulder, asking them if they are male or female and then asking their age and using this data to extrapolate the average age and sex of the population. Not many census or market research firms would view this as credible statistics.

Liberal Democrats slump in polls… but will win more seats

The Liberal Democrats continue to decline in the polls, now down to 15% according to Yougov for The Sun.  This has been interpreted as due to an exodus of left-wing LibDem voters turned-off by the fiscal sanity of the Coalition government, of which the LibDems are a part.

I don’t vote LibDem, I vote Conservative but I want the Coalition to succeed and that requires the LibDems to avoid decimation at future elections.  Whilst it is likely that they will lose seats which they are contesting with Labour, they should hold there own in contests with the Conservatives, although this assumption completely ignores how being in coalition affects this dynamic.

I’m not losing sleep though, because as I describe here and here, the number of LibDem seats is weakly negatively correlated with their share of the vote – i.e. more votes, fewer seats.

Evidence-based blogging – “Heart attack admissions fall after smoking ban”

The BBC reports that:

There were 1,200 fewer hospital admissions for heart attacks in England in the year after July 2007 – when the smoking ban came in, research suggests.

While the 2.4% drop was less dramatic than that reported in some areas where similar bans have been introduced, the figures suggest it saved the NHS £8.4m.

Researchers said even a small reduction had “important public health benefits”.

The Bath team analysed English hospital admissions between 2002 and 2009, the British Medical Journal reports.

Three of the four authors are based at the University of Bath Tobacco Control Research Group, and the full article is published here which, I’m delighted to report, is open access.

In coming to their conclusion the authors created a model, made predictions, controlled for various factors, and used specific statistical techniques.  All of this is of course good and proper, but the way these things get reported it always sounds like prior to the ban there were x number of admissions per year every year and after the ban the number dropped to x minus 1200.  Of course this isn’t the case, and people far more expert than me can debate the pros and cons of the tests, models and assumptions used.

Lets look at the data.  The figure below is taken directly from the paper and shows the number of admissions for myocardial infarctions in England.  The data are obviously quite variable, and there appears to be a downward trend before the ban came into force on 1st July 2007.

These figures, also taken directly from the paper, show the above data broken down by gender and age group.  Again, just from looking at the data it appears that there is a downward trend before the ban.  This does not of course invalidate the authors findings, but I’d be surprised if these graphs get shown in many of the media reports based on the research.  Consider this blog a public service.

Finally, I charted the data in Table 1 in the article, which gives the numbers for each year, so giving a smoother overview of the data shown above.  Again, note the downward trend prior to the ban.

Epidemiological data are usually really complicated, because every individual is different and there are so many other factors to consider beyond the one you are interested in.  As such, although it is not sufficient to just look at the data and say what you see, it is still necessary.  There are many different statistical tools, and many ways of using them, some less appropriate than others.  So never forget to use the tools you have in your face – your eyes!

Soaring Expulsion Rate is a “Blip”

This weeks Birmingham Post has an article entitled “Soaring expulsion rate is ‘blip’”.  Apparently:

A huge increase in the number of children permanently expelled from Birmingham schools is an unexplainable one-off blip, education chiefs have insisted.  Just over 80 pupils were regarded as so disruptive that they were removed from mainstream education by city head teachers in a single term.  The figure, for Autumn 2009, represented a 40% increase on the same period in the previous year

By the standards of some reporting of statistics, this isn’t a bad article.  However, it still provides no context, without which we cannot make a judgement.  For example, what if the history of school expulsions in recent terms was like this?

Very little change, and then the rate shoots up.  This could be something serious.  However what if it was like this?

Suddenly, the jump is just part of a history of wide fluctuations.  Also, notice how in both graphs the penultimate data point is missing.  The article tells us that over 80 (I called it 81) pupils were expelled in Autumn 2009, a 40% increase “on the same period last year” i.e. Autumn 2008.  What was the figure for Spring 2009?  We have no idea, it could be anything.  For all we know it could be higher than 81, in which case the most recent figures are part of an encouraging trend.

Finally, what is the figure in percentage terms?  I couldn’t immediately find the school age population of the city so I crunched some numbers.  Wikipedia tells us that in the 2001 Census the population of Birmingham was 1,113,000.  Of these, 23.4% were under 16.  This calculates as 260,442.  If we assume that there are equal numbers in all of the year groups under 16 (obviously false, but I’m a busy guy), then we can say that 11/16ths of this figure are at school – about 180,000.  So, last term, about 0.045% of pupils were expelled, against 0.031% a year before.  Obviously these population figures are derived from figures that are nearly 10 years out of date, and have rounding errors and incorrect (but reasonable) assumptions thrown in.  Nevertheless, the figure is very very small.

When reading an article about figures, always ask yourself what you are NOT being told.  Not that you are being mislead, but what different backgrounds could actually cast the results in a different light, as we have done here?

Disproportional Representation update

I figured you were desperate for me to update the analysis in the last blogpost with the results of the 2010 election, so as it’s you…

This graph shows how the 2010 election results (large blobs) fit into the overall picture.

The next graph shows how the parties did in terms of how many seats they won per % of the vote.

Last time we observed that the winner had always come out on top by this measure.  This time round though, despite the Conservatives ending up as the largest party (albeit without a majority), Labour actually got more seats per % of their vote.  The unfortunate LibDems saw this measure decline for them this time round.  Further, the negative correlation I identified last time has done for them again this time.  Between 2005 and 2010, their share of the vote increased from 22% to 23%; however, they actually lost 5 seats, going from 62 to 57.  Apparently this has caused much upset to Billy Bragg.

Disproportional Representation

In case you didn’t know, there is a general election in the UK tomorrow.  The Liberal Democrats, the third party, have been agitating for a change in the electoral system for some time, because their representation in parliament is consistently much lower than it’s overall share of the vote would suggest is fair.

Without getting into whether or not a change in the voting system is a good idea (personally, I like the current first past the post system), let’s take a look at the figures.  I took the results in terms of seats gained and proportion of the vote won in all general elections between 1983 and 2005.  Early results for the LibDems are actually for the Liberal-SDP alliance, their precursor party.  Data points are colour-coded – blue for Conservative, red for Labour, and yellow for LibDems.

As you can see by the pattern of the data, the parties have to get to 15-20% before they get any seats at all.  Given that this is close to where the LibDems have polled this obviously puts them at a disadvantage.

Another way of showing this disadvantage is by calculating the number of seats won per percentage of the total vote achieved.  For example, if a party won 25% of the vote and got 250 seats, that would be 10 seats per %.

This chart shows two things.  First, it confirms the disadvantage that the LibDems have to fight against.  But more interestingly, at every election the winning party gets more seats relative to thr proportion of their vote than the opposition.  The Conservatives won the 1983 and 1987 elections by large margins, but also got more seats proportional to their actual vote.  The same can be said for Labour in 1997, 2001 and 2005.  The Conservatives very narrowly one 1992, where the proportions for Labour and the Conservatives are almost identical, with the Conservatives just shading it.  It also shows that the increase for Labour of seats in relation to their vote is more or less matched by the LibDems.

Finally, some stats.  The correlation between Conservative seats and vote share is 0.99; that for Labour is 0.96.  This shows an almost perfect positive correlation between vote achieved and seats gained (a perfect positive correlation is 1, a perfect negative correlation is -1, and no pattern at all is 0).  Weirdly, the LibDem correlation comes out as -0.27, that is the bigger share of the vote they get the fewer seats they win.  They really don’t do well under this system do they?  Having said that, looking at the first graph does not really show any such pattern for the LibDems, so it is probably a spurious result.