Healthcare: how reliable is our information?

The Welsh Assembly Government and its Health Boards will soon be announcing their plans supported by the widely publicized new report from Professor Marcus Longley “Best configuration of hospital services for Wales”. One of the many points made is that Welsh performance is supposedly worse than England and is very variable between health boards – so we’ve got to put that right, haven’t we? You can bet the government will use that to convince us. But how reliable is this information? As Prof. L. himself puts it: “It is in the nature of this evidence sometimes to be frustratingly vague, inconclusive, contradictory, or simply non-existent Well, try to get your head around this. Risk Adjusted Mortality Index (RAMI for short). It is the actual number of deaths divided by the expected number. That's what's so poor in Wales. But how do you work out the expected number? From the hospital diagnosis codes. And how reliable are they?

An article by Dr Paul Robinson looking into this in England says it all. “Coders are only as accurate as the source they use, most frequently the discharge summary which is generally completed by the most junior doctors”. Anybody who has worked in healthcare knows how unreliable that is. And Dr Robinson found that the number of codes per patient varies enormously between hospitals – from about 2.8 to 5.5 with an average of 3.8. The higher the number of codes, the better the RAMI, and the better your hospital looks. What is more, patients having palliative care who cannot be saved are generally excluded (after all its not the hospital’s fault they died) – but this only happens if they get the right diagnosis code. And that is even more variable – from 0 to about 50% of patients in England!

This research hasn’t been done in Wales yet. So differences from England may well be due to coding rather than performance – and variability between Health Boards is highly likely to be due to that, rather than variation in performance.

So don’t believe everything you see. Or, in other words, beware of the RAMI.

Peter Milewski Pembrokeshire Health Concern