Its self-important press release captures the flavour of the report, telling us that the TPA "can today reveal that nearly 12,000 fewer people would die each year if the NHS matched standards in Europe".
Even this is weasel-worded, though. The comparison is not with "Europe", as might be inferred from the claim, but with some countries in Europe, specifically the Netherlands, France and Spain – chosen because the figures have been collected on a comparable basis - not that that means they are comparable.
And in that last comment lies the fatal flaw. The comparator chosen by the TPA is the "Mortality amenable to healthcare" index, using "aggregate mortality" amenable to healthcare to compare performance between the different system. This does not support the conclusions offered.
To establish this, one need only find that the TPA cites for its methodology Nolte & McKee and a paper in the BMJ in 2003. And here in that same paper, the authors state:
So how can amenable mortality be used to understand the performance of a health system? We argue that it does have some value, but not in terms of the aggregate values (although arguably the rankings produced by amenable mortality have greater face validity than those using disability adjusted life expectancy). Instead, it enables comparison of the elements that make up the overall figure, permitting investigators to dig down to look at specific policies and learn from different experiences.In other words, while the TPA claims that the measure "is a comparative method that is robust and well respected", that is very far from the case. And just to reinforce that, in a paper later than that cited by the TPA, Nolte & McKee revisit the concept of "avoidable mortality" which underpins the idea of "mortality amenable to healthcare".
Here, they re-affirm that, when the concept of "avoidable mortality" was first developed, it was seen as a way of identifying topics for further study rather than being an end in itself. "This", they add, "is even more true now that it was then". The information can be used to track changes within a national system, but it is less valid as an international comparator.
For this reason, another system is proposed, measuring the difference between amenable and non-amenable causes, which better reflect economic, cultural, lifestyle and ethnic effects, which may be obscured by a single index. Even marital status can have a significant effect on outcomes, independent of quality of intervention.
With that, we find that the OECD - an organisation dedicated to making international performance comparisons – warns that "avoidable deaths" should not be interpreted as absolute measures of outcome. They "do not provide definitive evidence that a particular service is wrong".
Yet, interpreting "avoidable deaths" as an absolute measure is precisely what the TPA does, going on to build a series of recommendations of a highly political nature, on what amounts to a foundation of sand.
One can, of course, concede that there are many things wrong with the NHS and in a highly politicised debate, there are plenty of views. No one is helped, though, by low-grade research of little more value than a polemic, in an area where good quality work is urgently needed. We do not have so many resources that we can afford to waste effort like this.
COMMENT THREAD
