The results they really didn’t want you to see: key ME/CFS trial data released

Researchers at Queen Mary University of London (QMUL), have been ordered to release a full data from a controversial ME/CFS (myalgic encaphalomyelitis/ chronic fatigue syndrome) study known as the PACE trial. A preliminary analysis of the data reveals a shocking difference to the original published results, if the trials original data analysis method had been followed. The new analysis suggests that the therapies tested in the DWP funded trial are not nearly as effective as the published results claimed them to be.

The new data calls into question the decision by the National Institute for Clinical Evidence (NICE) to recommend the therapies as standard treatment for ME/CFS in the NHS.

After repeatedly refusing to release the data, QMUL were ordered to do so by a freedom of information (FOI) tribunal appeal. QMUL spent £200,000 on legal costs trying to suppress this information.

The data released vindicates the many esteemed scientists and ME/CFS patients, who have heavily criticised the trial and its negative effects on the public perception of ME/CFS. One of the scientists critical of the study is Dr Jonathan Edwards of University College London, who said:

It’s a mass of un-interpretability to me…All the issues with the trial are extremely worrying, making interpretation of the clinical significance of the findings more or less impossible.

CFS is a disease characterised by a collection of symptoms including physical and mental fatigue, muscular/neurological pain  and cognitive dysfunction; these symptoms can be exacerbated by exertion. It varies in severity and a minority of CFS patients are severely affected and bed-bound. It is estimated that 250,000 people in Britain are currently affected by this debilitating chronic illness.

PACE trial

The PACE medical trial compared four therapies to determine their effectiveness in treating CFS, which is also known as myalgic encephalomyelitis (ME) and is commonly referred to in the UK as ME/CFS. The randomised trial included 641 people with mild to moderate CFS, those with severe CFS were excluded. The therapies under investigation were adaptive pacing therapy (APT) similar to an informal pacing strategy commonly used in CFS, cognitive behavioural therapy (CBT), graded exercise therapy (GET) and specialist medical care (SMC), which consisted of the standard NHS treatment for CFS.

There are two competing, and fiercely debated, theories for the causes of CFS. One theory favoured by patients and a number of prominent scientists, with much evidence to support it, is that the illness is caused by a pathological disease process, sometimes described as a physical or biomedical cause.

The other theory prominent with British mental health experts, and the authors of the PACE trial is that the illness is primarily due to psychological issues. This theory contends many of the symptoms of CFS are due to physical deconditioning caused by lack of exercise and could be cured if aberrant thought processes were changed.

The authors of the PACE trial are proponents of the psychological theory, and the medical trial was designed in light of this. The initial successful results of PACE added weight to that theory and the many years of previous work some of the authors had put into supporting it.

Conflicts of interest

But the results of the trial also suited the aims of the Department for Work and Pensions (DWP), who part funded the trial, and the medical insurance industry which have connections to the PACE trial via the researchers. Four of them declared working for the insurance industry in a conflicts of interest statement: Professors Peter White, Trudie Chalder and Michael Sharpe and physiotherapist Jessica Bavington.

The DWP is particularly keen on psychological explanations, a.k.a the bio-psychosocial (BPS) model for chronic ill health. Professor Tom Shakespeare says, in the Disability News Service, BPS “played a key role” in the narrowing of eligibility criteria for disability benefits such as Employment and Support Allowance (ESA). Shakespeare goes on to say BPS is used to “underpin increasingly harsh and at times punitive measures targeted at disabled people”, in an attempt by the government to cut the number of people receiving ESA.

One of the medical insurance companies White had previously done work for is called Unum Provident Insurance (UPI). They are also proponents of the BPS model of ill health as it helps them to deny disability claims. In 2003 UPI were fined $31.7 million in California for running ‘disability denial factories. In 2005 they were fined $15 million and the California Department of Insurance Commissioner said that “Unum Provident is an outlaw company. It is a company that has operated in an illegal fashion for years…”. UPI is one of the UK’s biggest providers of income protection insurance (IPI) . As a consequence of more holes appearing in the social security safety net, more people are likely to take out IPI, increasing UPI’s profits.

Sir Mansel Aylward, Chief Medical officer for the DWP, sat on the PACE Trial Steering Committee which approved the mid trial changes to the methods. Aylward has also worked for UPI and is a keen proponent of the BPS model, being very influential in promoting it to the DWP.

PACE results published

The results of the £5 million PACE trial were published in The Lancet in 2011. The paper stated that around 60% of the patients treated with CBT or GET gained improvements in fatigue and physical activity levels, while improvements for APT and SMC were minimal. There was extensive criticism of the published paper at this time, including : conflicts of interest, how trial participants were recruited and how the data was analysed and presented.

In 2013 a second PACE paper was published, in Psychological Medicine looking at the recovery of the PACE trial participants 52 weeks after the study. This paper claimed the following recovery rates after treatment: CBT 22%, GET 22%, APT 7% and SMC 7%. These claims again proved contentious as full recovery from CFS is uncommon. A previous systematic review had shown that only 5% fully recovered over time; another inconsistency of the 2013 PACE paper as it wrongly quotes this figure as 7%. Dr Charles Shephard of the ME Association was one of many voices that criticised the findings and inconsistencies of the paper, concluding:

There is widespread concern in the patient community about the way in which these research results relating to CBT and GET are being distorted in the media and adopted by organisations such as the National Institute for Clinical Excellence (NICE) as a ‘one size fits all’ approach to management. To conclude that they are safe and effective for everyone with mild or moderate ME/CFS, as NICE currently does, is simply wrong.

NICE did indeed adopt GET and CBT as therapies to treat CFS, even before the PACE trial had finished. As a result it has become the standard NHS treatment for CFS and is still recommended as such.

The press lapped up the stories emerging from the PACE trial. The Daily Mail declared in 2011 ‘Fatigued patients who go out and exercise have best hope of recovery, finds study”. The BMJs report on the trial said some PACE participants had been “cured”, before any claims for their recovery had been made by the PACE trial. Another Daily Mail headline in 2013 read “Chronic fatigue victims ‘suffer fear of exercise’: Patients are anxious activities such as walking could aggravate the condition”.

There were many more headlines like these, providing false hope and further stigmatising those with ME/CFS by implying that it is within their power to recover if only they would try a little harder.

The data PACE researchers didn’t want to see the light of day

The anonymised PACE trial data released to Aleem Mathees, of Perth, Australia, has undergone a preliminary analysis using the trial methods published in the original protocol. They are compared to the results published by the PACE trial researchers using a substantially altered version of the protocol, changed mid trial, that has been the subject of widespread criticism due to being poorly justified and overly lax.

The changes to the methods were equivalent to widening the goal posts during a game of football. The four criteria used to define their version of recovery were relaxed, resulting in participants being classed as recovered at much lower scores after the methods had been changed (a more detailed account of the changes can be found here)

This initial publication of the preliminary analysis is not peer-reviewed. However, the data analysis has been carried out by the authors which include two expert statisticians, Professor Philip Stark from the University of California and Professor Bruce Levin from Columbia University.

As can be seen in the graph below the mid-trial changes to the PACE protocol inflated the results. And, there were no significant differences between the groups, which contradicts the claims of the PACE trial results which showed that GET and CBT were significantly more likely to be associated with recovery when compared to SMC. The recovery scores calculated in Mathees publicatione were: CBT 7%, GET 4%, APT 2% and SMC 3%.

The authors of the preliminary report call for a thorough, transparent and independent analysis be carried out on the PACE trial results. They suggest:

Pending a comprehensive review or audit of trial data, it seems prudent that the published trial results should be treated as potentially unsound, as well as the medical texts, review articles and public policies based on those results.

CFS activists accused

The FOI request was made in March 2014 following several previous unsuccessful FOI requests which had asked for an analysis of the data according to the outcome measures of the original trial protocol be released. This would enable comparison to the published PACE data set which had been analysed with altered outcome measures in a revised protocol.

QMUL denied these previous requests citing the costs of recalculating the data were too expensive. To prevent a similar refusal, Mathees asked for the anonymised original data set so he could arrange the data analysis himself. This time, QMUL refused on the grounds that the data could not be sufficiently anonymised to protect the identity of the trial participants.

During these FOI requests, QMUL and the PACE researchers adopted a policy of demonising those ME/CFS patients who criticised the PACE trial by portraying them as unreasonable, obsessive and unstable. They hoped that these arguments would convince the authorities not to release information to such dangerous malcontents.

Professor Ross Anderson, of Cambridge University, offered evidence, in support of QMUL’s refusal to release information. His submission to the Tribunal drew parallels between animal rights pressure groups and CFS activists, and suggested there could be a serious risk of violence to participants and researchers if anonymised data was somehow used to identify participants of the trial. The Information Commissioner’s submission to the tribunal showed they were not taken in by this attempted demonisation, stating (p.30):

Professor Anderson’s ‘wild speculations’ about the possibility of ‘young men, borderline sociopathic or psychopathic’ attaching themselves to the PACE trial criticism ‘do him no credit’. Nor do his extrapolations from benign Twitter requests for information to an ‘organised campaign’ from an ‘adversarial group’ show that he has maintained the necessary objectivity and accuracy that he is required to maintain.

The Tribunal’s majority verdict continued on a similar theme, stating (p.40):

It was clear that [Anderson’s] assessment of activist behaviour was, in our view, grossly exaggerated and the only actual evidence was that an individual at a seminar had heckled Professor Chalder.

Investigation into PACE

In 2015, Dr David Tuller, of the University of California, wrote a comprehensive investigative report into the PACE study called Trial by Error. This report documented the many flaws that the PACE trial was accused of, including:

• Violation of the Declaration of Helsinki . Researchers failed to inform study participants of conflicting interests due to paid work from the disability insurance industry

• Changes to the methods protocol half way through the study. Which meant a more favourable outcome was likely to be observed for the therapies being tested.

• Changes to the methods protocol left an unusual artefact. Thirteen percent of the study participants although disabled enough to get on the trial, were also scored as recovered before any treatment took place; once the revised methods were in place.

• A newsletter was released to PACE participants part way through the study singing the praises of CBT and GET. It including positive testimonials from people on the trial (which didn’t indicate which treatment) saying how much the treatment had helped them.

• Claims of successful treatment and recovery were based on how the patients said they felt. Measured outcomes such as – a step test, a walking test, data on finance and employment – failed to support claims of successful treatment.

• Unsuitable use of statistical methods.

The three co-principal investigators of PACE , Professors Peter White, Trudie Chalder and Michael Sharpe, all refuted these allegations in a reply to Tuller’s report.

Calls for retraction

Professor Ron Davis, of Stanford University, is director of the Science Advisory Board overseeing the End ME/CFS Project. He is a leading researcher into the biomedical causes of ME/CFS, and appeared at an international Invest in ME conference in London this year.

I’m shocked that the Lancet published it…The PACE study has so many flaws and there are so many questions you’d want to ask about it that I don’t understand how it got through any kind of peer review.

He is also clear what needs to be done with the publication, saying:

The study needs to be retracted… I would like to use it as a teaching tool, to have medical students read it and ask them, ‘How many things can you find wrong with this study’?

Due to the PACE trial, GET and CBT are the recommended treatments for ME/CFS not just in the UK but around the world. But, the tide may be turning. The US Agency for Healthcare Research and Quality (AHRQ) has downgraded its conclusions on the effectiveness of CBT and GET, due to a re-analysis of the PACE trial and others, instigated by David Tuller’s report. The AHRQ now concludes that GET is ineffective for ME/CFS patients, and that CBT is barely effective.

Aleem Mathees is carrying out a more detailed and complete analysis, which will hopefully be published in a peer-reviewed journal. When this happens the calls for the retraction of the published PACE results will gain traction, and the extent to which the PACE researchers may have held back progress in ME/CFS research and treatment will become apparent.

What also needs to be seriously considered is how the conflicting interests (the DWP and the insurance industry) present in the PACE trial may have influenced the results to come out in their favour. It is troubling cases like the PACE trial controversy that highlights the importance of transparency in clinical trial data so that the results can be verified by other scientists. The PACE trial researchers repeated refusals to release the data discredits them and their science.

Maybe now with the results of the PACE trial being ‘potentially unsound’ more progress can be made in biomedical research into the causes of ME/CFS. Possibly leading to affective treatments for ME/CFS being developed, and decreasing the burden to individuals and society of this debilitating and misery inducing disease.

Get Involved!

– Support Invest in ME Research – charity campaigning for medical research into ME/CFS

– Support AllTrials – campaigning for all medical trials to be published

– Read David Tullers investigative report into the PACE trial

– Support The Canary so we can keep bringing you the news that matters.

Featured image via Invest in Me Research, artwork by Wolfgang Stiller