Congratulations, Ben Goldacre!  Damning Report From The Public Accounts Committee On Clinical Trial Results Being Withheld tells it all.

On 3 January, the Public Accounts Committee of the House of Commons issued a report which expressed concern at the fact that pharmaceutical companies tend to publish results of clinical trials which make them look good, but withhold publication of trials in which the results are less favourable. This affects doctors’ knowledge and perceptions of drugs and treatments, and possible problems.

Ben Goldacre has long campaigned for open publication of research, negative as well as positive. I have mentioned his most book, Bad pharma, and his TED Talk Battling bad science in an earlier article, Flattering Flaws. These are only recent examples of his writings and other presentations; he is very worth following up, his blog Bad Science is worth following.

As well as pharmaceutical companies and other vested interests failing to release research which might impact sales and reputations, there is also the problem of publication bias: the reluctance of the academic and professional press to publish studies which suggest failure to replicate earlier research.

It all makes for a distorted picture of what we know and what we are allowed to know. It surely happens in other fields too, from climate change studies to studies of unemployment, even to studies of the effectiveness of plagiarism detection text-matching software. When what we know and what we are allowed to know is medical, especially when what we know is medical, the outcome may lead to personal tragedies and even fatalities.

The problems of medical research are exacerbated in that the size of a sample or the number of people in any one study may be small; there are many variables which may or may not contribute to the effectiveness of a treatment. Simplistic as it may be, it is by aggregating the results of many studies that decisions can be made as to whether a treatment is effective, how effective and how often effective, and perhaps suggest whether there are some people who may be more at risk than others because of these other variables. Vested interests are always a problem – and who has more vested interest in a research study than a body which makes money from the product under investigation?

Suppression of data makes for distortion – and thus the campaign for open publication of research. Ben Goldacre is a founding member of alltrials.net, an organisation which campaigns for “All Trials Registered | All Results Reported.”

As a result of its investigations, the Public Accounts Committee has made a recommendation to the government:

The Department and the MHRA should ensure, both prospectively and retrospectively, that clinical trials are registered on an appropriate registry and that the full methods and results of all trials should be available for wider independent scrutiny, beyond the work undertaken by regulators during the licensing process (Public Accounts Committee: Lack of consensus over how well Tamiflu actually works). 

Things have moved on since I started this post: The Guardian today (6 January 2014) carries an article by Ben Goldacre: “The drugs have to work” (p. 22). He writes that, despite the report, it appears that the British government believes that the problem is resolved and this recognition may be ignored; the government has signed up to EU legislation “which requires better sharing for trials starting after 2014.”

“Better sharing” is not disclosure of “the full methods and results of all trials,” and “starting after 2014” is not the same as “both prospectively and retrospectively.”

Coincidentally, just the day before publication of the PAC report, Global Research News reprinted part of a PlosOne article by Daniele Fanelli, How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data.

In this paper, Fanelli states

What constitutes scientific misconduct? Different definitions are adopted by different institutions, but they all agree that fabrication (invention of data or cases), falsification (wilful distortion of data or results) and plagiarism (copying of ideas, data, or words without attribution) are serious forms of scientific misconduct [7], [10]. Plagiarism is qualitatively different from the other two because it does not distort scientific knowledge, although it has important consequences for the careers of the people involved, and thus for the whole scientific enterprise [11].

Reference [11] leads to an even earlier paper, Fostering Integrity in Research: Definitions, Current Knowledge, and Future Directions, by Nicholas H Steneck. Here Steneck says:

Although it might seem counter-intuitive to suggest, there are good reasons for questioning how much the three presumed most serious research misbehaviors—FFP—adversely impact the research record or society. Plagiarism has no necessary impact on the reliability of the research record. Results are results, whether or not the person reporting them deserves credit for their discovery.

I am not so sure.

Results are results, assuming that the plagiarist did not plagiarise the results as well as the introduction, review of the literature, discussion of methodology, discussion of results, conclusion or whatever else was plagiarised.

But if the paper is pulled for plagiarism, then the paper is invalidated, however good or innovative the methods or methodology, the findings, the implications, the use… If the paper is withdrawn, its results are not results, results which count for the record.

Steneck recognises that a plagiarist may gain in terms of recognition or reputation if the plagiarism is undiscovered, and that advice given may be flawed if (as a result of the plagiarism) the plagiarist delivers statements or opinions on matters which s/he does not really understand, but still maintains that plagiarism is qualitatively different from other forms of academic or scientific misconduct.

I am still not so sure. What is more, I believe that all this has relevance, at the school level as well as at the cutting-edge of genuine scientific research. I intend to explore this further in a future post.

For the time being though, the consideration: is failure to disclose and failure to publish a subset of falsification of results, or is it a fourth class of (scientific) misconduct? For us all to ponder.

Footnote:  

I came across the term “Texas sharpshooter” in a book by David McRaney, You are not so smart. It refers to “cherry-picking,” selecting items or results which make you look good and ignoring those which don’t.