Open debate: is it ever justified to withhold clinical trial data?
The debate over open access to clinical trial data is not a new one, and in August two editorials in the New England Journal of Medicine drew the battle lines once more. The big question: does it strengthen academic research, or lead investigators vulnerable to ‘data parasites’? Abi Millar looks at both sides of the argument.
Should clinical trial data be openly available to all, enabling other scientists to build on those findings? It’s a question with almost utopian overtones, suggesting a society in which collaborative endeavour, not individual glory, is the ultimate goal. If your raw data is lying there, ready to be re-examined, it is quite possible another researcher will spot something you’ve missed, enabling them to explore new lines of analysis.
It may also improve patient safety. If the data is there for all to see, there is less scope for scientists to misinterpret or exaggerate their own findings in a bid to rush their drug to market.
Benefits of data sharing
Think of the disclosure, 20 years after the drugs were first launched, that the antidepressants known as SSRIs might increase adolescents’ suicide risk. If the data had been available for review from the outset, it is likely some deaths could have been avoided.
On a similar note, the flu drug Tamiflu remained controversial for many years until, in 2014, the Cochrane Collaboration reviewed the data. They concluded the drug was not as efficacious as believed, and did not in fact reduce the number of hospitalisations.
Data sharing, in this context, has obvious benefits. And in August, a perhaps surprising advocate – Massachusetts senator Elizabeth Warren – waded into the debate. Writing in the New England Journal of Medicine (NEJM), she claimed that “data sharing has incredible potential to strengthen academic research, the practice of medicine, and the integrity of the clinical trial system”.
She was not saying anything new – the push towards more transparent research has been underway for years – but her editorial did stoke controversy. Featured in the same issue was another piece, by a group of scientists called the International Consortium of Investigators for Fairness in Trial Data Sharing, which sounded a note of caution.
According to the consortium, publishing data prematurely would lead to “analyses aimed at unfairly discrediting or undermining the original publication”, as well as diverting “resources, both financial and human, from the actual conduct of trials”.
A host of concerns
Both these editorials were published in response to the International Committee of Medical Journal Editors (ICMJE), which recently put forth new data sharing guidelines. Under the ICMJE’s proposal, scientists would need to make public their clinical trial data (stripped of any identifying patient information) within six months of publishing their first analysis.
While Warren called this requirement “a significant step forward in improving the transparency of clinical trials for consumers and the academic medical community”, the research consortium said it would be critical “to discuss the potential benefits, risks, and opportunity costs, as well as whether the same goals can be achieved by simpler means”. They felt that scientists should retain exclusive access to their data for considerably longer than six months.
According to Dr Gordon Guyatt, a professor at McMaster University and member of the consortium, data sharing initiatives such as the ICMJE’s have led to a “whole host of concerns”.
“One central one is the risk of undermining the research endeavour, particularly the research endeavour related to large trials, which involve a huge commitment on the part of the individuals leading it.
“If individuals are to be sufficiently motivated and rewarded, they need to have the opportunity to explore the data fully before it gets handed over to anybody else,” Guyatt says.
As he sees it, being forced to release data after six months could have a number of adverse consequences. To begin with, it might encourage investigators to delay the publication of their primary trial results. This is already a problem; according to one Nature report in 2013, only half of all clinical trial results are ever published, which leads to negative results being buried and skews the process of drug development.
“If the data’s going to be available to everybody after period X, people might say they want to withhold the first publication until they have done a lot of the secondary analyses, to ensure they will be able to take the lead and get the appropriate credit for these analyses,” explains Guyatt.
He feels it could also lead to data dredging, inappropriate subgroup analysis and misleading presentation of results. In a worst-case scenario, it might even dissuade talented people from getting involved in clinical research at all.
“Putting the data out there in a format people can use involves a considerable amount of time and effort,” he remarks. “So who’s going to pay for this to be done? It shouldn’t be up to the investigators. Getting a large trial funded in the first place is enormously difficult – the administrative barriers have got worse and worse.”
“I’m in the latter stages of my career, and I’m not sure if I was facing what young investigators are facing today, I’m not sure I’d have launched into it.”
A counter approach
The consortium’s editorial suggests some practical ways of addressing these concerns, without surrendering the idea of data sharing altogether. First, it recommends open discussion between the ICMJE, trialists and other stakeholders, to explore all facets of data access.
Secondly, it suggests the timeline for publication of data should be extended, allowing “a minimum of two years after the first publication of the results, and an additional six months for every year required to complete the study, up to a maximum of five years”. In practice, this would mean small trials would release their data within two and a half years, and large trials within five years.
Thirdly, it proposes that an “independent statistician should have the opportunity to conduct confirmatory analyses before publication of an article”, increasing reader confidence in the published data.
Finally, and perhaps most controversially, it stipulates that anyone who wasn’t involved in the trial, but wants access to the data, should financially compensate the investors for their efforts.
“The original proposals I think weren’t well thought through, so I’m optimistic the editors will listen to the feedback and take their time, instead of jumping into something that is not well thought out,” says Guyatt. “In the end we will have something that is of benefit to the scientific community.”
The consortium’s article, which was reviewed and endorsed by 282 investigators from 33 countries, has amassed widespread support among certain factions of the research community. So-called ‘research parasites’ have long been an area of concern. In January, NEJM editor Jeffrey Drazen stated that while “the aerial view of the concept of data sharing is beautiful… many of us who have actually conducted clinical research, managed clinical studies and data collection and analysis, and curated data sets have concerns about the details.”
He stated that if data sharing was to work, it needed to happen “symbiotically, not parasitically,” eliminating the risk that certain researchers will use another group’s work for their own ends.
Others, however, maintain that these kinds of concerns are missing the point.
“I’m genuinely sad and baffled that people with prosperous careers supported by public funding can argue that the research they have performed through the altruism of unpaid people taking voluntary risks is somehow their private property,” wrote Dr Richard Lehman, a notable proponent of open science, in the BMJ.
Dr Vinay Prasad, an assistant professor of medicine at Oregon Health and Sciences University, framed the debate as a matter of individual versus societal gain. “It pits someone’s personal motivation to have an exclusive data that they alone can profit from, set against society’s motivation to use these data for the greatest good,” he told STAT News.
Elizabeth Warren’s editorial, for its own part, stated that while she understood “the trepidation that some academics in medical research feel when they contemplate publicly sharing data”, “the stakes are too high to step back in the face of that challenge”.
Clearly, there is a need for greater transparency in medical research, and few would be prepared to argue that data sharing per se is a bad idea. However, what this should involve in practice is likely to remain a matter of some controversy.
“It has to be done in such a way that optimises its usefulness and will not have all these unintended negative consequences,” Guyatt explains.