All trials should be registered, with their full methods and results reported, and routine audit on the extent of information withheld
The House of Commons Public Accounts Committee delivered a remarkable report on 3 January. Its initial remit was the United Kingdom’s £424m (€510m; $697m) stockpile of oseltamivir (Tamiflu), but the committee soon broadened out—with evident surprise—into the ongoing problem of clinical trial results being routinely and legally withheld from doctors, researchers, and patients.
This situation has persisted for too long. The first quantitative evidence on publication bias was published in 1986.1 Iain Chalmers described in 2006 how progress in the 1990s soon deteriorated into broken promises.2 Recent years have seen extensive denial. The Association of the British Pharmaceutical Industry (ABPI) has claimed that these problems are historic, and that results are now posted on clinicaltrials.gov. The recently defunct Ethical Standards in Health and Life Sciences Group,3 which most UK medical and academic professional bodies signed up to, falsely claimed that a “robust regulatory framework” ensures access to trial results.4 US legislation requiring all results to be posted on clinicaltrials.gov within 12 months of completion has been widely ignored,5 with no enforcement. There has also been covert activity from industry—a leaked memo on its “advocacy” strategy included “mobilising patient groups” to campaign against transparency.6
Despite this, we have achieved considerable progress. The AllTrials.net campaign, started 12 months ago, calls for all trials on all uses of all currently prescribed treatments to be registered, with their full methods and results reported.7 It now has the support of most medical and academic professional bodies as well as the National Institute for Health and Care Excellence (NICE), Medical Research Council, Wellcome, more than 130 patient groups, 60 000 members of the public, and many in industry including GlaxoSmithKline. The Health Research Authority has announced that registration will be a condition of ethics committee approval.8 The BMA has passed a motion stating that withholding trial results is research misconduct,9 and the General Medical Council is re-examining its guidance on the matter.
There have also been extensive new proposals for greater transparency from European Union legislators, the European Medicines Agency (EMA),10 and industry bodies.11 All, however, share the same loophole—they all propose improved access to information on trials conducted from 2014 onwards. This means that almost all trials relevant to current medical practice would be exempt (including, for example, those on oseltamivir).
We now have an unprecedented opportunity for change, with considerable support from medical and academic professional bodies, policy makers, patient groups, and—importantly—the public. It’s time to consider what practical improvements can be made.
Firstly, by whatever means necessary, the methods and results of all previous trials must be accessible to the medical and academic community, which produces the guidelines and systematic reviews that inform patient care. It is commonly assumed that it would be difficult to enforce demands for trial results from diffuse global organisations, but we have never tried simply asking in an organised fashion. For example, the EMA could ask all research organisations and companies with a marketing authorisation for full methods and results of all trials they have conducted, so that these can be posted online, on the first ever register of trials that aspires to be a complete record of all research. If this invitation is declined, we could be told.
Secondly, while the current state of secrecy continues, there is much to be done with the most basic research tool in medicine—audit. Industry is quibbling over the precise proportion of trials that go undisclosed. This should not be a matter of debate. We need a trials observatory, covering all trials on all currently used treatments, that matches registry entries and other sources of information on completed trials against sources of results, whether those are in academic papers, clinical study reports, regulatory documents, or online postings. From these data we could derive live dashboards on transparency to drive up best practice, identify the best and worst companies for missing results, the treatments where most information is missing, the best and worst investigators, and more.
This is actionable information. If routine audit shows a particular principal investigator is performing badly, with many unreported results, should ethics committees grant them access to more trial participants? Will patients participate in trials for companies that withhold results? If two treatments have equivalent benefits, but one comes from a company with a track record of transparency and the other from a company that actively undermines the transparency campaign, are those two treatments still equivalent, and which should a cautious clinician prescribe?
One aspect of the committee’s report was missed by popular commentators, but it exemplifies the peculiarity of the current situation. Professor Kent Woods, head of the Medicines and Healthcare Products Regulatory Agency (MHRA), told the committee that European regulators had everything on oseltamivir. Evidence from the Cochrane Collaboration seems to show that this is not true. Cochrane asked the EMA for all the documents it held on oseltamivir, under that agency’s contested new transparency policy, and the agency complied—on several trials it held incomplete information on the methods and results, and for many more trials, it held nothing.12 The Public Accounts Committee expressed concern, and Professor Woods may wish to clarify this matter. But it is odd that there is any uncertainty about what evidence exists, or what the MHRA, EMA, and NICE have seen, on currently prescribed treatments (www.bmj.com/tamiflu).
It is also remarkable that the medical community needs a committee of generalist politicians to reflect these problems back to us. We spend millions on individual trials to exclude bias and often to detect subtle differences between treatments, but we let those biases pour back in unnecessarily when we permit whole trials to be withheld. Future generations may look back at our present tolerance of withheld trial results in the same way that we look back on medieval blood letting.
Notes
Cite this as: BMJ 2014;448:g213
Footnotes
Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests: BG is involved in a project to audit missing trial results and receives income from speaking and writing for lay audiences on problems in medicine, including publication bias. CH has received grant funding from the National Institute for Health Research Health Technology Assessment Programme to review the evidence for oseltamivir (Tamiflu) and zanamivir (Relenza). BG and CH are cofounders of the AllTrials campaign.
Provenance and peer review: Commissioned; not externally peer reviewed.
References
↵Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol1986;4:1529-41.Abstract/FREE Full Text
↵Chalmers I. From optimism to disillusion about commitment to transparency in the medico-industrial complex. J R Soc Med2006;99:337-41. FREE Full Text
↵Arie S. How a marriage with big pharma ended in divorce. BMJ2013;347:f6062.FREE Full Text
↵Goldacre B, Carroll D, Hall E. Guidelines for collaboration with industry should be transparent. BMJ. 2013;347:f6100.FREE Full Text
↵Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012;344:d7373.CrossRefMedline
↵Sample I. Big pharma mobilising patients in battle over drugs trials data. Guardian2013. www.theguardian.com/business/2013/jul/21/big-pharma-secret-drugs-trials.
↵AllTrials. What does all trials registered and reported mean? All Trials, 2013. www.alltrials.net/all-trials/.
↵Chalmers I. Health Research Authority’s great leap forward on UK trial registration. BMJ2013;347:f5776.FREE Full Text
↵BMA. Non-publication of trials results is research misconduct. http://ecancer.org/journal/news/4200-bma-----non-publication-of-trials-results-is-research-misconduct.php.
↵European Medicines Agency. Draft policy 70: Publication and access to clinical-trial data. 2013. www.ema.europa.eu/ema/doc_index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500144730&murl=menus/document_library/document_library.jsp&mid=0b01ac058009a3dc.
↵European Federation of Pharmaceutical Industries and Associations/PhRMA. Principles for responsible clinical trial data sharing. 2013. www.phrma.org/sites/default/files/pdf/PhRMAPrinciplesForResponsibleClinicalTrialDataSharing.pdf.
↵House of Commons Committee of Public Accounts. Access to clinical trial information and the stockpiling of Tamiflu. 2014. www.publications.parliament.uk/pa/cm201314/cmselect/cmpubacc/295/295.pdf.
http://www.bmj.com/content/348/bmj.g213?ijkey=5aXoYcMGOETixKf&keytype=ref
The House of Commons Public Accounts Committee delivered a remarkable report on 3 January. Its initial remit was the United Kingdom’s £424m (€510m; $697m) stockpile of oseltamivir (Tamiflu), but the committee soon broadened out—with evident surprise—into the ongoing problem of clinical trial results being routinely and legally withheld from doctors, researchers, and patients.
This situation has persisted for too long. The first quantitative evidence on publication bias was published in 1986.1 Iain Chalmers described in 2006 how progress in the 1990s soon deteriorated into broken promises.2 Recent years have seen extensive denial. The Association of the British Pharmaceutical Industry (ABPI) has claimed that these problems are historic, and that results are now posted on clinicaltrials.gov. The recently defunct Ethical Standards in Health and Life Sciences Group,3 which most UK medical and academic professional bodies signed up to, falsely claimed that a “robust regulatory framework” ensures access to trial results.4 US legislation requiring all results to be posted on clinicaltrials.gov within 12 months of completion has been widely ignored,5 with no enforcement. There has also been covert activity from industry—a leaked memo on its “advocacy” strategy included “mobilising patient groups” to campaign against transparency.6
Despite this, we have achieved considerable progress. The AllTrials.net campaign, started 12 months ago, calls for all trials on all uses of all currently prescribed treatments to be registered, with their full methods and results reported.7 It now has the support of most medical and academic professional bodies as well as the National Institute for Health and Care Excellence (NICE), Medical Research Council, Wellcome, more than 130 patient groups, 60 000 members of the public, and many in industry including GlaxoSmithKline. The Health Research Authority has announced that registration will be a condition of ethics committee approval.8 The BMA has passed a motion stating that withholding trial results is research misconduct,9 and the General Medical Council is re-examining its guidance on the matter.
There have also been extensive new proposals for greater transparency from European Union legislators, the European Medicines Agency (EMA),10 and industry bodies.11 All, however, share the same loophole—they all propose improved access to information on trials conducted from 2014 onwards. This means that almost all trials relevant to current medical practice would be exempt (including, for example, those on oseltamivir).
We now have an unprecedented opportunity for change, with considerable support from medical and academic professional bodies, policy makers, patient groups, and—importantly—the public. It’s time to consider what practical improvements can be made.
Firstly, by whatever means necessary, the methods and results of all previous trials must be accessible to the medical and academic community, which produces the guidelines and systematic reviews that inform patient care. It is commonly assumed that it would be difficult to enforce demands for trial results from diffuse global organisations, but we have never tried simply asking in an organised fashion. For example, the EMA could ask all research organisations and companies with a marketing authorisation for full methods and results of all trials they have conducted, so that these can be posted online, on the first ever register of trials that aspires to be a complete record of all research. If this invitation is declined, we could be told.
Secondly, while the current state of secrecy continues, there is much to be done with the most basic research tool in medicine—audit. Industry is quibbling over the precise proportion of trials that go undisclosed. This should not be a matter of debate. We need a trials observatory, covering all trials on all currently used treatments, that matches registry entries and other sources of information on completed trials against sources of results, whether those are in academic papers, clinical study reports, regulatory documents, or online postings. From these data we could derive live dashboards on transparency to drive up best practice, identify the best and worst companies for missing results, the treatments where most information is missing, the best and worst investigators, and more.
This is actionable information. If routine audit shows a particular principal investigator is performing badly, with many unreported results, should ethics committees grant them access to more trial participants? Will patients participate in trials for companies that withhold results? If two treatments have equivalent benefits, but one comes from a company with a track record of transparency and the other from a company that actively undermines the transparency campaign, are those two treatments still equivalent, and which should a cautious clinician prescribe?
One aspect of the committee’s report was missed by popular commentators, but it exemplifies the peculiarity of the current situation. Professor Kent Woods, head of the Medicines and Healthcare Products Regulatory Agency (MHRA), told the committee that European regulators had everything on oseltamivir. Evidence from the Cochrane Collaboration seems to show that this is not true. Cochrane asked the EMA for all the documents it held on oseltamivir, under that agency’s contested new transparency policy, and the agency complied—on several trials it held incomplete information on the methods and results, and for many more trials, it held nothing.12 The Public Accounts Committee expressed concern, and Professor Woods may wish to clarify this matter. But it is odd that there is any uncertainty about what evidence exists, or what the MHRA, EMA, and NICE have seen, on currently prescribed treatments (www.bmj.com/tamiflu).
It is also remarkable that the medical community needs a committee of generalist politicians to reflect these problems back to us. We spend millions on individual trials to exclude bias and often to detect subtle differences between treatments, but we let those biases pour back in unnecessarily when we permit whole trials to be withheld. Future generations may look back at our present tolerance of withheld trial results in the same way that we look back on medieval blood letting.
Notes
Cite this as: BMJ 2014;448:g213
Footnotes
Competing interests: We have read and understood the BMJ Group policy on declaration of interests and declare the following interests: BG is involved in a project to audit missing trial results and receives income from speaking and writing for lay audiences on problems in medicine, including publication bias. CH has received grant funding from the National Institute for Health Research Health Technology Assessment Programme to review the evidence for oseltamivir (Tamiflu) and zanamivir (Relenza). BG and CH are cofounders of the AllTrials campaign.
Provenance and peer review: Commissioned; not externally peer reviewed.
References
↵Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol1986;4:1529-41.Abstract/FREE Full Text
↵Chalmers I. From optimism to disillusion about commitment to transparency in the medico-industrial complex. J R Soc Med2006;99:337-41. FREE Full Text
↵Arie S. How a marriage with big pharma ended in divorce. BMJ2013;347:f6062.FREE Full Text
↵Goldacre B, Carroll D, Hall E. Guidelines for collaboration with industry should be transparent. BMJ. 2013;347:f6100.FREE Full Text
↵Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ. 2012;344:d7373.CrossRefMedline
↵Sample I. Big pharma mobilising patients in battle over drugs trials data. Guardian2013. www.theguardian.com/business/2013/jul/21/big-pharma-secret-drugs-trials.
↵AllTrials. What does all trials registered and reported mean? All Trials, 2013. www.alltrials.net/all-trials/.
↵Chalmers I. Health Research Authority’s great leap forward on UK trial registration. BMJ2013;347:f5776.FREE Full Text
↵BMA. Non-publication of trials results is research misconduct. http://ecancer.org/journal/news/4200-bma-----non-publication-of-trials-results-is-research-misconduct.php.
↵European Medicines Agency. Draft policy 70: Publication and access to clinical-trial data. 2013. www.ema.europa.eu/ema/doc_index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500144730&murl=menus/document_library/document_library.jsp&mid=0b01ac058009a3dc.
↵European Federation of Pharmaceutical Industries and Associations/PhRMA. Principles for responsible clinical trial data sharing. 2013. www.phrma.org/sites/default/files/pdf/PhRMAPrinciplesForResponsibleClinicalTrialDataSharing.pdf.
↵House of Commons Committee of Public Accounts. Access to clinical trial information and the stockpiling of Tamiflu. 2014. www.publications.parliament.uk/pa/cm201314/cmselect/cmpubacc/295/295.pdf.
http://www.bmj.com/content/348/bmj.g213?ijkey=5aXoYcMGOETixKf&keytype=ref
No comments:
Post a Comment