Tuesday, June 23, 2015

Ben Goldacre writes in the BMJ - How medicine is broken, and how we can fix it

The chief medical officer’s review on statins and oseltamivir may look for answers in the wrong places

Last week there was extensive news coverage of a leaked letter written by the chief medical officer to the Academy of Medical Sciences. This letter focused especially on concerns around statins and oseltamivir (Tamiflu) and asked the academy for an “authoritative independent report looking at how society should judge the safety and efficacy of drugs.”1 The academy has since announced that it is convening a working group on the subject.

With any such report there are two major risks. The first is a focus on “trust” or even—as a worst case—false reassurance for well documented problems. We do not believe the academy will choose this path. But there is another, bigger risk: the academy may accept shortcomings in the evidence as somehow inevitable. Here there are good grounds for concern. The academy has already announced that its work “will explore how evidence that originates from different sources (e.g. randomised clinical trials and observational data) are used to make decisions about the safety and efficacy of drugs and medical interventions.”2

Focusing solely on existing trials and observational studies would represent a failure of vision and ambition in an era when medicine has both the need and the opportunity to innovate. Well documented problems exist in the funding and prioritisation of research, the conduct of trials, the withholding of results, the dissemination of evidence, and its implementation with patients. Here we briefly examine six domains where the academy could call for simple practical improvements that would address legitimate concerns.

Publication bias—We conduct trials to detect modest differences, and spend vast amounts of money specifically to exclude bias, yet we allow that bias to flood back in through selective publication.3 4 Eminent bodies writing reports will not fix this, but practical action will. We need new funding for simple systematic work to audit which trials are unreported, to highlight the best and worst performers, and to shine a light on withheld studies.5

Independent trials—A recent cohort study found that 97% of head to head trials sponsored by industry give results that favour the sponsor’s drug.6 Doctors and patients are right to want independent trials. On statins and oseltamivir, there are two clear opportunities, and here we declare our own conflicts. With colleagues, one of us (CH) first proposed a trial of oseltamivir in a pandemic in 2009; the other (BG) first proposed a trial of statins examining side effects over a year ago. In both cases we could have the answer by now.

Cost of trials—Replication will be possible only if the cost of conducting trials is radically reduced. Much of this cost is driven by disproportionate regulation around trials of routinely used treatments.7 The National Institute for Health and Care Excellence’s guidance on cholesterol argues for head to head trials in low risk populations; this would require over 100 000 participants, followed up for a decade. Such trials can practically be delivered only by reducing the expensive and disproportionate regulatory burden,7 embedding them in everyday clinical care and gathering follow-up data from existing electronic health records.8

Better evidence—Treatments are routinely approved after trials with only surrogate outcomes.9 Drugs are then extensively promoted, at the moment of approval, when evidence on real world outcomes is paradoxically at its weakest. We could encourage better evidence by, for example, compelling companies to follow-up all phase III trial participants until real world benefits emerge, considering routine randomisation for newly approved drugs when benefits are unclear, and bartering with either patent extension or choice of the start date for market exclusivity. These suggestions would come at minimal cost and deliver more comprehensive data on treatment effects.

Shared decision making—Concern over statins has recently been reawakened by the introduction of a financial incentive for general practitioners to prescribe the drugs to low risk patients. This is ill judged because patients’ informed choices vary widely.10 11 An incentive to prescribe a treatment that many adequately informed patients do not want undermines informed decision making and inflicts avoidable reputational harm on the profession. If instead we incentivise shared decision making then—for the same financial outlay—best practice will be recognised, rewarded, and laid down in the everyday templates of what doctors do.12

Declare conflicts of interest—Declaration of conflicts of interest is currently chaotic, inconsistent, and incomplete. We clearly need a central system of declarations, ideally maintained by the General Medical Council.13 Conflicts, however, become particularly salient when evidence is unclear: when decisions about which treatment works best are made on the basis of a speculative, superficially plausible narrative about a drug’s mechanism of action, or on the interpretation of weak, confounded, observational data when randomised trials are feasible. If we are able to generate better evidence and ensure that we see the complete evidence, then competing interests—although they must always be declared—will become less salient.

This is just a brief tour of six domains, and there is much more to be done. Most of our suggestions are rapidly deliverable. Some require modest funding; most do not. Some require legislative changes. But we should remember that evidence based medicine, in its true modern incarnation, has a relatively short history and that when randomised trials were first introduced they were often regarded as a transgressive, expensive, unnecessary, and unwelcome challenge to medical authority.14 The public is increasingly aware of the shortcomings we collectively tolerate in the evidence base for clinical practice. We now have the opportunity to use public frustration as fuel to update our implementation of evidence based medicine in the light of new technology and get our house in order. To restrict a review of these problems to the interpretation of inadequate existing data—as the academy apparently proposes—would be recklessly backward looking.

Cite this as: BMJ 2015;350:h3397

Competing interests: We have read and understood BMJ policy on declaration of interests and declare BG and CH are founders of the AllTrials campaign calling for all trials to report their results. BG receives funding from the Wellcome Trust and the Laura and John Arnold Foundation and income from speaking, writing, and broadcasting on problems in science and medicine. CH has received funding from a UK National Institute for Health Research grant (HTA—10/80/01 Update and amalgamation of two Cochrane reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children www.nets.nihr.ac.uk/projects/hta/108001). CH also receives occasional payments for running educational courses at the University of Oxford (see www.phc.ox.ac.uk/team/researchers/carl-heneghan for full details).

Provenance and peer review: Commissioned; not externally peer reviewed.

↵Wise J. England’s chief medical officer asks for review of drug evaluation in wake of statins controversy. BMJ2015;350:h3300.FREE Full Text
↵Academy of Medical Sciences. Academy launches new project on evaluating evidence. Press release, 16 Jun 2015. www.acmedsci.ac.uk/more/news/evaluating-evidence-launch/.
↵Schmucker C, Schell LK, Portalupi S, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One2014;9:e114023.CrossRefMedline
↵Song F, Parekh S, Hooper L, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess2010;14(8).iii, ix-xi,1-193.
↵Goldacre B. How to get all trials reported: audit, better data, and individual accountability. PLoS Med2015;12:e1001821.CrossRefMedline
↵Flacco ME, Manzoli L, Boccia S, et al. Head-to-head randomized trials are mostly industry sponsored and almost always favor the industry sponsor. J Clin Epidemiol2015;68:811-20.CrossRefMedline
↵Van Staa T-P, Dyson L, McCann G, et al. The opportunities and challenges of pragmatic point-of-care randomised trials using routinely collected electronic records: evaluations of two exemplar trials. Health Technol Assess2014;18:1-146.CrossRef
↵Van Staa T-P, Goldacre B, Gulliford M, et al. Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ2012;344:e55.FREE Full Text
↵Yudkin JS, Lipska KJ, Montori VM. The idolatry of the surrogate. BMJ2011;343:d7995.FREE Full Text
↵Fontana M, Asaria P, Moraldo M, et al. Patient-accessible tool for shared decision making in cardiovascular primary prevention: balancing longevity benefits against medication disutility. Circulation2014;129:2539-46.Abstract/FREE Full Text
↵Van Staa T-P, Smeeth L, Ng ES-W, Goldacre B, Gulliford M. The efficiency of cardiovascular risk assessment: do the right patients get statin treatment? Heart Br Card Soc2013;99:1597-602.Abstract/FREE Full Text
↵Goldacre B, Smeeth L. Mass treatment with statins. BMJ2014;349:g4745.FREE Full Text
↵McCartney M, Goldacre B, Chalmers I, et al. Why the GMC should set up a central registry of doctors’ competing interests. BMJ2014;348:g236.FREE Full Text
↵Blythe M, Cochrane A. One man’s medicine: an autobiography of Professor Archie Cochrane. BMJ Publishing, 1989.

No comments: