How closely do the great and the good, for example, scrutinise the promotional material for medical drugs? The latest paper looking at this question is published this month.
Researchers in Holland went through the world's biggest medical journals – the Journal of the American Medical Association, Lancet, the New England Journal of Medicine, and so on – between 2003 and 2005. Adverts were included, once each, if they made a claim about the effect of a drug. For all the claims, they checked the references, found the trials referred to, and gave them out to easily exploited assessors: 250 medical students who'd just finished their evidence-based medicine teaching.
Each student independently assessed two trials, and associated adverts, following a questionnaire and a well-established scoring system to assess quality of trials. Scores were given for factors including:
• Whether the method of randomly assigning patients to one treatment or another was adequate, and clearly described.
• Whether patients could know which treatment they were getting.
• Whether drop-outs were appropriately included in the analysis, and so on.
These are good measures of whether a trial is a fair test of a treatment.
By now you will rightly be worrying that medical students – although cheap and easy to come by – are not reliable raters, so you will be pleased to hear that each trial was scored by between two and six students, and any discrepancy reviewed by a panel of four academics.
The results were abysmal. Only half of the claims in the adverts were supported by the specific trials referenced and, of all the trials, only 55% got a score of "high quality". Overall, only 39.2% of these adverts referenced a high-quality trial which supported their claim.
This is not the first time such a study has been conducted. Villanueva and colleagues, in 2003, published a paper in the Lancet assessing claims for cardiac medication adverts in six Spanish medical journals: of the 102 references they could trace, 44% did not support the promotional statement. Similar results have been found in psychiatric drug adverts, and in the field of rheumatology.
To offset any suggestion that I am cherry-picking, a review in the Public Library of Science's open access journal PLoS One found 24 similar studies, and overall only 67% of the claims in adverts were supported by a systematic review, a meta-analysis or a randomised control trial.More
No comments:
Post a Comment