LAUSR

To cite: Jefferson T. BMJ EvidenceBased Medicine Epub ahead of print: [please include Day Month Year]. doi:10.1136/ bmjebm-2020-111348 © Author(s) (or their employer(s)) 2020. No commercial reuse. See rights and permissions. Published by BMJ. This is the second in a series of notes addressing the need to change our evidence sources for assessing the effects of pharmaceuticals and biologics because of the danger of unrecognised reporting bias. In this note I shall discuss the possibility of using a shorter way of redefining the E than using clinical study reports. In the first note—‘Redefining the ‘E’ in EBM’— I made the point that trial publications (if they exist) are the tip of the data iceberg. The ‘iceberg’ is made up of a complex and massive reality of thousands of pages of clinical study reports, manuals, forms and slides of meetings invisible below the waterline. The submerged part is likely to represent anything between 1 and 8000 pages of published material. Even more importantly, the submerged material has the potential to change the conclusions of systematic reviews. To prove this point we have presented a list of references in our first editorial and at the Restoring Invisible and Abandoned Trials (RIAT) (https:// restoringtrials. org/ whatisriat/) initiative. I have another reason to be wary of the published written word, especially in prestigious journals. The reason is not new, but is not widely known. It is the refusal of some authors and editors to correct misreporting in their trials, even when presented with the evidence of misreporting. BMJEBM recently published the evidence of misreporting of an active ingredient as an ‘inert placebo’ in pivotal trials of human papillomavirus vaccines. Long before we embarked in the restoration, following RIAT rules, we contacted sponsors and authors of the misreported articles publicly inviting them to take action. We got no response. Once the restoration was complete we sent our manuscript to the same journal who had presided over the publication of three of the misreported trials. No action was taken. We encountered the same problem with the journals which published ghostwritten pivotal trials of the antiviral Tamiflu. In our initial note, we suggested addressing reporting bias by including regulatory data in our reviews and briefly mentioned “different types of regulatory data now on offer”, such as regulators’ reports. We first came across regulators’ reports in the preparation of our Cochrane review on neuraminidase inhibitors, but we were not the first to use them. Because clinical study reports had not yet been released to us, we accessed 2000 pages of letters, medical and statistical officer reports and minutes of meetings preparatory to the granting of the licence by the US Food and Drugs Administration (FDA). The documents (which are cited by file number and content in our review) provided a unique insight into the workings of regulators and their reviews of what we could not (yet) see: the thousands of pages of clinical study reports on both neuraminidase inhibitors, Relenza and Tamiflu. The FDA documents were extremely detailed and impressive kingsized peerreview reports. Unlike journal peerreview reports, they addressed several trials in the same development programme, and provided tables listing all the trials in the programme of interest to the regulator. The listing sometimes eliminated the problem of identification and (at times) publication bias altogether. These reports (all with different names) are now available from several sources and easily downloadable. Some are very difficult to follow but others are eminently readable and authoritative. Over time, FDA reports have become more readable, although there is a worry that recent proposal to conflate various reports might affect detailing.