A long time ago, following work on the Restoration of Study 329, I was approached by Peter Goetzsche, featured above, to work on restoring the two trials that had got Prozac (fluoxetine) approved for children for depression.
I may have suggested this idea to Peter in that from 2004 or so I was aware that reviewers in the US FDA had concluded that although they were going to approve Prozac for children there were grounds to think the trials had not shown it worked. Despite mentioning this fact widely, nobody had ever decided to look into it and challenge the mantra that we know Fluoxetine works for children.
A decade later Peter got stuck into this problem – more and more so as the obstacles to getting anything done accumulated.
At the same time, Jon Jureidini, also a veteran of the Study 329 Restoration, picked up the Prozac Treatment of Adolescent Depression Study (TADS) for Restoration.
Between them they were told that Lilly were not going to provide any data, or found that Graham Emslie the notional ‘investigator’ for the pivotal Prozac trials wasn’t going to reply to requests for data on these trials or were told that Duke University, where the TADS data were lodged, had destroyed it. See Evidence Based Medicine in a Post-Truth World.
Anyone who knows Peter will know that putting obstacles in his way is a bad idea. They will also know he has a keen sense of moral outrage and says the things that many of us figure we should be saying but don’t. Many of us wonder how he can get away with saying some of these things.
So it will come as no surprise to anyone to learn that the first few drafts of an article with Peter and me as authors, that finally appeared years later, had to be edited, pruned, hosed down with ice and generally steered toward an acceptably scientific tone. There is of course no such thing as an acceptably scientific tone – this is code for alleviating a journal’s concern that they might get sued.
The tone was definitely not completely muted and I gave Reviewer 1# some leeway in this regard, expecting his/her comments might be more forgiving with the revision.
When writing Study 329 we were aiming for a manuscript as Flat as Kansas. Kansas is technically flatter than a Pancake. There is no way to get Peter that flat. The best that can be hoped for is to approach Pancake flatness. But Reviewer 1#s tone was just as incensed, if not more so, by the second semi-pancake version compared to the Colorado first version.
We offered Ralph Edwards, the editor of the Journal of Risk and Safety in Medicine, where the article has been published, the option to publish Reviewer 1# comments along with the article. Ralph deserves a lot of credit for steering a path through this mess.
Some weeks ago RxISK ran a post on Narinder Banzal’s article on Antidepressants and Premature Death, mentioning that it can take great skill to get an article like hers published in a mainstream journal. Some readers were very surprised. They expected the review process to be sacred. They expected scientists to be able to speak plainly and tell the truth.
I mentioned in the comments that we would soon be able to offer a post showing something of what can really go on behind the scenes. This post is that post. These are the Colorado reviews – with their Himalayan first review.
Manuscript Number: JRS-210034 Full Title:
Restoring the two pivotal fluoxetine trials in children and adolescents with depression
Reviewer’s opinion: Reject
Summary: The authors have attempted to review two trials which they consider to be misreported. The overarching aim remains to discredit the use of fluoxetine as an antidepressant
This manuscript presents very important work regarding the safety of fluoxetine in children and adolescents and its findings should be disseminated to the scientific/ medical community. However, as it stands, this manuscript needs significant revisions in order to communicate its findings more effectively in an unbiased, systematic and thorough manner. It may be useful to engage an external party to critically review your writing, I tried to write some detailed comments for consideration.
This manuscript is not strictly a trial restoration. The authors have conducted a review of contrasting reports between the CSR and publications. It remains unclear what parts are the CSR/authors/ new authors. This may be perceived as a bias. The parts of the manuscript that are presented as a critical appraisal should be kept separate to the analysis.
While I have prepared a number of comments that should be addressed, I think this work is critical to the field of child psychiatry and this paper should certainly be published when it is refined. I hope that these comments are helpful rather than discouraging to the authors.
*Specific comments* (these are not exhaustive, but may be useful as examples to illustrate points made by general comments above)
3. Abstract, objective- specifics need to be added. What framework did you use for reanalysis? State your specific aims, e.g. benefit/ harms/ change in depression score… to reassure/ reassess safety? Mentions of trial misreporting should occur in the background and they should be specified. For the results, was information missing from both trial CSRs? The conclusion needs more detail here.
4. Introduction: first paragraph, explain the context of the trials/ where reports came from so that people can follow up; second paragraph, second sentence about adults needs a reference; second paragraph, last sentence CI reporting needs to be fixed.
5. Methods: was Emslie PI for both trials?
6. Methods Second paragraph, last sentence ‘paid attention to’ is not scientific language/ not specific, same with ‘dropped a plan’.
7. Methods Third paragraph last sentence why was it not necessary to present terms to a third party?
8. What did you use fisher’s tests for?
9. Overall, methods need to frame other aspects of your reanalysis like framework/ grounds for critical review and how your results are reported/ structured seeing as it’s not conventional (also see feedback below), for example: trial results are presented separately regarding each CSR. Our critical appraisal of each study’s protocol and whether it was conducted by the trial is summarised… etc
10. The results section needs to be restructured and refined significantly. It is hard to follow which parts are critiques of the protocol/ CSR/ literature versus your actual findings. I suggest keeping these separate (although it is tempting to present them together as they relate to the same issue, this should be saved for the discussion).
11. S3.1 reference for the fact that Lilly’s analysis plan was written a posteriori
12. 3.1.2 third paragraph regarding origin of trial drugs- what were the different origins? The pharmacy’s Prozac/ lactose vs lilly’s pills?
13. 3.1.4 first sentence unclear, not sure how this favours fluoxetine? Could you explain this? Is it based on their analysis
14. Page 6 second sentence Use of unscientific language- ‘larger’
15. Page 6 paragraph 2 first sentence needs reference
16. Table 1 it’s unclear which are yours/ lilly’s p-values in the table
17. Page 6 text under table 1, ‘lumped’ is unscientific
18. Page 7, second paragraph- no significant differences in what?
19. 3.1.5 discontinuations should be a separate section
20. Table 2 why are discontinuations presented weekly? What does this add? Maybe just present an overall total
21. Page 7, text under table 2. Are the adverse events predisposing a suicide attempt your conclusions/ descriptions or the CSRs or Emslie’s? There needs to be more clear delineation between your interpretation of their findings and your findings from the CSR.
22. Descriptions of adverse events appear as a list. These would be better presented as a summary and/ or tabulated or added to the appendix (if you are including lots of detail).
23. In some cases, the use of emotive language can appear biased/ accusatory- e.g. Emslie downplaying suicide attempts. It makes more of an impact to state what they said and explain why it was wrong or at least qualify the language
24. There are parts of the results that either belong in a critical appraisal section of the results or should be moved to the discussion for extrapolation, e.g. terms like ‘which suggests Lilly’s analysis… page 8 paragraph 3.
25. 3.1.6 is this your p-value? What test did you use? Is it appropriate for the sample size.
26. As far as you can ascertain, did trial HCJE use an active or inactive placebo?
27. Did either study report on compliance? How was it measured?
28. The reporting of each trial should be consistent. There is more detail about HCJE’s statistics. I understand the level of detail may be different between the reports but please clarify. Perhaps this will become more clear once the manuscript is refined/ restructured
I have refrained from collating additional detailed comments about the results thus far, but am happy to provide more detailed comment -IF the major issues from (10) are addressed.
I am hesitant to make detailed comment on conclusions made by the discussion, pending review of the results being presented- but have made some general points:
29. The section on comments from the results could accommodate concerns I stated in points 10, 21,24
30. Comments on the results- important arguments are made here by the authors. However, the flow is disjointed, they might consider grouping common themes of their reflections and/ or adding linking sentences. As it stands, this reads like a series of dot points critiquing the CSRs/ Lilly. This section should focus on the authors’ findings, with other criticisms moved to a separate/ new section?
31. Comments on the context- fix the subheading numbering. Presents a good summary of the history of fluoxetine in children which further highlights the importance of this work to the reader.
32. Again, the use of emotional/ alarmist language should be minimised. As a scientific paper, use of terms like ‘public health emergency’ are extreme. The use of such language suggests a certain level of bias from authors/ readers may lose confidence in the quality of their findings.
A window with the view on the details of how drug approval proceeds. Very informative and helpful for understanding mechanisms of drugs (and vaccines!!) launching on the market.
To be continued – later this week
RxISK acknowledges that the experiences of those who have been harmed by medical treatments are the cornerstone on which it is built, and believes this should be the case for all of medicine.
See Black Robe, White Coat for more detail on this people acknowledgement