Science, Kansas and Pancakes

A long time ago, following work on the Restoration of Study 329, I was approached by Peter Goetzsche, featured above, to work on restoring the two trials that had got Prozac (fluoxetine) approved for children for depression.

I may have suggested this idea to Peter in that from 2004 or so I was aware that reviewers in the US FDA had concluded that although they were going to approve Prozac for children there were grounds to think the trials had not shown it worked.  Despite mentioning this fact widely, nobody had ever decided to look into it and challenge the mantra that we know Fluoxetine works for children.

A decade later Peter got stuck into this problem – more and more so as the obstacles to getting anything done accumulated.

At the same time, Jon Jureidini, also a veteran of the Study 329 Restoration,  picked up the Prozac Treatment of Adolescent Depression Study (TADS) for Restoration.

Between them they were told that Lilly were not going to provide any data, or found that Graham Emslie the notional ‘investigator’ for the pivotal Prozac trials wasn’t going to reply to requests for data on these trials or were told that Duke University, where the TADS data were lodged, had destroyed it.  See Evidence Based Medicine in a Post-Truth World.

Anyone who knows Peter will know that putting obstacles in his way is a bad idea.  They will also know he has a keen sense of moral outrage and says the things that many of us figure we should be saying but don’t.  Many of us wonder how he can get away with saying some of these things.

So it will come as no surprise to anyone to learn that the first few drafts of an article with Peter and me as authors, that finally appeared years later, had to be edited, pruned, hosed down with ice and generally steered toward an acceptably scientific tone.  There is of course no such thing as an acceptably scientific tone – this is code for alleviating a journal’s concern that they might get sued.

The tone was definitely not completely muted and I gave Reviewer 1# some leeway in this regard, expecting his/her comments might be more forgiving with the revision.

When writing Study 329 we were aiming for a manuscript as Flat as Kansas.  Kansas is technically flatter than a Pancake.  There is no way to get Peter that flat.  The best that can be hoped for is to approach Pancake flatness.  But Reviewer 1#s tone was just as incensed, if not more so,  by the second semi-pancake version compared to the Colorado first version.

We offered Ralph Edwards, the editor of the Journal of Risk and Safety in Medicine, where the article has been published, the option to publish Reviewer 1# comments along with the article. Ralph deserves a lot of credit for steering a path through this mess.

Some weeks ago RxISK ran a post on Narinder Banzal’s article on Antidepressants and Premature Death, mentioning that it can take great skill to get an article like hers published in a mainstream journal.  Some readers were very surprised. They expected the review process to be sacred.  They expected scientists to be able to speak plainly and tell the truth.

I mentioned in the comments that we would soon be able to offer a post showing something of what can really go on behind the scenes.  This post is that post.  These are the Colorado reviews – with their Himalayan first review.

Reviewers’ comments:

Manuscript Number: JRS-210034 Full Title:
Restoring the two pivotal fluoxetine trials in children and adolescents with depression

Reviewer #1:

Reviewer’s opinion: Reject

Summary: The authors have attempted to review two trials which they consider to be misreported. The overarching aim remains to discredit the use of fluoxetine as an antidepressant

Major points:

1. It is worrisome to see the biased approach the authors pursue here and is unfortunate. In the abstract itself, they seem to lose their scientific focus and by the time they reach their conclusion, they have resorted to sensationalization. Scientific literature, as the authors will themselves know the best, is not a vindication.
2. Eli Lilly and Graham Emslie did not respond as to the authors’ expectation, which seems to have further bolstered their misconception of evil hidden behind the trials and their statistics. This has made their approach bitter, reflecting the same in this manuscript.
3. The authors have used their skills and judgements to restore and review the trial data, and focus their wrath towards fluoxetine. Their narrative in this manuscript would not have changed if “fluoxetine” was replaced by an evil character from history or fiction. Fluoxetine is the villain, hiding many deaths. The authors have both misplaced and displaced their dissent. If at all anything is to be blamed, it should be how RCTs are conducted and how people can find fault in almost any RCT. We should accept that the data need not be perfect for the drug to “work”. The ultimate aim is to help those with mental illnesses and psychological difficulties. This seems to be replaced by statistical analysis and data digging. This is a shame.
4. Fluoxetine has helped numerous individuals, and the black box warning by FDA on increased suicidal risk may itself be questioned. Again, this should be to understand more, and not to discredit. The authors are knowledgeable but seem to have missed literature like:
   *       Fornaro M, Anastasia A, Valchera A, Carano A, Orsolini L, Vellante F, Rapini G, Olivieri L, Di Natale S, Perna G, Martinotti G, Di Giannantonio M, De Berardis D. The FDA “Black Box” Warning on Antidepressant Suicide Risk in Young Adults: More Harm Than Benefits? Front Psychiatry. 2019 May 3;10:294. doi: 10.3389/fpsyt.2019.00294. PMID: 31130881; PMCID: PMC6510161.
   *       Beasley CM Jr, Dornseif BE, Bosomworth JC, Sayler ME, Rampey AH Jr, Heiligenstein JH, Thompson VL, Murphy DJ, Masica DN. Fluoxetine and suicide: a meta-analysis of controlled trials of treatment for depression. BMJ. 1991 Sep 21;303(6804):685-92. doi: 10.1136/bmj.303.6804.685. Erratum in: BMJ 1991 Oct 19;303(6808):968. PMID: 1833012; PMCID: PMC1670974.
   *       Gibbons RD, Brown CH, Hur K, Davis JM, Mann JJ. Suicidal Thoughts and Behavior With Antidepressant Treatment: Reanalysis of the Randomized Placebo-Controlled Studies of Fluoxetine and Venlafaxine. Arch Gen Psychiatry. 2012;69(6):580-587. doi:10.1001/archgenpsychiatry.2011.2048
   *       Pompili M, Serafini G, Innamorati M, Ambrosi E, Giordano G, Girardi P, Tatarelli R, Lester D. Antidepressants and Suicide Risk: A Comprehensive Overview. Pharmaceuticals (Basel). 2010 Aug 30;3(9):2861-2883. doi: 10.3390/ph3092861. PMID: 27713380; PMCID: PMC4034101.
5. I have absolutely no doubt that the authors will find missing data and statistical fault in these studies too, but I will have only regret if they do so.
6. It is equally surprising and sad to see such learned authors write such a manuscript for an educational journal, which specifically evaluates the safety of drugs. There is always a balance that needs to be understood, or else, data digging and fault-finding can be done better by computers and artificially intelligent beings. I sincerely request the authors to understand what mental illness is, what psychiatric pharmacology is and what it claims to be. It is not an antibiotic with a narrow and large spectrum. Neither is mental illness an infection. Suicidality fluctuates in populations across the world, we have had suicidal thoughts ourselves too. It is not an all or none phenomenon, unlike fever.
7. I understand that I have not used educational narrative in this review myself, and I do this deliberately, to reflect what the authors have done to seemingly every sentence that was published in these two trials. Looking back, I am glad that the authors were not the reviewers of these trials, because that would have resulted in humongous tragedy to depression and maybe a miniscule achievement in “how to find fault”.
8. As for those who criticise others’ references, the authors’ have done themselves poorly. For instance, Rosenbaum et al. (ref number 22) do not indicate that “The company already knew from a prior study that abrupt withdrawal of antidepressants could cause abstinence depressions in many patients”.
9. The manuscript can only be published outside academic journals to continue the farce of what fluoxetine can do and cannot. The authors may find more acceptance in that, and further help in building resistance towards mental illness and acceptance of these by the society. I hope the authors find trials that show that antidepressants work. I hope they find peace. Maybe they need to understand they have not “restored” much, other than mistrust in those who read their publications.
10. I shall give just one piece of data for them to ponder over, a self-citation rate of about 30% in this manuscript. They should reflect on why is this high level of self-citation required. Is their work benefiting others, or only themselves?

Reviewer #2:

This manuscript presents very important work regarding the safety of fluoxetine in children and adolescents and its findings should be disseminated to the scientific/ medical community. However, as it stands, this manuscript needs significant revisions in order to communicate its findings more effectively in an unbiased, systematic and thorough manner. It may be useful to engage an external party to critically review your writing, I tried to write some detailed comments for consideration.

This manuscript is not strictly a trial restoration. The authors have conducted a review of contrasting reports between the CSR and publications. It remains unclear what parts are the CSR/authors/ new authors. This may be perceived as a bias. The parts of the manuscript that are presented as a critical appraisal should be kept separate to the analysis.

While I have prepared a number of comments that should be addressed, I think this work is critical to the field of child psychiatry and this paper should certainly be published when it is refined. I hope that these comments are helpful rather than discouraging to the authors.

*General comments*

1. A thorough review of the manuscript’s language in general needs to be conducted, including grammar, the use of emotive language, imprecise statements and unscientific language.
2.  The format of reporting does not follow appropriate conventions and is hard to follow (particularly in the results section). I would suggest the use of CONSORT/RIAT guidelines for reporting trial reanalyse

*Specific comments* (these are not exhaustive, but may be useful as examples to illustrate points made by general comments above)

3.      Abstract, objective- specifics need to be added. What framework did you use for reanalysis?  State your specific aims, e.g. benefit/ harms/ change in depression score… to reassure/ reassess safety? Mentions of trial misreporting should occur in the background and they should be specified. For the results, was information missing from both trial CSRs? The conclusion needs more detail here.
4.      Introduction: first paragraph, explain the context of the trials/ where reports came from so that people can follow up; second paragraph, second sentence about adults needs a reference; second paragraph, last sentence CI reporting needs to be fixed.
5.      Methods: was Emslie PI for both trials?
6.      Methods Second paragraph, last sentence ‘paid attention to’ is not scientific language/ not specific, same with ‘dropped a plan’.
7.      Methods Third paragraph last sentence why was it not necessary to present terms to a third party?
8.      What did you use fisher’s tests for?
9.      Overall, methods need to frame other aspects of your reanalysis like framework/ grounds for critical review and how your results are reported/ structured seeing as it’s not conventional (also see feedback below), for example: trial results are presented separately regarding each CSR. Our critical appraisal of each study’s protocol and whether it was conducted by the trial is summarised… etc

*Results*
10.     The results section needs to be restructured and refined significantly. It is hard to follow which parts are critiques of the protocol/ CSR/ literature versus your actual findings. I suggest keeping these separate (although it is tempting to present them together as they relate to the same issue, this should be saved for the discussion).
11.     S3.1 reference for the fact that Lilly’s analysis plan was written a posteriori
12.     3.1.2 third paragraph regarding origin of trial drugs- what were the different origins? The pharmacy’s Prozac/ lactose vs lilly’s pills?
13.     3.1.4 first sentence unclear, not sure how this favours fluoxetine? Could you explain this? Is it based on their analysis
14.     Page 6 second sentence Use of unscientific language- ‘larger’
15.     Page 6 paragraph 2 first sentence needs reference
16.     Table 1 it’s unclear which are yours/ lilly’s p-values in the table
17.     Page 6 text under table 1, ‘lumped’ is unscientific
18.     Page 7, second paragraph- no significant differences in what?
19.     3.1.5 discontinuations should be a separate section
20.     Table 2 why are discontinuations presented weekly? What does this add? Maybe just present an overall total
21.     Page 7, text under table 2. Are the adverse events predisposing a suicide attempt your conclusions/ descriptions or the CSRs or Emslie’s? There needs to be more clear delineation between your interpretation of their findings and your findings from the CSR.
22.     Descriptions of adverse events appear as a list. These would be better presented as a summary and/ or tabulated or added to the appendix (if you are including lots of detail).
23.     In some cases, the use of emotive language can appear biased/ accusatory- e.g. Emslie downplaying suicide attempts. It makes more of an impact to state what they said and explain why it was wrong or at least qualify the language
24.     There are parts of the results that either belong in a critical appraisal section of the results or should be moved to the discussion for extrapolation, e.g. terms like ‘which suggests Lilly’s analysis… page 8 paragraph 3.
25.     3.1.6 is this your p-value? What test did you use? Is it appropriate for the sample size.
26.     As far as you can ascertain, did trial HCJE use an active or inactive placebo?
27.     Did either study report on compliance? How was it measured?
28.     The reporting of each trial should be consistent. There is more detail about HCJE’s statistics. I understand the level of detail may be different between the reports but please clarify. Perhaps this will become more clear once the manuscript is refined/ restructured

I have refrained from collating additional detailed comments about the results thus far, but am happy to provide more detailed comment -IF the major issues from (10) are addressed.

*Discussion*
I am hesitant to make detailed comment on conclusions made by the discussion, pending review of the results being presented- but have made some general points:
29.     The section on comments from the results could accommodate concerns I stated in points 10, 21,24
30.     Comments on the results- important arguments are made here by the authors. However, the flow is disjointed, they might consider grouping common themes of their reflections and/ or adding linking sentences. As it stands, this reads like a series of dot points critiquing the CSRs/ Lilly. This section should focus on the authors’ findings, with other criticisms moved to a separate/ new section?
31.     Comments on the context- fix the subheading numbering. Presents a good summary of the history of fluoxetine in children which further highlights the importance of this work to the reader.
32.     Again, the use of emotional/ alarmist language should be minimised. As a scientific paper, use of terms like ‘public health emergency’ are extreme. The use of such language suggests a certain level of bias from authors/ readers may lose confidence in the quality of their findings.

Reviewer #3:

A window with the view on the details of how drug approval proceeds. Very informative and helpful for understanding mechanisms of drugs (and vaccines!!) launching on the market.

To be continued – later this week

People Acknowledgement

RxISK acknowledges that the experiences of those who have been harmed by medical treatments are the cornerstone on which it is built, and believes this should be the case for all of medicine. 

See Black Robe, White Coat for more detail on this people acknowledgement