Two weeks ago I described a flawed mortality study by National Cancer Institute staff that led to a correction but no recognition for our group’s work in uncovering the errors. Here we detail a second, similar case, involving a vaping study by faculty at Columbia and the University of Arkansas, including a former member of the FDA Tobacco Products Scientific Advisory Committee (TPSAC).
Emine Bircan and colleagues in April 2021 published a study, “Electronic cigarette use and its association with asthma, chronic obstructive pulmonary disease (COPD) and asthma- COPD overlap syndrome among never cigarette smokers,” in the journal Tobacco Induced Diseases (here). It asserted that vaping “is associated with increased odds of self-reported asthma, COPD, and [asthma-COPD overlap syndrome, ACOS] among never combustible cigarette smokers. The odds of ACOS were twice as high among e-cigarette users compared with never smokers of conventional cigarettes.”
Coauthor Pebbles Fagan is a former member of the FDA TPSAC, which is charged with evaluating quality of research relating to tobacco company submissions.
As with Case 1, my colleague Nantaporn Plurphanswat and I had worked with the same data, so we knew immediately that the article was flawed. I emailed senior author Mohammed Orloff on April 22, citing numerous serious problems and noting that I was unable to identify any formal channel or forum at the journal to communicate these concerns. I copied the journal’s two co-editors in chief on my email, which is here.
Following are some of the errors I detailed:
1. There were glaring numeric errors, such as 46079 instead of 469077 in the abstract, and 46077 instead of 469077 on page 2. The authors also said that 624,387 participants had “missing data” when, in fact, those participants had a history of cigarette smoking.
2. In the methods discussion, the authors incorrectly called e-cigarette users cigarette smokers.
3. The authors at least 13 times used the term “e-cigarette smokers”, but I informed Dr. Orloff that the “term is inaccurate and not in standard use in the professional medical literature. E-cigarettes do not generate smoke.”
4. The authors inaccurately claimed they had used a question in the survey data, “Do you still have COPD?” But that question didn’t exist in the survey.
5. Their article’s graphic contained an element labeled “E-cig Smokers [sic] (n=4,368)” and did not appear to be age-restricted. However, Table 1 on page 5 had an entry “Smokers (N=4368)” and 18-24 years. I asked which was correct.
I ended by writing, “It is imperative that these mistakes are corrected and additional information is provided before your results can be viewed as valid.”
Dr. Orloff responded the same day: “I should say I feel very embarrassed. Indeed this was sloppy on our side and we will fix and connect with the Editors in Chief on how to move forward.”
Over the next three weeks, Dr. Orloff and I exchanged emails discussing specific problems with the study (here). The bottom line was that we were not able to reconcile the number of participants reported by Bircan et al. with our re-analysis of the same data.
My team then made a critical discovery: Bircan et al. had deleted over 27,000 ever e-cigarette users without mentioning or explaining it -- a cardinal error that could have affected their results.
Dr. Orloff’s response to these deletions was one of the most bizarre in my 29 years in tobacco harm reduction (here), as I explained in an email to him (here).
“…With all due respect, your response to the 20,000+ exclusions is post-hoc and illogical. First, nothing about your explanation was presented in the methods or the flowchart. At the very least your article needs revisions to make these choices and their justification crystal clear. You talked about ‘former’ e-cigarette users, but you never used that term in the manuscript. Even if you do that now, the definition will not be consistent in any way with ‘former’ smokers, who are precisely defined.
“There is a more important consideration: three other reasons you used revolve around ‘residual chemicals’ and misclassification, which affects the validity of your study. I agree. You excluded 20,000+ participants because they might have had a prior exposure to vapor, but you haven’t done the same thing for a much more important and serious exposure: cigarette smoking. BRFSS doesn’t have an ‘ever-smoked-a-cigarette’ question, so you have kept those smokers in your analysis by default. In fact, if transient exposure to vapor is as hazardous as you suggest, then any former exposure to cigarette smoke would be magnitudes more hazardous. Furthermore, notwithstanding that you excluded participants who smoked 100 lifetime cigarettes, it’s likely that current e-cigarette users have that smoke exposure.
“Your only choice wasn’t to exclude ever users of e-cigs. You could have run the same model with this group included to see if it changed the results, or you could have run the model only using this group.
“I’ll close by making one final suggestion to review your manuscript for passages like ‘since the age of those at risk in our study was 18-24 years.’ Attached please find an Excel chart showing the age distribution of the 156 cases of COPD among current e-cig users in your study, compared with never, current and former smokers in both BRFSS and NHIS 2016-2018. COPD cases among current e-cig users (never smokers) in NHIS 2016-2018 are not in this chart because there were only n=3.”
As shown in the chart, national surveys, including the BRFSS that Bircan used, demonstrate that 70% to 90% of COPD cases are seen in people 55+ years of age, regardless of smoking status. Very few or no cases are seen in 18-24 year olds. Yet Bircan’s current e-cigarette users-with-COPD column on the right indicates that over 50% of COPD cases occurred in those 34 and under, in total contrast to all the other survey groups. This is next to impossible.
Dr. Orloff stopped corresponding after my last email, and eventually a “correction” was published. While the authors cleaned up some obvious mistakes, they did not resolve important problems like the deletion of 27,000 participants. The results didn’t change at all, which means that they did not conduct a thorough re-analysis.
As with last week’s Case 1 (here), the scientific value of this correction is seriously compromised, and the authors and editors evaded a full accounting of their errors.
No comments:
Post a Comment