Two weeks ago I described a flawed mortality study by National Cancer Institute staff that led to a correction but no recognition for our
group’s work in uncovering the errors. Here
we detail a second, similar case, involving a vaping study by faculty at
Columbia and the University of Arkansas, including a former member of the FDA
Tobacco Products Scientific Advisory Committee (TPSAC).
Emine Bircan and colleagues in April 2021 published a study,
“Electronic cigarette use and its association with asthma, chronic obstructive
pulmonary disease (COPD) and asthma- COPD overlap syndrome among never
cigarette smokers,” in the journal Tobacco Induced Diseases (here). It asserted that vaping “is associated with
increased odds of self-reported asthma, COPD, and [asthma-COPD overlap
syndrome, ACOS] among never combustible cigarette smokers. The odds of ACOS
were twice as high among e-cigarette users compared with never smokers of
conventional cigarettes.”
Coauthor Pebbles Fagan is a former member of the FDA TPSAC,
which is charged with evaluating quality of research relating to tobacco
company submissions.
As with Case 1, my colleague Nantaporn Plurphanswat and I
had worked with the same data, so we knew immediately that the article was flawed. I emailed senior author Mohammed Orloff on
April 22, citing numerous serious problems and noting that I was unable to
identify any formal channel or forum at the journal to communicate these
concerns. I copied the journal’s two
co-editors in chief on my email, which is here.
Following are some of the errors I detailed:
1. There were glaring numeric errors, such as 46079 instead
of 469077 in the abstract, and 46077 instead of 469077 on page 2. The authors also said that 624,387
participants had “missing data” when, in fact, those participants had a history
of cigarette smoking.
2. In the methods discussion, the authors incorrectly called
e-cigarette users cigarette smokers.
3. The authors at least 13 times used the term “e-cigarette
smokers”, but I informed Dr. Orloff that the “term is inaccurate and not in
standard use in the professional medical literature. E-cigarettes do not
generate smoke.”
4. The authors inaccurately claimed they had used a question
in the survey data, “Do you still have COPD?” But that question didn’t exist in the survey.
5. Their article’s graphic contained an element labeled
“E-cig Smokers [sic] (n=4,368)” and did not appear to be age-restricted.
However, Table 1 on page 5 had an entry “Smokers (N=4368)” and 18-24 years. I asked which was correct.
I ended by writing, “It is imperative that these mistakes
are corrected and additional information is provided before your results can be
viewed as valid.”
Dr. Orloff responded the same day: “I should say I feel very
embarrassed. Indeed this was sloppy on
our side and we will fix and connect with the Editors in Chief on how to move
forward.”
Over the next three weeks, Dr. Orloff and I exchanged emails
discussing specific problems with the study (here). The bottom line was that we were not able to
reconcile the number of participants reported by Bircan et al. with our
re-analysis of the same data.
My team then made a critical discovery: Bircan et al. had
deleted over 27,000 ever e-cigarette users without mentioning or explaining it
-- a cardinal error that could have affected their results.
Dr. Orloff’s response to these deletions was one of the most
bizarre in my 29 years in tobacco harm reduction (here),
as I explained in an email to him (here).
“…With all due respect, your response to the 20,000+
exclusions is post-hoc and illogical. First, nothing about your explanation was
presented in the methods or the flowchart. At the very least your article needs
revisions to make these choices and their justification crystal clear. You
talked about ‘former’ e-cigarette users, but you never used that term in the
manuscript. Even if you do that now, the definition will not be consistent in any way
with ‘former’ smokers, who are precisely defined.
“There is a more important consideration: three other
reasons you used revolve around ‘residual chemicals’ and misclassification,
which affects the validity of your study. I agree. You excluded 20,000+ participants because they
might have had a prior exposure to vapor, but you haven’t done the same thing
for a much more important and serious exposure: cigarette smoking. BRFSS
doesn’t have an ‘ever-smoked-a-cigarette’ question, so you have kept those
smokers in your analysis by default. In fact, if transient exposure to vapor is
as hazardous as you suggest, then any former exposure to cigarette smoke would
be magnitudes more hazardous. Furthermore, notwithstanding that you excluded
participants who smoked 100 lifetime cigarettes, it’s likely that current
e-cigarette users have that smoke exposure.
“Your only choice wasn’t to exclude ever users of e-cigs.
You could have run the same model with this group included to see if it changed the
results, or you could have run the model only using this group.
“I’ll close by making one final suggestion to review your
manuscript for passages like ‘since the age of those at risk in our study was 18-24 years.’
Attached please find an Excel chart showing the age distribution of the 156 cases of COPD among
current e-cig users in your study, compared with never, current and former smokers in
both BRFSS and NHIS 2016-2018. COPD cases
among current e-cig users (never smokers) in NHIS 2016-2018 are not in this chart because there were only n=3.”
As shown in the chart, national surveys, including the BRFSS
that Bircan used, demonstrate that 70% to 90% of COPD cases are seen in people
55+ years of age, regardless of smoking status.
Very few or no cases are seen in 18-24 year olds. Yet Bircan’s current e-cigarette users-with-COPD
column on the right indicates that over 50% of COPD cases occurred in those 34
and under, in total contrast to all the other survey groups. This is next to impossible.
Dr. Orloff stopped corresponding after my last email, and eventually
a “correction” was published. While the
authors cleaned up some obvious mistakes, they did not resolve important
problems like the deletion of 27,000 participants. The results didn’t change at all, which means
that they did not conduct a thorough re-analysis.
As with last week’s Case 1 (here),
the scientific value of this correction is seriously compromised, and the
authors and editors evaded a full accounting of their errors.
