Exposing Defective Research, But Denied Credit for the Effort: Case 2

 

Two weeks ago I described a flawed mortality study by National Cancer Institute staff that led to a correction but no recognition for our group’s work in uncovering the errors.  Here we detail a second, similar case, involving a vaping study by faculty at Columbia and the University of Arkansas, including a former member of the FDA Tobacco Products Scientific Advisory Committee (TPSAC).

Emine Bircan and colleagues in April 2021 published a study, “Electronic cigarette use and its association with asthma, chronic obstructive pulmonary disease (COPD) and asthma- COPD overlap syndrome among never cigarette smokers,” in the journal Tobacco Induced Diseases (here).  It asserted that vaping “is associated with increased odds of self-reported asthma, COPD, and [asthma-COPD overlap syndrome, ACOS] among never combustible cigarette smokers. The odds of ACOS were twice as high among e-cigarette users compared with never smokers of conventional cigarettes.” 

Coauthor Pebbles Fagan is a former member of the FDA TPSAC, which is charged with evaluating quality of research relating to tobacco company submissions.

As with Case 1, my colleague Nantaporn Plurphanswat and I had worked with the same data, so we knew immediately that the article was flawed.  I emailed senior author Mohammed Orloff on April 22, citing numerous serious problems and noting that I was unable to identify any formal channel or forum at the journal to communicate these concerns.  I copied the journal’s two co-editors in chief on my email, which is here. 

Following are some of the errors I detailed:

1. There were glaring numeric errors, such as 46079 instead of 469077 in the abstract, and 46077 instead of 469077 on page 2.  The authors also said that 624,387 participants had “missing data” when, in fact, those participants had a history of cigarette smoking. 

2. In the methods discussion, the authors incorrectly called e-cigarette users cigarette smokers.

3. The authors at least 13 times used the term “e-cigarette smokers”, but I informed Dr. Orloff that the “term is inaccurate and not in standard use in the professional medical literature. E-cigarettes do not generate smoke.”

4. The authors inaccurately claimed they had used a question in the survey data, “Do you still have COPD?”  But that question didn’t exist in the survey.

5. Their article’s graphic contained an element labeled “E-cig Smokers [sic] (n=4,368)” and did not appear to be age-restricted. However, Table 1 on page 5 had an entry “Smokers (N=4368)” and 18-24 years.  I asked which was correct.

I ended by writing, “It is imperative that these mistakes are corrected and additional information is provided before your results can be viewed as valid.”

Dr. Orloff responded the same day: “I should say I feel very embarrassed.  Indeed this was sloppy on our side and we will fix and connect with the Editors in Chief on how to move forward.”

Over the next three weeks, Dr. Orloff and I exchanged emails discussing specific problems with the study (here).  The bottom line was that we were not able to reconcile the number of participants reported by Bircan et al. with our re-analysis of the same data.   

My team then made a critical discovery: Bircan et al. had deleted over 27,000 ever e-cigarette users without mentioning or explaining it -- a cardinal error that could have affected their results.

Dr. Orloff’s response to these deletions was one of the most bizarre in my 29 years in tobacco harm reduction (here), as I explained in an email to him (here).

“…With all due respect, your response to the 20,000+ exclusions is post-hoc and illogical. First, nothing about your explanation was presented in the methods or the flowchart. At the very least your article needs revisions to make these choices and their justification crystal clear. You talked about ‘former’ e-cigarette users, but you never used that term in the manuscript. Even if you do that now, the definition will not be consistent in any way with ‘former’ smokers, who are precisely defined.

“There is a more important consideration: three other reasons you used revolve around ‘residual chemicals’ and misclassification, which affects the validity of your study. I agree.  You excluded 20,000+ participants because they might have had a prior exposure to vapor, but you haven’t done the same thing for a much more important and serious exposure: cigarette smoking. BRFSS doesn’t have an ‘ever-smoked-a-cigarette’ question, so you have kept those smokers in your analysis by default. In fact, if transient exposure to vapor is as hazardous as you suggest, then any former exposure to cigarette smoke would be magnitudes more hazardous. Furthermore, notwithstanding that you excluded participants who smoked 100 lifetime cigarettes, it’s likely that current e-cigarette users have that smoke exposure.

“Your only choice wasn’t to exclude ever users of e-cigs. You could have run the same model with this group included to see if it changed the results, or you could have run the model only using this group.

“I’ll close by making one final suggestion to review your manuscript for passages like ‘since the age of those at risk in our study was 18-24 years.’ Attached please find an Excel chart showing the age distribution of the 156 cases of COPD among current e-cig users in your study, compared with never, current and former smokers in both BRFSS and NHIS 2016-2018.  COPD cases among current e-cig users (never smokers) in NHIS 2016-2018 are not in this chart because there were only n=3.”

As shown in the chart, national surveys, including the BRFSS that Bircan used, demonstrate that 70% to 90% of COPD cases are seen in people 55+ years of age, regardless of smoking status.  Very few or no cases are seen in 18-24 year olds.  Yet Bircan’s current e-cigarette users-with-COPD column on the right indicates that over 50% of COPD cases occurred in those 34 and under, in total contrast to all the other survey groups.  This is next to impossible.

Dr. Orloff stopped corresponding after my last email, and eventually a “correction” was published.  While the authors cleaned up some obvious mistakes, they did not resolve important problems like the deletion of 27,000 participants.  The results didn’t change at all, which means that they did not conduct a thorough re-analysis. 

As with last week’s Case 1 (here), the scientific value of this correction is seriously compromised, and the authors and editors evaded a full accounting of their errors. 

 

 

Deficient and Unreliable Studies of Vaping and Disease

 

At least a dozen medical studies over the past few years have found that vaping is associated with heart attack, stroke, hypertension, emphysema and other lung disease, as well as diabetes (see the list at the end of this entry).  The studies are cross-sectional; they are drawn from population samples at one point in time.  They only produce associations, but those are then inflated to imply or outright claim that vaping causes those disease conditions.  The studies appeared in respected medical journals and, together, form a powerful disincentive for smokers to switch to vaping.  They also add fuel to the regulatory fire to suppress vape products, which are actually vastly safer than cigarettes. 

My research group has published a new analysis entitled, “Cross‑sectional e‑cigarette studies are unreliable without timing of exposure and disease diagnosis,” in the journal Internal and Emergency Medicine.

Our analysis shows that the dozen are irrevocably deficient and unreliable.  Some used data from National Health Interview Surveys (NHIS), the Behavioral Risk Factor Surveillance System (BRFSS), or the FDA’s Population Assessment of Tobacco and Health (PATH) surveys, all of which profile participants who smoked and/or vaped and who had ever been diagnosed with numerous diseases.  However, there is absolutely no information in the NHIS or BRFSS about when participants started to smoke or vape, or when they were first diagnosed with a disease. Absent this critical data, it is impossible to demonstrate that smoking or vaping preceded the disease, and therefore had any role in causing it.

While authors of some of the studies tried to use a statistical sleight of hand by claiming that their results were “adjusted” for smoking, that is not a legitimate work-around for the missing cause-and-effect information.

Some studies used PATH data, which does contain all of the information needed to analyze whether smoking and vaping preceded the diseases and therefore might have caused them.  Readers of my blog know that my research group in 2019 used PATH data to demonstrate that a heart attack study published in the flagship journal of the American Heart Association was pure fiction, resulting in its retraction by journal editors.  Now we have analyzed PATH data to demonstrate that none of the vaping-caused-disease claims in the recent studies are valid.

Our design was simple: We started by determining at what age participants were diagnosed with smoking-attributable diseases – chronic obstructive pulmonary disease (COPD), emphysema, myocardial infarction (MI, or heart attack) and stroke.  We then noted when participants started smoking or vaping.  The results were crystal clear.  As we reported in our study, “Only 4% of COPD and emphysema cases, 6% of MIs and 11% of stroke had occurred in PATH participants who had used e-cigarettes prior to the diagnoses. In contrast, over 94% of all cases occurred after smoking cigarettes fairly regularly.” 

In summary, the cited studies’ claims that vaping is related to or causes airway and circulatory diseases are not valid, as the authors lacked or ignored causative pathway information.  As for those few cases in which vaping preceded the disease, we note in our article, this may indicate “a potential reverse association between e-cigarette use and these diseases. In other words, having a diagnosis for respiratory and cardiovascular diseases leads smokers to use e-cigarettes.”

 

   Deficient and Unreliable Studies of Vaping and Disease

   Alzahrani T, Pena I, Temesgen N, Glantz SA (2018).  Association between electronic cigarette use and myocardial infarction.  Am J Prev Med 55(4):455-461. https://pubmed.ncbi.nlm.nih.gov/30166079/   

Vindhyal MR, Okut H, Ablah E, Ndunda PM, Kallail KJ, Choi WS (2020).  Cardiovascular outcomes associated with adult electronic cigarette use.  Cureus 12(8): e9618.  https://pubmed.ncbi.nlm.nih.gov/32923219/ 

Wills TA, Pagano I, Williams RJ, Tam EK (2019).  E-cigarette use and respiratory disorder in an adult sample.  Drug Alc Depend 194:363-370.  https://doi.org/10.1016/j.drugalcdep.2018.10.004 

Osei AD, Mirbolouk M, Orimoloye OA, et al (2019).  Association between e-cigarette use and cardiovascular disease among never and current combustible-cigarette smokers.  Am J Med 132:949-954.  https://doi.org/10.1016/j.amjmed.2019.02.016 

Parekh T, Pemmasani S, Desai R (2020).  Risk of stroke with e-cigarette and combustible cigarette use in young adults.  Am J Prev Med 58(3):446-452.

   Bircan E, Bezirhan U, Porter A, Fagan P, Orloff MS (2021).  Electronic cigarette use and its association with asthma, chronic obstructive pulmonary disease (COPD) and asthma-COPD overlap syndrome among never cigarette smokers.  Tob Ind Dis 19:23. https://doi.org/10.18332/tid/132833 

   Wills TA, Choi K, Pokhrel P, Pagano I (2022).  Tests for confounding with cigarette smoking in the association of e-cigarette use with respiratory disorder: 2020 national-sample data. Prev Med  https://doi.org/10.1016/j.ypmed.2022.107137

Perez MF, Atuegwu N, Mead E, Oncken C, Mortensen EM (2018).  E-cigarette use is associated with emphysema, chronic bronchitis and COPD.  Am J Respir Crit Care Med 197:A6245.  

Zhang Z, Jiao Z, Blaha MJ, et al (2022).  The association between e-cigarette use and prediabetes: results from the Behavioral Risk Factor Surveillance System, 2016-2018.  Am J Prev Med 62(6):872-877.  https://doi.org/10.1016/j.amepre.2021.12.009 

Atuegwu NC, Perez MF, Oncken C, Mead EL, Maheshwari N, Mortensen EM (2019).  E-cigarette use is associated with a self-reported diagnosis of prediabetes in never cigarette smokers: results from the Behavioral Risk Factor Surveillance System Survey.  Drug Alc Depend 205: 107692. doi:10.1016/j.drugalcdep.2019.10769

Miller CR, Shi H, Li D, Goniewicz ML (2021).  Cross-sectional associations of smoking and e-cigarette use with self-reported diagnosed hypertension: findings from Wave 3 of the Population Assessment of Tobacco and Health Study.  Toxics 9:52 https://doi.org/10.3390/toxics9030052 

Osei AD, Mirbolouk M, Orimoloye OA et al. Association between e-cigarette use and chronic obstructive pulmonary disease by smoking status: Behavioral Risk Factor Surveillance System 2016 and 2017.  Am J Prev Med 58, ISSUE 3, P336-342, March 01, 2020.  https://doi.org/10.1016/j.amepre.2019.10.014  

Osei AD, Mirbolouk M, Orimoloye, OA et al. The association between e-cigarette use and asthma among never combustible cigarette smokers: Behavioral Risk Factor Surveillance System (BRFSS) 2016 & 2017. BMC Pulm Med 19, 180 (2019). https://doi.org/10.1186/s12890-019-0950-3

More Bad Behavior from Journal Authors & Editors on A Fatally Flawed Study of Vaping & COPD

Following is another example of junk science published by anti-tobacco crusaders, abetted by negligent journal editors.

The journal Tobacco Induced Diseases on April 7 published an article by Emine Bircan and colleagues at the University of Arkansas.  They claimed that e-cigarette use “is associated with increased odds of self-reported asthma, COPD, and [asthma-COPD overlap syndrome] among never combustible cigarette smokers.” In their abstract, the authors assert that “the age of those at risk in our study was 18–24 years.” 

First, take a look at the chart.  It is axiomatic that COPD is mostly seen in current and former smokers age 55+ years.  Now look at Bircan’s vapers in the last column: the distribution is completely upside down! This led me to examine the article in greater detail, whereupon I discovered numerous obvious errors.

Since the journal prohibits submissions from industry-funded scientists (my research is supported by unrestricted grants to the University of Louisville from tobacco companies), the only avenue to attempt to correct the errors was to write the authors directly.  I exchanged numerous emails with corresponding author Mohammed Orloff and journal editor James Scott over the period April 22 – May 12. Copies of this correspondence are available here.

In addition to raising the issue of young adults with COPD, I listed these significant errors:

1. There were two obvious wrong numbers of never cigarette smokers.

2. The inaccurate and non-standard term “e-cigarette smokers” was used throughout the article.

3. The authors referred to a question (“Do you still have COPD?”) that did not appear in the analyzed surveys.

4. The authors reversed two numbers in a flow chart describing important exclusions.

5. The authors failed to define “missing variables”.

These egregious errors should have been discovered and corrected by the authors, and by the journal’s reviewers and editors. Among the five authors is Pebbles Fagan, Ph.D., a former member of the FDA Tobacco Products Scientific Advisory Committee.

Dr. Orloff responded to my letter: “I should say I feel very embarrassed. Indeed this was sloppy on our side and we will fix and connect with the Editors in Chief on how to move forward.”

Still, the mess wasn’t immediately resolved. I downloaded the datasets from the U.S. government’s Behavioral Risk Factor Surveillance System (BRFSS).  But I was unable to reproduce the article’s study groups, the first basic step of any serious review. When Dr. Orloff disclosed further details of the analysis, I discovered that the authors had deleted 27,000 participants from the BRFSS data who had “ever” used an e-cigarette – even just one time – but did not use them at all at the time of the survey. This critical deletion was made without mention or explanation anywhere in the article. I advised Dr. Orloff that this large cohort should be classified as non-users of e-cigarettes, and the analysis should be re-run.

Dr. Orloff ended the correspondence shortly thereafter, around May 12.

Fast forward to the week of October 12, when the journal published a completely new version of the article, accompanied by the following statement: “The authors would like to express their apologies and regret for the errors in the original published version of the abovementioned article. The corrected article follows.”

The authors corrected their obvious errors, but failed to correct or add any reference to the 27,000 excluded ever vapers. Instead, they added the following brand new text to the discussion section to explain it away, as if it had been part of their research plan:

“Additionally, participants who reported having used e-cigarettes in their lifetime and currently do not use e-cigarettes (defined as former e-cigarette users) were excluded from the study population because of lack of clarity of exposure status and residual effects that have resulted from the e-cigarette derived chemical exposure which contribute to changes in the continuum of the disease progression leading to molecular changes detectable by biomarker analysis rather than reports. These former e-cigarette users have used e-cigarettes sometime in their entire life and have quit for different reasons. Studies have shown that while cessation of tobacco use diminishes the risk of experiencing long-term adverse health effects, past history of tobacco use is still associated with increased risk of lung diseases compared to never having smoked [citation here to a 1997 article ‘Smoking, alcohol consumption, and leukocyte counts.’]”

This explanation makes no sense. It is not valid to define someone who ever used an e-cigarette – even one time -- as a former user. The authors, post-hoc, unsuccessfully defended tossing out 27,000 ever vapers by inventing a “chemical exposure…leading to molecular changes… sometime in their entire life.”  But they kept in their study people who smoked up to 100 cigarettes – a far more toxic product. 

While Tobacco Induced Diseases inappropriately deleted the original article from their website, it is still available at the U.S. National Library of Medicine (here). I have also placed a copy of that original work here.

If the authors had adhered to professional standards, they would have acknowledged my major contributions in their corrective comments, but they did not.

I subsequently learned that Riccardo Polosa, a University of Catania (Italy) physician specializing in respiratory diseases, submitted a letter to the editor of Tobacco Induced Diseases, strongly questioning the validity of respiratory diseases in 18- to 24-year-old e-cigarette users. His letter was rejected.