Exposing Defective Research, But Denied Credit for the Effort: Case 2

 

Two weeks ago I described a flawed mortality study by National Cancer Institute staff that led to a correction but no recognition for our group’s work in uncovering the errors.  Here we detail a second, similar case, involving a vaping study by faculty at Columbia and the University of Arkansas, including a former member of the FDA Tobacco Products Scientific Advisory Committee (TPSAC).

Emine Bircan and colleagues in April 2021 published a study, “Electronic cigarette use and its association with asthma, chronic obstructive pulmonary disease (COPD) and asthma- COPD overlap syndrome among never cigarette smokers,” in the journal Tobacco Induced Diseases (here).  It asserted that vaping “is associated with increased odds of self-reported asthma, COPD, and [asthma-COPD overlap syndrome, ACOS] among never combustible cigarette smokers. The odds of ACOS were twice as high among e-cigarette users compared with never smokers of conventional cigarettes.” 

Coauthor Pebbles Fagan is a former member of the FDA TPSAC, which is charged with evaluating quality of research relating to tobacco company submissions.

As with Case 1, my colleague Nantaporn Plurphanswat and I had worked with the same data, so we knew immediately that the article was flawed.  I emailed senior author Mohammed Orloff on April 22, citing numerous serious problems and noting that I was unable to identify any formal channel or forum at the journal to communicate these concerns.  I copied the journal’s two co-editors in chief on my email, which is here. 

Following are some of the errors I detailed:

1. There were glaring numeric errors, such as 46079 instead of 469077 in the abstract, and 46077 instead of 469077 on page 2.  The authors also said that 624,387 participants had “missing data” when, in fact, those participants had a history of cigarette smoking. 

2. In the methods discussion, the authors incorrectly called e-cigarette users cigarette smokers.

3. The authors at least 13 times used the term “e-cigarette smokers”, but I informed Dr. Orloff that the “term is inaccurate and not in standard use in the professional medical literature. E-cigarettes do not generate smoke.”

4. The authors inaccurately claimed they had used a question in the survey data, “Do you still have COPD?”  But that question didn’t exist in the survey.

5. Their article’s graphic contained an element labeled “E-cig Smokers [sic] (n=4,368)” and did not appear to be age-restricted. However, Table 1 on page 5 had an entry “Smokers (N=4368)” and 18-24 years.  I asked which was correct.

I ended by writing, “It is imperative that these mistakes are corrected and additional information is provided before your results can be viewed as valid.”

Dr. Orloff responded the same day: “I should say I feel very embarrassed.  Indeed this was sloppy on our side and we will fix and connect with the Editors in Chief on how to move forward.”

Over the next three weeks, Dr. Orloff and I exchanged emails discussing specific problems with the study (here).  The bottom line was that we were not able to reconcile the number of participants reported by Bircan et al. with our re-analysis of the same data.   

My team then made a critical discovery: Bircan et al. had deleted over 27,000 ever e-cigarette users without mentioning or explaining it -- a cardinal error that could have affected their results.

Dr. Orloff’s response to these deletions was one of the most bizarre in my 29 years in tobacco harm reduction (here), as I explained in an email to him (here).

“…With all due respect, your response to the 20,000+ exclusions is post-hoc and illogical. First, nothing about your explanation was presented in the methods or the flowchart. At the very least your article needs revisions to make these choices and their justification crystal clear. You talked about ‘former’ e-cigarette users, but you never used that term in the manuscript. Even if you do that now, the definition will not be consistent in any way with ‘former’ smokers, who are precisely defined.

“There is a more important consideration: three other reasons you used revolve around ‘residual chemicals’ and misclassification, which affects the validity of your study. I agree.  You excluded 20,000+ participants because they might have had a prior exposure to vapor, but you haven’t done the same thing for a much more important and serious exposure: cigarette smoking. BRFSS doesn’t have an ‘ever-smoked-a-cigarette’ question, so you have kept those smokers in your analysis by default. In fact, if transient exposure to vapor is as hazardous as you suggest, then any former exposure to cigarette smoke would be magnitudes more hazardous. Furthermore, notwithstanding that you excluded participants who smoked 100 lifetime cigarettes, it’s likely that current e-cigarette users have that smoke exposure.

“Your only choice wasn’t to exclude ever users of e-cigs. You could have run the same model with this group included to see if it changed the results, or you could have run the model only using this group.

“I’ll close by making one final suggestion to review your manuscript for passages like ‘since the age of those at risk in our study was 18-24 years.’ Attached please find an Excel chart showing the age distribution of the 156 cases of COPD among current e-cig users in your study, compared with never, current and former smokers in both BRFSS and NHIS 2016-2018.  COPD cases among current e-cig users (never smokers) in NHIS 2016-2018 are not in this chart because there were only n=3.”

As shown in the chart, national surveys, including the BRFSS that Bircan used, demonstrate that 70% to 90% of COPD cases are seen in people 55+ years of age, regardless of smoking status.  Very few or no cases are seen in 18-24 year olds.  Yet Bircan’s current e-cigarette users-with-COPD column on the right indicates that over 50% of COPD cases occurred in those 34 and under, in total contrast to all the other survey groups.  This is next to impossible.

Dr. Orloff stopped corresponding after my last email, and eventually a “correction” was published.  While the authors cleaned up some obvious mistakes, they did not resolve important problems like the deletion of 27,000 participants.  The results didn’t change at all, which means that they did not conduct a thorough re-analysis. 

As with last week’s Case 1 (here), the scientific value of this correction is seriously compromised, and the authors and editors evaded a full accounting of their errors. 

 

 

How NIH Funding Undermines Tobacco Harm Reduction

 

Have you ever considered the recent years’ tsunami of research that emphasizes the dangers of e-cigarette and vaping products?  Was this work spontaneously generated?  How was it funded? 

Philip Cole, Elizabeth Delzell and I answered these questions in a manuscript published in 2000 in the journal Epidemiology (available here).  We described how after World War II, universities “changed their mission and became research arms of the federal government…But, with considerable money comes influence and eventually some control. The funds had been flowing for little more than a decade when President Eisenhower, in his 1961 farewell address to the nation, warned universities against accepting encumbered federal funds. He saw that the giving hand already had become the heavy hand. One by one, ever more onerous conditions were attached to federal awards. This has continued to the point that many universities now harbor research programs, sometimes entire institutes, that function essentially as satellites of the federal government.”

The U.S. National Institutes of Health is the primary funding source for researchers at universities countrywide.  In 2018, NIH provided 54,000 awards to 2,900 institutions -- nearly $40 billion in all. 

Research awards have two components.  The first is “direct costs,” which covers actual research.  These funds are managed by the researchers and include salaries for faculty and staff, equipment and other necessary expenses. 

The second component is “indirect costs,” covering things not directly related to the research – items like buildings, utilities and administrative expenses.  These items are managed by university administrators.  For every $100 in direct costs that a researcher gets from the NIH for their project, university administrators take $55-70 for indirect costs.  That is why we wrote that “…the need [for indirect costs] is so great that, perversely, Moneychangers [i.e., administrators] now prefer encumbered research grants, which offset overhead costs, to unencumbered gifts which do not but which do give faculty members freedom to pursue their interests.”

When Uncle Sam gives universities money, there’s always a catch, which we described in our article: “Federal bureaucrats fuel the fires of growth by urging academics to do research consistent with their agency’s mission. Through master agreements, cooperative agreements, requests for proposals and inducements of every sort, professors are led into doing the government’s research.”

It gets worse.  Money is so scarce that “Many universities now are over committed to…the aggrandizement of research, and some have gone so far that research fundraising has become more important than research itself.”

  

This brings us to tobacco research.  As I have noted, the government’s stated mission is “to create a world free of tobacco use.”  In fiscal year 2020, the NIH disbursed 1,489 awards identifying the key word “tobacco” to 303 unique organizations.  The grants totaled $693 million.  Over one-quarter of this money ($186 million) was devoted to research on e-cigarettes and vaping products.  Note in the following chart how increases in this funding have driven the number of research publications.

 

In summary, NIH is the most critical source of support for all American researchers, universities and other academic institutions.  Researchers and administrators recognize the value, if not imperative, of comporting with the government’s tobacco-termination mission.  Despite this oppressive influence, researchers routinely declare that they have no conflicts of interest or bias.

The steady flow of research reports demonizing e-cigarettes and vapor products provides “scientific” evidence to support harsh regulation.  The Tobacco Products Scientific Advisory Committee (TPSAC) that advises the FDA on regulations is currently comprised of seven members who together received $25 million from NIH in fiscal year 2020, and $54 million in other years.  

This blog post was developed from a presentation I gave on “How Institutional Funding in the U.S. Works Against Tobacco Harm Reduction” at the 2021 Global Forum on Nicotine Conference.  The recorded video is available here.  

Note added July 16, 2021: 

I was notified by @phil_w888 via Twitter that the Canadian equivalent of the NIH, the Canadian Institutes of Health Research, announced funding for a project last year on the Health Effects of Vaping.  Incredibly, the description contained the following: 

"Research projects exploring the use of vaping as a smoking cessation tool or as a harm reduction strategy are not eligible for this funding opportunity."