The scientific publishing community is in love with meta-analyses. Metas apply statistical techniques to combine results from multiple studies with the aim of producing more precise estimates of specific effects.
A Medline search reveals almost 215,000 metas in the medical literature, with 75% of them published since 2015. Of the 9,488 Medline studies involving e-cigarettes, 104 are metas.
I have written about the tsunami of deficient studies on e-cigarettes and vaping fueled by funding from the National Institutes of Health. My colleague and I have spent considerable time over recent years documenting the flawed and downright fraudulent findings in many of these. Unfortunately, we are now seeing an outbreak of metas that compound the distorted results of those prior studies, resulting in further obfuscation of the facts.
One such meta-analysis is the work of retired professor Stanton Glantz and his colleagues, published in a spin-off journal, the New England Journal of Medicine – Evidence. The article includes 181 references, 107 of which were used as sources for their risk calculations, and it runs to 18 pages, with a supplementary appendix of 117 pages.
The main conclusion reads: “…the odds of disease between current e-cigarette and cigarette use were similar…There is a need to reassess the assumption that e-cigarette use provides substantial harm reduction across all cigarette-caused diseases.”
In short, “E-cigarettes = cigarettes = deadly = no harm reduction.”
Following publication, several issues were raised in letters posted on the journal website (here), including the small number of e-cigarette users and short duration of exposure, the lack of dose-response assessment, and two serious methodologic flaws involving assessing bias and certainty of evidence.
My colleague Nantaporn Plurphanswat and I had additional concerns. We spent several months dissecting the meta, and we had help from Jordan Rodu, a statistics professor at the University of Virginia. Our results have just been published (here). In our abstract we identified “three principal deficits that were avoidable: (1) mixing unjustified and incomprehensible disease outcomes; (2) using survey datasets containing no temporal information about smoking/vaping initiation and disease diagnosis; (3) using longitudinal studies that didn’t account for changes in vaping and smoking during follow-up waves.” Let’s take a closer look at these deficits.
We chose to focus on the Glantz meta results for e-cigarettes and cardiovascular disease (CVD), stroke and chronic obstructive pulmonary disease (COPD), because these are serious, and often fatal conditions. When we looked at the CVD results, we were astounded to find that Glantz et al. included erectile dysfunction. Maybe ED is associated with smoking, and it is medical problem, but there is no scientific rationale for including it in this study, but for the fact that it contributed the largest, most significant risk estimate to the CVD category.
The meta authors made a similar error with COPD: they included a study of influenza, which isn’t remotely related.
Medical diagnoses are not suggestions that can be categorized sloppily. They are organized and structured by the World Health Organization in the International Classification of Diseases, and they should be employed legitimately.
We also documented that Glantz and colleagues based their results on numerous studies of cross-sectional datasets such as the National Health Interview Surveys (NHIS) and the Behavioral Risk Factor Surveillance System (BRFSS). We have previously published research showing that these datasets contain no information about the age at initiation of e-cigarette and cigarette use, nor on the age that participants were diagnosed with diseases. None of these studies should have been used in the Glantz meta.
The Glantz team’s results were partially derived from longitudinal studies, mainly from the FDA’s Population Assessment of Tobacco and Health (PATH) survey, which is appropriate. However, the only study of this type that showed a positive result for e-cigarettes and COPD was done by Xie et al. (here). We re-analyzed that entire study, and we proved that the original results were almost entirely confounded by smoking.
In summary, we demonstrated that the Glantz meta “failed to meet a basic criterion, described by Egger et al. as “Garbage in – garbage out?: The quality of component trials is of crucial importance: if the ‘raw material’ is flawed, the findings of reviews of this material may also be compromised.”
Our study concluded, “the results of the [Glantz et al.] meta-analysis are invalid.”
