The scientific publishing community is in love with
meta-analyses. Metas apply statistical techniques to combine results from
multiple studies with the aim of producing more precise estimates of specific
effects.
A Medline search reveals almost 215,000 metas in the medical
literature, with 75% of them published since 2015. Of the 9,488 Medline studies involving
e-cigarettes, 104 are metas.
I have written
about the tsunami of deficient studies on e-cigarettes and vaping fueled by
funding from the National Institutes of Health.
My colleague and I have spent considerable time over recent years
documenting the flawed and downright fraudulent findings in many of these. Unfortunately, we are now seeing an outbreak
of metas that compound the distorted results of those prior studies, resulting
in further obfuscation of the facts.
One such meta-analysis
is the work of retired professor Stanton Glantz and his colleagues, published
in a spin-off journal, the New England Journal of Medicine – Evidence. The article includes 181 references, 107 of
which were used as sources for their risk calculations, and it runs to 18
pages, with a supplementary appendix of 117 pages.
The main conclusion
reads: “…the odds of disease between current e-cigarette and cigarette use were
similar…There is a need to reassess the assumption that e-cigarette use
provides substantial harm reduction across all cigarette-caused diseases.”
In short,
“E-cigarettes = cigarettes = deadly = no harm reduction.”
Following publication, several
issues were raised in letters posted on the journal website (here), including the small number of e-cigarette
users and short duration of exposure, the lack of dose-response assessment, and
two serious methodologic flaws involving assessing bias and certainty of
evidence.
My colleague Nantaporn Plurphanswat and I had additional
concerns. We spent several months
dissecting the meta, and we had help from Jordan Rodu, a statistics professor
at the University of Virginia. Our
results have just been published (here). In our abstract we identified “three
principal deficits that were avoidable: (1) mixing unjustified and
incomprehensible disease outcomes; (2) using survey datasets containing no
temporal information about smoking/vaping initiation and disease diagnosis; (3)
using longitudinal studies that didn’t account for changes in vaping and
smoking during follow-up waves.” Let’s
take a closer look at these deficits.
We chose to focus on the Glantz meta results for
e-cigarettes and cardiovascular disease (CVD), stroke and chronic obstructive
pulmonary disease (COPD), because these are serious, and often fatal conditions. When we looked at the CVD results, we were
astounded to find that Glantz et al. included erectile dysfunction. Maybe ED is associated with smoking, and it
is medical problem, but there is no scientific rationale for including it in
this study, but for the fact that it contributed the largest, most significant
risk estimate to the CVD category.
The meta authors made a similar error with COPD: they
included a study of influenza, which isn’t remotely related.
Medical diagnoses are not suggestions that can be
categorized sloppily. They are organized
and structured by the World Health Organization in the International Classification of
Diseases, and they should be employed legitimately.
We also documented that Glantz and colleagues based their
results on numerous studies of cross-sectional datasets such as the National
Health Interview Surveys (NHIS) and the Behavioral Risk Factor Surveillance
System (BRFSS). We have previously published
research showing that these datasets contain no information about the age
at initiation of e-cigarette and cigarette use, nor on the age that
participants were diagnosed with diseases.
None of these studies should have been used in the Glantz meta.
The Glantz team’s results were partially derived from
longitudinal studies, mainly from the FDA’s Population Assessment of Tobacco
and Health (PATH) survey, which is appropriate.
However, the only study of this type that showed a positive result for
e-cigarettes and COPD was done by Xie et al. (here). We re-analyzed that entire study, and we
proved that the original results were almost entirely confounded by
smoking.
In summary, we demonstrated that the Glantz meta “failed to
meet a basic criterion, described by Egger et
al. as “Garbage in – garbage out?: The quality of component trials is of
crucial importance: if the ‘raw material’ is flawed, the findings of reviews of
this material may also be compromised.”
Our study concluded, “the results of the [Glantz et al.] meta-analysis
are invalid.”
