Monday, September 21, 2009

Magnifying Smokeless Tobacco Risks By Cherry Picking Data: A Case Study

A month ago, I discussed in this blog the definitive study of cancer risks among smokeless tobacco users. This comprehensive meta-analysis, which produced a series of summary relative risks for cancer among smokeless tobacco users by systematically combining all of the primary epidemiologic studies, was completed by Peter Lee and Jan Hamling. They concluded that there is very little evidence that smokeless tobacco use is associated with any cancer.

The results in the Lee-Hamling study were surprisingly different from an earlier meta-analysis performed by Paolo Boffetta, an epidemiologist at the International Agency for Research on Cancer, and co-authors, which was published in the journal Lancet Oncology. The Boffetta team had produced relative risk estimates for cancers of the oral cavity and pharynx, esophagus and pancreas that were considerably higher than those from Lee and Hamling, as illustrated in the following table:

Cancer SiteBoffetta Relative Risk Estimate (95% CI) Lee-Hamling Relative Risk Estimate (95% CI)
Oral cavity/pharynx1.8 (1.1 – 2.9)1.36 (1.04 – 1.77)
Esophagus1.6 (1.1 – 2.3)1.13 (0.95 – 1.36)
Pancreas1.6 (1.1 – 2.2)0.99 (0.71 – 1.60)

Why did the Boffetta meta-analysis always produce higher risk estimates than the Lee-Hamling study if both were using essentially the same data? A commentary published by Lee and Hamling in the journal BMC Cancer provides the answer to that question. The answer is simple and disturbing.

First, let me give you some background information. Most primary epidemiologic studies report results for many different sets of circumstances. For example, a study about smokeless tobacco might report risk estimates for all users and these subgroups: current users, former users, exclusive users who never smoked, and users who also smoked. So it is important for investigators conducting meta-analyses to develop specific rules for combining estimates from different studies. Lee and Hamling developed rules and followed them systematically in their meta-analyses. In contrast, Boffetta did not specify any rules but appears to have followed one: Use only the highest risk estimate.

The Lee/Hamling commentary focused on the inherent contradictions in Boffetta’s approach, the best example of which is illustrated in the table below. There are two important primary epidemiologic studies on pancreatic cancer among smokeless tobacco users, one published in the Lancet and the other in the International Journal of Cancer. Each study provided risk estimates for exclusive smokeless users (i.e., those who never smoked) and for users who might also have smoked, as seen in this table:

Primary Study Journal, YearRisk Among Exclusive Users (95% CI)Risk Among Users +/- Smoking
Lancet, 20071.8 (1.1 – 2.9)0.9 (0.7 – 1.2)
Int J Cancer, 20050.85 (0.24 – 3.07)1.67 (1.12 – 2.50)

Lee and Hamling make it clear that they produced their summary estimates by either combining risks for exclusive users (i.e. 1.8 and 0.85), or risks for users who might have smoked (0.9 and 1.67). This is both logical and scientifically valid. But which estimates did Boffetta use? The highest, of course. For the Lancet study he used the estimate of 1.8 in exclusive users, but for the International Journal of Cancer study he used the estimate of 1.67 in smokeless users who might have smoked. His goal was to maximize the health risks from smokeless tobacco use, thereby feeding scaremongers who portray smokeless tobacco as deadly, despite decades of research to the contrary. The scientific term for this epidemiologic method is cherry picking.

This is just one example of the unscientific manner in which the Boffetta meta-analysis was conducted. The Lee/Hamling commentary describes many others.

According to its website, the International Agency for Research on Cancer places emphasis on “elucidating the role of environmental and lifestyle risk factors” in cancer development. The IARC should not condone the fabrication of estimates misrepresenting the scientific evidence that cancer risks from smokeless tobacco use in Sweden and the U.S. are minimal to nonexistent. Kudos to Peter Lee and Jan Hamling for documenting this travesty.

No comments: