Bad science, bad reporting and will somebody think of the children!


We all know that smoking is bad, and marijuana smoking is bad, and alcohol is bad. And obviously obesity is bad and having a sedentary life style is bad. And eating your fruits and veggies is good. And we’re mulling the same, non-actionable information over and over. In the Globe and Mail and on CTV and on the Canadian Healthcare Network and of course the Huffington Post.

“It shows that over the four years of high school, the number of smokers went up 170 per cent, binge drinking jumped 167 per cent and marijuana use rose by 124 per cent.” … Also, they got taller [shocker]!

Is this research really adding anything to what we already know?

Here’s the ACTUAL graph from the article (something the Globe and Mail won’t show you):


1. The study does not and cannot show that the number of anything went up, or down or make any claims about the rate of change of anything. It’s a cross-sectional study for god’s sake! This graph only shows what the difference is between the different grades. It’s entirely possible that we’re doing a good job and those Grade 9 kids won’t smoke any more than they already do.

In order for them to show that the numbers “went up” they should have done a cohort study, where they followed Grade 9 students throughout highschool and found that more smoked by the end. But they didn’t do that.

2. What’s up with the relative percent reporting?

What they should have done is say 5 out of 100 Grade 9 students smoke, in comparison to 13 of 100 in Grade 12 [1]. Same goes for all the other “relative increases”.

Now, I want 0% of people to smoke obviously, because it’s bad for you. Causes respiratory diseases, cancers and all kinds of other nasty things (like impotence in males). But the question is, are we doing a good job of decreasing the number of smokers?

The Health Canada Youth Smoking Survey (incidentally done between the same years 2010-2011 as the study in question) reports 10% of Grade 10-12 students smoked (similar to this study, nothing new there) [2]. Here’s the caveat, the same survey showed that 13% of Grade 10-12 students smoked between 2008-2009 [2].

If we reported these numbers in the same “relative” percentages we could say that between ’08-09 and ’10-11 there was a 23% decrease in smokers among high school students!

So while, it’s not perfect, but we’re improving pretty quickly. In fact, if we maintain this “23%” reduction every 3 years or so, in a decade we’ll have almost no smokers.

“But Pharmer, you can’t extrapolate numbers like that!”… why not? They do it in national publications!

3. Other things are more worrisome.

Absolute numbers are VERY important here. 13/100 – 5/100 = 8/100 extra smokers in Grade 12 vs. Grade 9 [1]. BUT marijuana, while having a lower relative percent difference (123% vs. 170%) actually looks something like this in absolute terms: 26/100 – 12/100 = 14/100 [1]. Although according to Health Canada, this was a decrease from 2008-09 as well [2].

So they make tobacco look like a big problem, but we’re doing pretty good at reducing that. Why such a absolute jump for marijuana? Are students switching from tobacco to marijuana? Are we sending conflicting public health messages? How do students perceive the risks of smoking, alcohol and drugs? Can a well designed argument flip students to the dark side? [3]

4. Alcohol use.

Same thing as with smoking. Relatively speaking the percentages went up but according to Health Canada, that’s actually a decrease compared to how many were binge drinking in 2008-09 (it actually went down from 33% on average to 39% on average) [2].

Also, “binge drinking” is defined as “5 or more drinks on a single occasions in the past 5 months” [1,2]. While this is fine as an operational definition for public health (and these kids shouldn’t be drinking anything anyway, because they are under the legal age), 5 drinks is hardly “binge drinking” in terms of the real world. It’s certainly “social” drinking but few people will be trashed after 5 drinks. It’s also bizarre to me that 2 drinks/day is OK according to Health Canada, but 5 drinks on one occasion every 5 months, and you’re a binge drinker?

5. I’m not against getting students to stop smoking/drinking/using drugs. I’m against bad science and bad reporting. Both this study and Health Canada reported similar statistics and these come from the same place – the School for Public Health at the University of Waterloo, so why are they painting a different picture? Is the message “we’re doing good, but we need to continue improving”? That’s fine, but why not conduct a well designed cohort study and actually track how many students will take up smoking/binge drinking/what-have-you?

How about adjusting for bias? The Health Canada website reports extremely overlapping ranges between the different grade levels [4].

6. It’s all political

Healthcare budgets are getting thin and everyone wants a piece of the pie. Obviously it’s more dramatic to report on increasing numbers than on the successes. The way forward should never be based on misrepresenting reality, after all such data manipulation gets pharmaceuticals withdrawn from the market (case-in-point Vioxx).

Just because getting students to quit smoking is a noble goal, doesn’t mean that white lies are OK. Stop publishing incomplete and poorly interpreted data in popular news outlets!


1. ALeatherdale ST, Rynard V. cross-sectional examination of modifiable risk factors for chronic disease among a nationally representative sample of youth: are Canadian students graduating high school with a failing grade for health? BMC Public Health. 2013 Jun 11;13(1):569. [Epub ahead of print].


3. Anna V. Song, PhD, Holly E. R. Morrell, PhD, Jodi L. Cornell, MSW, MA, Malena E. Ramos, EdM, Michael Biehl, MA, Rhonda Y. Kropp, MPH, and Bonnie L. Halpern-Felsher, PhD. Perceptions of Smoking-Related Risks and Benefits as Predictors of Adolescent Smoking Initiation Am J Public Health. 2009 March; 99(3): 487–492. doi: 10.2105/AJPH.2008.137679



Tags: , , , , , ,

Trackbacks / Pingbacks

  1. Google “breasts” to find good science | Science Pharmer - July 17, 2013

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: