Skeptophilia (skep-to-fil-i-a) (n.) - the love of logical thought, skepticism, and thinking critically. Being an exploration of the applications of skeptical thinking to the world at large, with periodic excursions into linguistics, music, politics, cryptozoology, and why people keep seeing the face of Jesus on grilled cheese sandwiches.

Monday, May 8, 2017

Study shows readers of Skeptophilia have above-average intelligence!

One rather frustrating tendency, amongst those of us who have a skeptical bent, is that people tend to believe anything they read if it begins with "Study Proves" or "Research Shows."

Even better if it says "Harvard Researchers Show."

Apparently, it doesn't matter much whether the study actually proved the claim, or who the researchers were, or if the "Harvard research" was peer reviewed.  Merely claiming that some scientist, somewhere, of whatever credentials, said something -- well, that's enough.

Especially if what the scientist allegedly said fits in with what you already believed.

I ran across a particularly good example of this in Natural News, which I will not provide a link to because Mike "Health Ranger" Adams definitely doesn't need unsuspecting people generating any more clicks or ad revenue.  Least of all if they come from here, of all places.  The headline was "Study Shows Unvaccinated Children Are Healthier," and referenced a paper at Open Access Text called "Pilot Comparitive Study on the Health of Vaccinated and Unvaccinated 6- to 12-year-old U.S. Children," by Anthony R. Mawson, Brian D. Ray, Azad R. Bhuiyan, and Binu Jacob.  So I decided to check out the paper itself.  Here's a bit of it, so you can see their basic argument:
Vaccinations have prevented millions of infectious illnesses, hospitalizations and deaths among U.S. children, yet the long-term health outcomes of the vaccination schedule remain uncertain.  Studies have been recommended by the U.S. Institute of Medicine to address this question. This study aimed 1) to compare vaccinated and unvaccinated children on a broad range of health outcomes, and 2) to determine whether an association found between vaccination and neurodevelopmental disorders (NDD), if any, remained significant after adjustment for other measured factors.  A cross-sectional study of mothers of children educated at home was carried out in collaboration with homeschool organizations in four U.S. states: Florida, Louisiana, Mississippi and Oregon.  Mothers were asked to complete an anonymous online questionnaire on their 6- to 12-year-old biological children with respect to pregnancy-related factors, birth history, vaccinations, physician-diagnosed illnesses, medications used, and health services.  NDD, a derived diagnostic measure, was defined as having one or more of the following three closely-related diagnoses: a learning disability, Attention Deficient Hyperactivity Disorder, and Autism Spectrum Disorder.  A convenience sample of 666 children was obtained, of which 261 (39%) were unvaccinated.  The vaccinated were less likely than the unvaccinated to have been diagnosed with chickenpox and pertussis, but more likely to have been diagnosed with pneumonia, otitis media, allergies and NDD.  After adjustment, vaccination, male gender, and preterm birth remained significantly associated with NDD.  However, in a final adjusted model with interaction, vaccination but not preterm birth remained associated with NDD, while the interaction of preterm birth and vaccination was associated with a 6.6-fold increased odds of NDD (95% CI: 2.8, 15.5).  In conclusion, vaccinated homeschool children were found to have a higher rate of allergies and NDD than unvaccinated homeschool children.  While vaccination remained significantly associated with NDD after controlling for other factors, preterm birth coupled with vaccination was associated with an apparent synergistic increase in the odds of NDD.  Further research involving larger, independent samples and stronger research designs is needed to verify and understand these unexpected findings in order to optimize the impact of vaccines on children’s health.
Okay.  So, boiled down to its essence, (1) they admit that vaccines save lives, given that hardly anyone dies of diphtheria, polio, or tetanus anymore; (2) they claim that there seems to be an increased risk amongst vaccinated children of learning disability, autism spectrum disorder, ADHD, allergies, and asthma.

[image courtesy of the Wikimedia Commons]

Here's the problem, though.  Virtually all of the increased risks they describe are for conditions in which there is a huge spectrum of severity.  About 30% of adults and 40% of children are allergic to something, but these vary from sneezing during ragweed season to dying of anaphylactic shock if you consume a crumb of residue from a tree nut.  ADHD, in my experience as a teacher, ranges from kids who are a little fidgety to students who seem to be physiologically incapable of concentrating for more than five minutes on anything.

Hell, I have a learning disability in decoding written material myself; if I were in school today, I'd probably qualify for special services.  But I get along just fine, and in fact love to read even though I'm a bit slow at it and tire quickly.  I've had students, however, whose learning disabilities profoundly impacted their ability to manage most of the tasks they were expected to master in school.

So self-reported (or, in this case, mom-reported) conditions for which there is tremendous variability already makes the study a little questionable, especially given that those data are being compared to ones which are unequivocal -- such as whether the child in question ever got measles.

But there's a deeper problem still, and that is that the authors come into this question with an axe to grind.  As physician and blogger David Gorski points out over at his wonderful blog Respectful Insolence, the lead author of the study, Anthony Mawson, is an anti-vaxxer and had shopped around without success for a home for his paper in peer-reviewed journals, and finally had to post it at an open-access site because it couldn't pass review.  He did get an abstract accepted at Frontiers in Public Health -- but they retracted it only days after it was published, saying it was "under re-review."

Hardly a ringing endorsement.

Another problem is that homeschooled children are not a representative sample, nor are their parents.  There are lots of reasons for homeschooling -- one of my best friends homeschooled her daughter for the best of reasons, and she came out of the experience with a finely-honed mind and a deep passion for learning.  But there's a significant correlation between an homeschooling parent and being an anti-vaxxer, especially given the crackdown in many states on allowing belief-related exemptions for unvaccinated children to enter public school.

So right away there are some questions about the legitimacy of the data.  As I point out to my Critical Thinking students, sample bias doesn't mean the conclusion is wrong, necessarily, but it does cast it in a rather dubious light.

Last, and perhaps most damning, is the fact that Mawson's "study" was funded to the tune of $500,000 -- by an anti-vaxx group.  You'll note that nowhere in the paper cited above was any mention of a conflict of interest vis-à-vis financial support.

Which is a major no-no for a peer-reviewed study.

So mainly what the "Study Shows" is that if you walk in with your conclusion already in hand, you can bend the data whatever way you want to support it.  Especially if your pocketbook is being filled by people who would like very much for you to prove that their pet theory is right.

The bottom line: be careful when you see anything that claims that "Researchers Prove X."  Go to the source, and ask yourself some hard questions about the veracity of the study itself.  (Especially if you're inclined to believe its conclusions; confirmation bias plagues us all, and we're much more likely to accept something unquestioningly if it squares with what we already believed.)

And that goes double if it's Harvard researchers.

1 comment:

  1. Stanford-educated scientist and former Cornell professor supports assertion that Skeptophilia readers are intelligent and have a good sense of humor!