Key Porter, 1989, 234 pages, C$24.95 hc, ISBN 0-55013-172-9
Politics have changed considerably during the last century, and nowhere is this more true than in the now-omnipresent usage of polls. Media use them to boost viewership, establish predictions and build up front-page stories. Politicians use them to gauge the popularity of policies, track down their popularity and plan campaign strategies. Regular polls have become a regular part of the process, protected by an aura of scientific respectability in a field where impressions can often be more important than facts.
Claire Hoy is a well-respected Canadian journalist who, in 1988, reached his boiling point regarding this issue. How is it that the methodology of polls is never questioned? What is the impact of regular polling on Canadian politics? What are the implications of media/pollster relations when some pollsters are obviously biased in favor of political parties? Margin of Error is an attempt to answer these questions, and it makes for fascinating reading.
If you’re like this reviewer, Hoy’s central thesis -that pollsters have enjoyed uncritical admiration for too long, and that they now occupy a central position as decision-shapers- is initially suspect, if not outright paranoid. How can these friendly people with the Numbers be in any way dangerous to the democratic system?
The first section of Margin of Error paints an historical portrait of polling in Canada. Beginning during World War II by way of exiled American specialists, polling quickly established itself as an instrument of knowledge, and soon as a replacement for decision-making; Hoy traces the evolution of the usage of polls from being simple indicators for politicians, to smoke-screens behind which true vision can disappear and where the “best” politicians simply follow the polls.
Ah, but if only it stopped there… As Hoy demonstrates through chapters about the largest Canadian pollsters, the very perception of pollster impartiality (“just the numbers, ma’am”) is ludicrously absurd. Pollsters have long been associated with political parties, courting leaders to become official party pollsters.
It gets worse. Hoy clearly demonstrates, through example and a bit of logic, how questions can be slanted to obtain desired results, how precise formulation can affect results and how special-interest groups can, for a relatively low price, get “official” validation for their viewpoint by hitching a carefully-worded questions onto a “general survey”. Pollsters, despite their reputation as number wizards, can independently skew results with bad survey methodologies in an effort to save a few dollars. (Margin of Error shows, dollar-figures in print, just how expensive a good survey truly is, and how badly results are affected by skimping.)
Not only does it stop there, but as Hoy shows -again through several mind-boggling examples-, media outlets who report this information are most often than not incapable to make an accurate usage of these statistics. They’ll often misrepresent the question (forgoing the precise wording for a more audience-friendly “meaning”), ignore the shaky methodologies and try to buy results on the cheap, resulting in news that are, at best, not paining an accurate picture of reality.
Your reviewer, somewhat of a stats geek himself, started the book with a decidedly skeptical mind. But Hoy does his job properly, and the overall accumulation of facts, citations and -yes- statistics are simply too revealing to ignore. The misuse of polls represented in Margin of Error borders on the actionable, and yet, with eleven year’s insight, things have most probably gotten worse, not better.
In any case, Claire Hoy has produced, with Margin of Error, an essential piece of reading for anyone too easily trusting of polls. As it is showing significant age, an update might be in order. But don’t let that stop you from picking up the book and getting an eye-opener on statistical abuse.