“Yes” and “No” campaign placards adorn a lamppost on Sept. 16, 2014, in Glasgow, Scotland. (Peter Macdiarmid/Getty Images)
On Thursday, Sept. 18, Scotland could jump back more than 300 years in time. Residents will have the opportunity to vote in a referendum on independence from the United Kingdom.
Polling from a variety of sources suggests that the Scottish public has moved in the past month from clear, double-digit margins for a “no” vote to a more narrow single-digit divide on the question. But before we accept that the vote is too close to call, let’s take a finer look at the limitations of the available polls.
Among the 14 public polls completed in September, 11 have been among Internet-recruited samples, two are phone polls, and one was conducted with face-to-face interviews.
This is not a small detail. How a poll is conducted can tell us a lot about the reliability of the data and whether it is representative of a larger population.
The online polls being reported in Scotland are not based on random samples of the entire population but on people who agreed to join a panel for the purpose of answering surveys. This methodology has the advantage of being cost-effective but lacks full coverage of the population. When voters’ attitudes in an election are difficult to predict — such as in a unique referendum on independence — such non-random methods may have more difficulty controlling for natural biases in the panel.
Polls that rely instead on phone or in-person methodologies use procedures to randomly sample homes or people, thereby giving all or nearly all of the people in a population a chance to be selected. These polls are more costly than the online methodology.
The scattershot results from these varying sources and competing polling methods are instructive. Four polls were completed on Sept. 11, three online and one by phone. These polls tell us that the vote margin could be five points for the “no” side or seven points for the “yes” side.
Perhaps these inconsistencies from the ICM surveys are what led Martin Boon, the director of the polling company, to auger a “polling Waterloo,” according to the Herald Scotland. Boon said there is a “real danger for the accuracy of the polls” due to the limited samples of the online methods.
But overall, few polls point to a likely “yes” vote, and the most rigorous survey tilts clearly toward defeat for the nationalists. Of the 14 polls conducted this month, just two find supporters outnumbering opponents, and no poll has found a majority of Scots supporting independence — the necessary threshold.
So what is the best indicator of what will happen on the question of Scottish independence? The most recent poll using the most robust methodology will generally offer the best picture of where things will end up.
In this case, that would be the phone poll from Survation completed Sept. 12that showed 42 percent in favor of independence and 49 percent opposing, with 9 percent unsure. The survey reached respondents on both land lines and cellphones.
The 9 percent who are unsure in the Survation poll are a critical group. If they do turn out to vote on Thursday, their choices have the potential to sway the vote one way or another.
It is notable that the Survation poll was sponsored by Better Together, a group promoting a “no” vote. But beyond the sponsorship, the poll’s questions and methods are consistent with other independent polls.
What all the most recent polls point to — from phone or online methods — is a tilt toward a “no” result.
Scott Clement contributed to this report.
