Was it the sample that did it?
A report by Proff Pat Sturgess looked at why the pools got the 2015 election results a bit wrong. Proff Sturgess re-ignited the old argument about academic rigour versus business tendency to cut corners. The heart of the argument is random versus non-random samples. Academics with plenty of time and sometimes quite large budgets which are not subject to high opportunity cost constraints naturally favour classical random samples. Random samples are more likely to be accurate if carried out properly and they provide a more theoretically rigorous statistical treatment. But they are very expensive and cumbersome to administer. Private pollsters can’t afford this approach. And so turn instead to non-random methods. This helps them provide rapid, regular and mostly accurate results from smaller but more economic samples. To do this they turn to purposive sampling, mostly quota sampling which involves much smaller samples but where the researcher chooses respondents to fit the profile of the population. This is much cheaper to do than random sampling and uses a smaller sample.
But it is subject to a potentially higher level of sampling error than random sampling. This arises if the sample quotas (segments of respondents eg, male female etc.) misrepresent either the diversity or sizes of the various segments which are under scrutiny. This is what seems to have happened here. Older people were underweighted in the sample and as they are more likely to vote Conservative the poll results were skewed toward Labour.
It’s a surprising mistake to make but was perhaps caused by two issues. One older people’s reluctance to take part in telephone questionnaires or online polls and two the strata of ‘older people could have been under sized in respect to the total sample in effect stifling the true size of older opinion.’ So whilst Proff Sturgess report criticises the sampling method, the real problem could actually be better considers as a non-response issue or r strata size issue.
Under this arguemet the problem isn’t quota purposive sample per se, indeed Tongco (2007)
Tongco M DC, 2007 Purposive Sampling as a Tool for Informant Selection, Ethnobotany Research & Applications 5:147-158
Comments