What's Your Opinion on Polls?

With election season ramping up in more than one country at the moment you will be hearing plenty about opinion polls as they are gathered, published, dissected and discussed over the coming weeks and months. But just how good are they and what is their track record in predicting what voters actually do?

In short, opinion polls often get it right but they also frequently get it badly wrong. The most recent case in point was last year’s British General Election. In the six months running up to that vote, a total of 230 opinion polls of voters were conducted by a variety of research organisations and published in a range of media outlets. The overall trends in the polls are shown below and from about mid-March onwards we see something of a dead heat emerging between the two main parties, the Conservatives and Labour. Of the last twenty polls taken before that election featuring voter samples ranging from 1,001 to over 18,000 people, eight predicted a tie while a further seven gave either side a one per cent lead. Every indication pointed to a really tight race but the actual outcome was a comfortable win for the Tories.

The results of the obligatory inquiry into this failure were published last month. It found that the probable cause was an under-representation of certain groups of voters such as over-70s, under 30s and ‘busy’ voters in certain types of polls that were conducted. Carried out on behalf the British Polling Council – an organisation with a major vested interest here, a methodological failure was probably the lesser of two evils, though an inability to construct representative samples is a pretty damming criticism. The more significant issue of course is that opinion polls in particular and market research more generally are very poor at predicting what consumers will actually do at some future point for a whole variety of psychological reasons. So enjoy all the prediction and discussion and let’s see if anyone can call it correctly this time!

Related Articles

The Problem With Market Research

The Problem With Market Research

The Irish Government shouldn’t be the only ones reeling from last Friday’s decision by the electorate to reject a constitutional amendment to abolish the Seanad or Upper House of government. It was another bad day for the market research industry as well. Consider this. Just three days before the election an Irish Times/IpsosMRBI opinion poll predicted the following result. Those in favour – 62 per cent and those against – 38 per cent when the undecided were excluded. How did the market researchers get it so wrong?

Let’s first examine a couple of ‘mitigating’ factors. After the poll, government ‘sources’ described themselves as ‘confident, but not complacent’ because 21 per cent of those surveyed responded that they were undecided while a further 8 per cent said they would not vote. There was some worry about which way the ‘undecideds’ would go but with a solid 17 point lead among the majority that had already made up their minds, the ‘Yes’ camp had reasons to be confident. And second, there was the issue of what might happen in the intervening three days up to the vote that might swing opinions, though, in hindsight, little of significance occurred during this period.

More to the point though is that this election demonstrated again, what is a common issue throughout the world namely, that opinion polls are not a reliable guide to how voters are going to behave. Earlier this year in the British Columbia elections in Canada, the NDP were given an eight to nine point lead the day before the election by two respected opinion polling organisations only to lose the election to the Liberals by five percentage points. Even early exit polls conducted on the day of the election did not show the extent of the swing. And of course, one of Margaret Thatcher’s election wins in the UK, came famously after polls predicted a win for Labour.

The commercial arena is similarly littered with famous examples of market research ‘mistakes’, the launches of New Coke, Red Bull and the Millennium Dome in London to name just three. While experts can argue forever about data collection instruments and sample sizes, one thing is certain – what people say they are going to do is not a good guide to what they actually will do. It is not that humans are dishonest though this can be the case. Instead there is a much bigger issue. People are constantly being asked to rationally explain what they have done in the past or will do in the future when in fact most of the decisions we make are emotionally driven and happen at a sub-conscious level. So it is little wonder that the market researchers so frequently get it wrong.