The Problem With Market Research

The Irish Government shouldn’t be the only ones reeling from last Friday’s decision by the electorate to reject a constitutional amendment to abolish the Seanad or Upper House of government. It was another bad day for the market research industry as well. Consider this. Just three days before the election an Irish Times/IpsosMRBI opinion poll predicted the following result. Those in favour – 62 per cent and those against – 38 per cent when the undecided were excluded. How did the market researchers get it so wrong?

Let’s first examine a couple of ‘mitigating’ factors. After the poll, government ‘sources’ described themselves as ‘confident, but not complacent’ because 21 per cent of those surveyed responded that they were undecided while a further 8 per cent said they would not vote. There was some worry about which way the ‘undecideds’ would go but with a solid 17 point lead among the majority that had already made up their minds, the ‘Yes’ camp had reasons to be confident. And second, there was the issue of what might happen in the intervening three days up to the vote that might swing opinions, though, in hindsight, little of significance occurred during this period.

More to the point though is that this election demonstrated again, what is a common issue throughout the world namely, that opinion polls are not a reliable guide to how voters are going to behave. Earlier this year in the British Columbia elections in Canada, the NDP were given an eight to nine point lead the day before the election by two respected opinion polling organisations only to lose the election to the Liberals by five percentage points. Even early exit polls conducted on the day of the election did not show the extent of the swing. And of course, one of Margaret Thatcher’s election wins in the UK, came famously after polls predicted a win for Labour.

The commercial arena is similarly littered with famous examples of market research ‘mistakes’, the launches of New Coke, Red Bull and the Millennium Dome in London to name just three. While experts can argue forever about data collection instruments and sample sizes, one thing is certain – what people say they are going to do is not a good guide to what they actually will do. It is not that humans are dishonest though this can be the case. Instead there is a much bigger issue. People are constantly being asked to rationally explain what they have done in the past or will do in the future when in fact most of the decisions we make are emotionally driven and happen at a sub-conscious level. So it is little wonder that the market researchers so frequently get it wrong.