Are Surveys Headed For Extinction?

Every year an increasing number of surveys are being carried out in more and more countries. Only a decade or so ago many people lived out their lives without a single interviewer asking their opinion on the government of the day or paper towel brands. In major cities at least, it’s a rare person who can say that today. Yet, these staples of our industry are increasingly under threat, partly by new alternatives, partly by social changes and possibly, in largest part, by our collective inattention as an industry. Time, in my view, to rethink what a survey is good for and reinvent what we do with them.

The “reinvention” aspect though, is a post on its own, and I’ll deal with that one later. For now let’s consider the threats. Some of the external ones are well known – response rates are declining and good samples are hard to achieve by traditional methods. As industry initiatives in several countries show, new methods like online, while offering many advantages, are not a panacea and can produce some quite notable biaises. As traditional survey samples get more suspect, the reaction seems to be to retreat to a new acceptance of varieties of convenience sampling: more and more we see “polls” done by anyone in any old manner, getting huge publicity. Worse, I have heard quite senior MR figures talk of surveys of “thousands” or large panel sizes as if numbers implied accuracy. The 1936 US Presidential poll lesson of the Literary Digest vs George Gallup at times seems to have been lost: inevitably this will result in some major disasters, and the reputation of surveys will decline still further.

Simultaneously, new techniques based on nueroscience, social networking analytics, and (less sexy, but possibly more far-reaching) better data-base and panel analysis tools offer the potential to get at – more accurately and cost-effectively – much information that used to be the private domain of the ad hoc survey. OK, some of these advances are “over-hyped” but they should not be mocked: there is real potential for some of these to change the face of MR.

But the biggest threat to the survey is in fact ourselves, and this is sad. As an industry we’ve cut back on skills training and in-depth mentoring, (especially in the last few years) to such an extent that base survey quality is frankly in decline. OK this is a gross generalization, and there are exceptions, but my observation is that compared to when I was a junior, young researchers are better at explaining or identifying marketing issues (which is good) but worse at designing a clever questionnaire to address the issues or creatively digging into the data to find alternative solutions. Result: superficially more attractive studies, with high sounding conclusions, but little depth and hence little ability to provide a client with competitive advantage.

This is a worrying trend, not only for the clients, but also for MR profits. Survey design and analysis is the beating heart of most of the MR process. If structured badly almost everything else; fieldwork, DP, delivery timelines, rework percentages etc. etc. goes awry. Message to CFO’s: there’s more productivity gains and cost savings to be made by getting researchers to design better questionnaires and more effective samples than there is in out-sourcing back-office functions (which does not mean I’m opposed to the latter, but that’s another topic!).

Addressing this, alas, is not simply about getting someone in to do a couple of hours on the basics of questionnaire design (although in some case that would be a good start), it’s about having a plan to develop and maintain an appropriate research culture – there is a lot to this, and it’s easy see why it gets put on the back-burner while firms try and tackle what they regard as “low hanging” fruit solutions to sales and profitability woes.

The issue for our industry though, is if we want survey research to maintain its relevance then MR companies need to have such a plan: both to improve what we deliver to clients and to tackle the core issues that are impacting current financial performance. Developing strategies for building more effective “research cultures” (where effective implies both quality and productivity) is worth serious investment in my view.

Doing the basics better though, is only half the battle – as I mentioned at the beginning the whole purpose of the survey and how we use them needs re-thinking. I’ll tackle reinvention in the second part of this post.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: