Are Surveys Heading For Extinction? The Approaching Storm.

In the next few years there will be a significant lift in the use of traditional “fact gathering”,  ad hoc surveys. This will be driven by demand for data in growth economies and in sectors where currency services and data-bases are still in their infancy. At the same time, in developed markets, as the recession recedes and firms start to generate cash they will realise a need to reacquaint themselves with consumers whose lives and values have been transformed by recent events. This Indian Summer of the traditional survey will not last, however, and firms that get fooled by the burst of prosperity and fail to prepare (by investing in R&D, software, and training) for a new world will ultimately fail.

Fundamentally the core purpose of the ad hoc survey needs reinvention. We need to rebuild MR businesses around focusing on what surveys do well and invest in alternatives to the things they do poorly. Why? The immediate threats to the survey are issues of quality and response rate. But the longer-term threats revolve around the fact that the “what were you doing yesterday?” reportage-type questioning is increasingly being supplanted by more effective techniques. Panels and electronic data capture have long been recognised as better ways to capture sales and audience estimates, but as online makes setting up other sorts of panels increasingly more cost-effective and new scanning and recording techniques come into play, the effectiveness of asking people “what brands of shampoo have you bought recently?”” seems increasingly limited.

Years ago, at a British university, I worked on a major longitudinal study of electoral behaviour that followed specific individuals over 3-4 election cycles. It was an enormously powerful tool for understanding patterns of loyalty, and why vote switching was happening. Far better than ad hoc polls at understanding the underlying drivers of changes in voting patterns.  In those days such panels were very expensive, and the lack of flexible statistical tools meant that they were hard to control and slow to analyse. Today we have the tools that can overcome these issues, yet commercial panel analytics have been sluggish to take full advantage of areas where they have innate advantages over conventional surveys. I detect signs this is beginning to change.

Custom panels are being built in all sorts of categories and this will increase. On the mass market online front though, most large online panels are still under-utlised, providing less than optimal value for clients, and making less money than they should. In particular, too many online panels are still treated as simply sources of cheap ad hoc survey sample, and clever added value services based on longitudinal analytics are developing more slowly than should be expected. To a degree, this reflects a reluctance to risk exhausting the panel and thus not having sample when needed for one-off surveys. But, in part it also reflects an historic lack of imagination and investment in panel R&D. In the current economic climate, the pressure on under-performing panels can only increase, and there are signs that the more progressive companies are actively seeking ways to utilise them more imaginatively. The use of ad hoc surveys to collect base behaviour and trend information will decline as a consequence.

Equally threatening to the dominance of the old-style survey are the new analytic and data-mining tools that, when used in conjunction with clients own ever-expanding data-bases, can produce almost magical insights. Data-mining, nueral networking, latent class segmentation and similar techniques allow data-bases to be used to categorise and target groups of consumers with a precision never before possible. Why do a segmentation survey, when you’ve already got much of the necessary data?

At the other end of the spectrum, the traditional “depth” versus “precision” distinction between qualitative and quantitative data is getting blurred. Nuero-science based techniques and better designs mean that small sample data need not be merely “indicative”. Moreover, the use of new online qualitative techniques means that small sample sizes and difficulties of recruitment are not quite the problem they once were. Meanwhile, social networking analyses, analytics based on blogs and other ways of mining internet information, provide feedback that combines something of both richness and scale. It’s becoming easier to get “richer” data while still talking to a decent cross-section of your customers.

There are signs then, of a “perfect storm” building up against traditional surveys, perhaps made more dangerous because a brief period of warm weather likely to precede the storm may lure many researchers down to relax on the beach.  So what do we do? Should we abandon custom surveys and retreat to analysis of data-bases, blogs, panels and online groups? This would be a mistake: there are still many areas where the good old survey can deliver superior insights, and provide decent profits. However, that requires us to rethink why surveys are useful and in particular to focus on three crucial areas where they can deliver real competitive advantage. This involves a change in approach to training our staff, as well as how we position and target survey-based services. More on that, and the three key opportunity areas, in my next post.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: