Myth.ology and Market Research


Tales of the Market Research Monster - Should Clients be Frightened? (Image via Wikipedia)

Our recent post on the “Myth of Market Research’s Failure” attracted a lot of readers, and recently a stern rebuttal (see comments at bottom of the post) from Philip Graves who is the author of a book called “Consumer.ology” (not to be confused with Martin Lindstrom’s similarly titled tome, “buy.ology”). Sub-titled “The Market Research Myth, The Truth about Consumers and the Psychology of Shopping” according to an unnamed reviewer on Mr Graves’ website the book “will send a shiver down the spine of the research industry”. Given that, it is not surprising he did not like our post!

Published last September, I confess I have not yet read his book, but it has got a “top 10” spot in the UK business books list on Amazon, and Philip seems to have been interviewed extensively including by magazines like “Wired” and even “Research Live”. It is then seemingly indicative of the trend towards wholesale lambasting of market research methods that we discussed in our post. The core argument as far as we can see is that surveys address the wrong side of the brain (i.e. miss emotive/intuitive response) and our qualitative methods are subject to bias and group think.

The great thing about this kind of attack is that it draws attention to the need for research to be more serious about providing better quality information to clients. The downside is that they tend to work by creating a “straw man” version of market research, in which out-dated and poor practices are treated as the norm and used as an excuse to belittle the whole industry.

Our view, stated simply, is that it is often perfectly possible to design fairly conventional research that can measure a good deal of underlying emotional or subconscious response and that where that is not possible with traditional methods there are a number of ways many of the “new MR” methods that are emerging can be applied to fill in the gaps. Clients or agencies who are dissatisfied with the quality delivered by their research deliverables should consider what can be done to improve them, rather than giving up on market research entirely. (As a disclaimer, we should mention that Gordon & McCallum provides clients with review and R&D support to improve MR systems and outputs – so we would believe that!)

Let us then, return to the specifics of Philip’s criticism of the “Myth” post. We are not defending all market research in all circumstances. Instead we are pleading for more effort in areas like research design, interpretive thinking and training rather than simply focusing on a search for superficially attractive “silver bullet” solutions. In this context the “70%” success figure applies to the helpfulness of “conventional” MR methods (e.g. surveys, focus groups) and does not imply that MR would fail 30% of the time, only that there are a proportion of marketing issues that are now better answered by newer MR techniques (e.g. those based on the emerging neuroscience, social media research, community panels and so on) or which are simply so complex or nebulous that (as Philip implies) a judgment call is going to be as helpful as doing research.

On the issue of the value of experience and seniority we do not argue against frameworks or systematic application of research practices (indeed helping clients get more ROI from research by applying such frameworks is what Gordon & McCallum specialises in). But research often tackles complex issues and no matter what systems you put in place, it is improved if you add some time for thought about design and interpretation and involve people with relevant experience. A core reason why research projects sometimes fail is that the role of time and brainpower is not fully appreciated. When we do research reviews, one of the big things we to look at is the development of frameworks and processes so senior researcher time can be redeployed from “mundane” tasks to areas like interpretation and research design. Few things add more value to research outputs, and in our view the fact that adjusting the research process can make such a significant impact on client satisfaction is in itself evidence that our problems are more about how we do things, less about what we do.

It is simply untrue that the fact that experience is helpful means that market research is unsystematic. The analogy here is with doctors – all doctors (hopefully) are well trained and have diagnostic and interpretive frameworks to work with. However, a 15-minute consultation with a junior general practitioner is not going to yield the level of treatment advice that a longer visit to an experienced specialist will provide.  That does not mean, however, that you should rely on self-diagnosis rather than seeing the junior doctor when you are sick!

As I mentioned, I have not yet read “Consumer.ology”, and I’m willing to believe it may avoid the issues noted in our post. However, we would note that “proof by example” is a poor method of debunking market research. MR is a multi-billion dollar industry consisting of a wide range of techniques and skill sets. In its current form it goes back to the 1920’s and examples of it’s success abound in journals (e.g. the Journal of Marketing Research) whilst every year in forums like ESOMAR conferences, clients voluntarily get up and present case studies of how MR has helped them. Even in the area of prediction (where most criticism of MR focuses), companies like Novaction or BASES have put extensive effort into validation and testing of their services. The scale of MR means it is relatively easy to find examples of poor research, but this is a tiny percentage of the total – critics of our practices need to show they have scoured the journals and can refute the far more numerous examples of success and validation.

This does not mean that market research cannot be improved or made more cost-effective. But this is primarily a matter of applying new methods creatively or designing questions and qualitative research with more care. As a simple example, Philip’s  point about “group-think” in focus groups identifies a real problem, but it is an ancient and well known one and avoiding it is frankly a “qualitative research 101” issue. In most cases it results from poor moderation, lack of the right exercises and techniques or superficial analysis. Understanding of consumer emotions is another area where MR is often criticized, but here too there are a plethora of techniques and many exciting new developments that mean that the superficial responses derived from badly designed survey questions are perfectly avoidable. (On this one, Alastair is currently writing a chapter on developments in emotional research, in a book called “Leading Edge Marketing Research” to be published by Sage next year  – it will contain more of the references Philip requests).

Finally, we reject the false dichotomy between the role of a client’s judgment and the utilisation of market research. Good marketing is part art, part science and market research is only one of the inputs required in decision-making. But well designed research has proved to be hugely valuable to numerous clients and it is doubtful that companies like Proctor & Gamble or Unilever would be the size they are without their strong commitment to the systematic utilisation of market research. To reiterate: rather than rejecting the application of market research altogether, clients would be better advised to focus on improving how they carry out research. In many cases an independent review to ensure their research systems and practices are up-to-date and efficient would save huge amounts of money and solve most of the problems Philip mentions.

So yes, market research cannot solve all problems or predict every possibility – but it does a lot better than Philip implies in his response. While we can (and should) improve in many areas, the starting point for such improvements is thought and planning, not a simplistic rejection of the whole industry.


4 Responses to Myth.ology and Market Research

  1. Just to clear up a couple of points…

    The “unnamed reviewer” was the victim of my poor website layout – he was actually named: it was Fat Face Chairman and Associate Fellow of the Said Business School, Alan Giles (a man I have never met, incidentally).

    If anyone thinks they might confuse the word “Buy” with the word “Consumer” they should probably go for a book with more pictures and fewer words than mine or Martin Lindstrom’s. Equally, they should be careful not to confuse “Research.Opinionated.Insightful” with “”.

    I didn’t “dislike” your post, I just thought it over-simplified the arguments being made against a lot of traditional research, the most significant of which come from the fields of psychology and neuroscience (rather than the selective examples used to illustrate them or the analogies that are tortured long after their moment to be illustrative has passed).

    I agree entirely that proof by example is not proof (except when it is in the form of demonstrable, repeatable examples of the type that has propelled scientific thought). I accept too that the understanding from science is not fixed. However, when dependable, repeatable science has amassed a large amount of evidence that shows people are poor at understanding themselves (subjects that Daniel Wegner’s ‘The Illusion of Conscious Will’ and Timothy Wilson’s ‘Strangers to Ourselves’ cover well), questioning the value of people’s post-rationalisations and claimed attitudes is, I believe, prudent. When similar evidence exists to show that asking questions introduces influences on respondents in ways that most had never imagined were likely, the problems of finding out what people think are exacerbated further.

    I haven’t rejected the whole industry, but I have pointed out (in Consumer.ology) why a lot of what the industry does is not valid and dependable in the way it likes to portray, and why better decisions can be taken by approaching the problem of understanding consumers differently. No one, to my knowledge, has accused the book of being simplistic; but then perhaps that’s because they wanted a different book, bought mine by mistake and then didn’t read it.

    Consumer.ology is intentionally destructively constructive: the acceptance and reliance on market research is so extensive that it takes quite a shake to push users to re-evaluate its role. Recently someone in an organisation with a larger research budget than most was reading the book whilst a focus group project was under way. I imagine it took no small amount of courage to acknowledge that the research was a complete waste of money (something the client was able to prove to himself by applying what he’d read), but this caused less pain than would have been caused by the commercial failure the research was directing the business towards. The focus group moderators were experienced, senior and working for a large and successful research company, but they (or any of their peers) could have run a similar project 100 times and they still wouldn’t have been close to getting an accurate or valid answer once, never mind on 70 occasions.

    I am all for improving market research; I just believe that doing so necessitates much more radical action than some people are comfortable contemplating; and that’s because of what science is revealing about how people’s brains work.

    Finally, Consumer.ology was reviewed by a researcher in the US a couple of weeks ago. She acknowledged that she should hate the book, but said she “loved it” and recommended it highly; so there’s a chance that you might not end up shivering if you read it.

    • Alastair Gordon says:

      Thanks for taking the time to reply Philip. In your critique of my post you might have added, I think, that in part I am also guilty of the “straw man” sin, in treating your earlier response as emblematic of the many recent attacks on our industry – attacks which I do feel are often very thin and superficial. In that sense the “simplistic” comment is aimed not at “Consumer.ology” per se (which I have no right to attack specifically until I have read it – which I will!), but at the wider trend towards almost complete dismissal of the virtues of MR as a discipline. Having said that, I’d make three points:

      1. Your distinction between neuroscience and traditional MR is unfair and limited in my view. The developments in neuroscience are extraordinarily exciting (indeed Gordon & McCallum will shortly be offering our clients a significant support service in this area), but they expand on and amplify traditional MR – they don’t replace it. As a simple example, currently much (most?) nueroscience MR provides insight into short term triggers and stimuli – incredibly valuable for areas like prediction I agree, but without much more data on usage, social context the impact of social influences etc. clients risk trading long term brand equity for short-term gains if they rely solely on such data. Combining several sources of information on behaviour is a client’s best bet.
      2. Few practitioners think of MR as “science” in a pure sense. Since it studies complex human behaviour it’s antecedents are in the “social sciences”: anthropology, psychology, political science etc. Like those disciplines, the complexity of the subject matter means that precision and replicability etc. have limits, and that theories and frameworks need to be constantly revised — but like those disciplines the insights provided are valuable and the process of improvement is generally an evolutionary one.
      3. Your point about people being (in some circumstances) poor about understanding their own motivations is well understood by most (half-decent) MR practitioners. But many MR companies have put considerable effort into alleviating this by applying different questioning methods, more sophisticated analytical methods, a bigger emphasis on spontaneous listening (communitity panels), indirect exercises etc. The fact that direct, obvious and superficial questions are still asked in too many surveys is an indictment of our training and individual companies — but clients suffering from research that doesn’t tackle real motivations should seek advice on what research methods are really available — not just reject MR per se. Similarly, in your FGD example, without knowing any details, I would wonder whether the client and agency have really communicated on objectives and needs. In a similar setting with a client facing agency problems, I’ve found a couple of hours on the phone discussing the research design issues and how to work with the agency solves many of these problems. On the face of it his/her decision does not seem courageous to me – more a waste of a valuable investment of time and money that likely could have been alleviated by seeking some alternative advice.

      However, having said all this I must say I do agree that the issues that you are raising are hugely important. Whether, as you suggest, the industry needs a revolutionary change of paradigm, or (as I think) it needs investment in reform and a top-to-toe housekeeping exercise to clean out the poor practices and improve deliverables — there is no debate between us that big change is necessary.

  2. Neil Gains says:

    Alastair & Philip

    I won’t even try to respond to all the discussion here, but I would like to flag a couple of points which are nagging me (and for the record I have read the book).

    I would never rely on reading one book to understand market research or consumer psychology, as that would only give me a limited perspective on the topic (perhaps not even as much as 70% of the truth). Similarly I would never use one approach to understand a consumer.

    While Philip’s one book is a necessary and useful contribution, and raises legitimate questions, it also simplifies many issues, as does some of the discussion.

    Consumers are subject to many influences on their behaviour (as are new product forecasts which is why it is impressive that the best models tend to be close to the truth more than 70% of the time). These influences include environment, culture & society, individual goals and objectives and the consumer’s mental state (hidden and public).

    You can’t understand all of these by using one method (even one of the one’s Philip is so keen on). Indeed to understand the social influences on consumers you need to look at how they interact with others (would focus groups be a better model for this than observation perhaps?).

    There are no “truths” in research, just useful pointers to help clients make better decisions – better ones than they would often make based on their judgement alone. Those decisions are not just about hard facts but about having empathy and understanding with customers, something good researchers are more often than not able to provide. The best researchers, with time and proper design, will help uncover some truths much better than less able researchers with less money & less time (in the same way that if I only pay 5 Singapore dollars for a haircut, I should expect a few laughs form my friends).

    At the end of the day, clients need to change behaviour, which requires an answer to ‘why?’ as well as ‘what?’. Observation and test markets can’t provide that alone. I believe that good researchers using the right approaches (ie more than one) can. Consumer-ology tries to persuade us that this is not possible. If that’s true, and behavioural data & analytics take over the ‘what?’ question, then we’ll all be out of jobs!

    I don’t believe that for a minute.

    • Alastair Gordon says:

      Thanks Neil, I cannot but agree with all you say, and particularly agree with the point about using a variety of methods and approaches to get the best results. It’s a point I made in an earlier post on shopper research (“It’s the Journey Not the Destination”) in which I argued the need for a more holistic approach to the subject than is currently fashionable. The same point applies to almost all key research issues in my view. I would also hope that researchers and consumer psychologists take note of your point about the importance of social influences – it’s one which our traditional “individual consumer” focus tends to under-play and is especially important (as you’ll well know) in understanding consumers in emerging markets.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: