Big Data & Market Research – Distant Cousins or Siamese Twins?

I recently attended a 2 day Big Data Summit conference in Sydney, run by Innovation Enterprise. From the show of hands, I was one of two market researchers among the 150 strong audience.

There a number of roles and opportunities for market researchers to play in this arena as from the quality and content of the presentations, it is clear Big Data is here stay, can only get ‘bigger’.

The retail, banking, and utilities sectors were well represented (fmcg was conspicuous in its absence) as was the Government sector with the Australian Bureau of Statistics, CSIRO, and the State of Queensland. The attendees’ profile, based on skill sets, was similar to an MR audience for a similar event but with a quantitative bias and an average age around 8-10 years younger.
big data 3 needs

Sportsbet’s Tony Greubner’s succinctly described 8 factors that would drive future of Big Data. His first three, the variety of sources, proscriptive analytics, and increased scope of application are perhaps the most pertinent (with no 6 “Geek is the new cool” being a favourite with the audience). Tony also highlighted three skill sets in short supply (see picture above) and here is where those with solid MR expertise could contribute.

Jason Millett of Billabong delivered an interesting case study on merging shopper data sets from 3 key markets. The outcome gave better understanding price elasticises and helped Billabong offer more profitable multi-product ‘market baskets’ to maximise the return from customers’ visits. On concluding, Jason demonstrated how his company aligns this data with ‘conventional’ consumer tracking to see how the packages impacted customer satisfaction.

One theme running throughout concerned definition of Big Data and several common ideas with nuanced differences were offered. In addition, a couple of speakers bemoaned that the current label didn’t do justice to the value they delivered – after all would you call the Sydney Opera House “Large Amount of Concrete and Glass”?

A second theme revolved around the challenges of combining structured and unstructured data, with the explosion of social media sources constituting most of the latter.

My main takeaway was that MR and Big Data have a lot in common, as well as a lot to complement. The challenges of both are not dissimilar, e.g.

1. Getting the end users to define clear objectives determines the quality of the deliverable (Ben Smithee of SPYCH has said the same of social data analysis).
2. Big Data professionals have strong technical proficiency and reasonable presentation/visualization ability. However, to better communicate findings in context they need work to work on their commercial skills. As with MR, the latter are need to enjoy better acceptance from senior management/C-suite in the broader business community.
3. The ‘Big Data’ industry is still getting to grips with how to define itself in terms of what it offers, what the benefits are and so forth. For how long has the research community re-visited that debate?
4. In reality, MR has dealt with Big Data (EPOS input for Nielsen/IRI type reports) for over 30 years. I mentioned to a couple of data scientists that at Nielsen Japan we delivered market reports in 5 days, but 3.5 days were spent data cleaning. They both nodded simultaneously stating that 70% was the time they allocated to preparing and aligning the data to get it useable.

Overall, there are many areas where MR suppliers and technicians can enhance Big Data’s offerings and vice-versa. I can see where professionals in both areas would come together both appreciating each other skill-sets and in have empathy for each other’s imperatives. So, maybe not Siamese twins but certainly immediate family.

(This blog is an abbreviated version of an earlier article appearing in the November 2013 edition of the Australian Market & Social Research Society’s monthly magazine – Research News)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: