Based on my personal experiences

Around 2011 the owner of the company that I then worked for came to Toronto and challenged the employees to go on social media. At that point, few had considered how important this was about to become for our jobs.

In the pre-historic days of market research, consumer data were collected by walking around the city, knocking on doors and interviewing people face to face. I caught the tail end of this when I started in market research in Russia (which is another story). Then, call centres became industry standard and most survey research was conducted on the phone. However, qualitative discussions were still held in person at focus group facilities.

In the 1990ies, online surveys took over as the main means of eliciting consumer data. Rather than finding consumers through random digit dialing, companies now started to build online panels of people who were willing to answer surveys. There was some concern about the representativeness of these panels at first, but it was soon put to rest when the considerable cost savings of this approach became apparent. Initially, research companies needed specific expertise, expensive software and sufficient server capacity to program, host and run these online panels and surveys. Quite a few even tried to develop their own software.

The 2000s saw a rise in self-serve online research solutions. The idea of software as a service (SaaS), and ‘freemium’ products such as SurveyMonkey made it possible for anyone to collect the data they wanted. It seemed no longer necessary to have any particular expertise or resources. At that time, I felt that many established market researchers were reluctant to open their eyes to the new reality. A few companies were blazing ahead, but the majority sat back and hoped it wouldn’t get too bad.

I realized myself around 2012 that things would never be the same again. When I had the opportunity to start my own business in 2014, I jumped with both feet into the world of SaaS, social media and startups, looking for ways to innovate and carve out a niche for myself in the insights community. I read Clayton Christensen and Eric Ries. I experimented with different tools and methodologies. I immersed myself into WeAreWearables Toronto, then a monthly event at the MaRS Discovery District, an innovation hub that offers services for startups and scaleups, and various other incubators and accelerators around the city.

Innovation was the buzzword, and also agile, real-time, lean and design thinking. The word disruption became popular in my neck of the woods a little later. Hard to believe that this was only six years ago. Now the word already seems a little tired. I feel that the dust of disruption has settled in market research, and firms have made adjustments.

Those who started off with the new technologies – for example Qualtrics, or the Canadian company Vision Critical – have probably done very well (not that I know their books). The traditional firms have incorporated various technologies into their offering. Many have social media analysis products. Some run large-scale customer feedback platforms for their clients. Some use virtual reality goggles for concept testing or shopper studies. Some offer online communities or hold virtual discussion groups. And to cut cost, many move business functions into the cloud, and outsourcing is widely used.

Whether this is sufficient to keep traditional market research organizations profitable, I don’t know. What I have learned in my own business is that I am still selling ATUs, segmentations, concept tests and one-on-one interviews. After the frenzy to be innovative and different, what clients seem to appreciate is my expertise. They trust me to know how market research is done properly, to execute the project for them, and to deliver them results that reflect a thorough understanding of their business questions, and in a form that adheres to their internal processes and requirements.

That’s why I stopped following the hype in the last couple of years. Disruption happened, but that was in the past. No amount of research automation can substitute understanding. And understand the customer we must.

But…I recently started renting a co-working desk at a business centre. I am with ease twenty years older than the majority of the other tenants there. So now I am back in a startup environment, and I see that while the hype has decreased, startups are here to stay for the foreseeable future. Businesses who are trying to disrupt various fields are numerous and, with the myriad applications of AI, far from done.

It will remain important to understand how new technologies fit into and change existing businesses. For example, I am intrigued by the anatomy of the Cloud, where servers are located, how data are moved around and what implications different factors have on data loading speeds, data security etc. When some people that I recruit for surveys complain about it taking longer than they thought, is that because their Internet is slow, or because the survey hosting company has switched from using their own servers to the cloud?

Data governance also interests me. With many market research companies using subcontractors, it is almost impossible to see all the way down the supply chain where data may be stored, processed or transferred to. The business risk related to data governance has increased exponentially for market research firms. Good times for lawyers and insurance companies.

But understanding AI, Cloud and computing in general is and will be immensely important for anyone interested in the affairs of this world. I think this is a topic that they should teach kids about at school, so that future citizens can make informed decisions about it. I want to learn more. So far, I have read three books on AI – it’s a start.

Barbara’s AI reading list:

• Kartik Hosanagar: A Human’s Guide to Machine Intelligence
• Ajay Agrawal, Joshua Gans, and Avi Goldfarb: Prediction Machines
• Virginia Eubank: Automating Inequality

Update, also… Janelle Shane: You look like a thing and I love you