Milton Pedraza is founder/CEO of the Luxury Institute
In 2021, the market research industry will undergo a radical reinvention.
Since the 1920s, when Daniel Starch started approaching people in the street to ask about their ad recall and a process of “polling” soon advanced by George Gallup, market research has evolved into an art, a science and a major industry that supports critical brand decisions through consumer insights.
From the 1930s through the mid-1960s, classic consumer goods icons such as Procter & Gamble, Coca-Cola and General Foods relied heavily on the first face-to-face, pencil-and-paper quantitative surveys, and face-to-face qualitative focus groups.
From the 1960s to the 1980s, market researchers used telephones and computers to automate parts of the same processes.
Over time, innovative researchers began to use psychology, economics, sociology, anthropology and other new techniques to try to get deeper into the consumer psyche: to understand their attitudes, preferences, motivations and behaviors.
New techniques and automation have emerged throughout the first 100 years of market research.
For those who have worked in marketing and advertising for decades, the market research industry has made incremental progress in many areas.
However, the industry has failed to innovate any radically disruptive breakthrough to dramatically advance its understanding of consumers to support more effective brand marketing decisions.
For example, the efficiency of recruiting of consumers, programming of responses and the execution and analysis within survey projects have all improved through automation, with reduced labor and costs.
But these efficiency improvements are still being applied to the same survey questionnaires and, in the case of qualitative research, the same discussion guides, as those of the 1980s.
Focus groups have moved to Zoom during the pandemic, but this process simply virtualizes traditional methods.
While historically it has been difficult to falsify responses in a qualitative project, the scaled automation of quantitative research and the advent of new programmatic sample panel exchanges have spawned an army of unqualified professional survey takers, many in invalid geographies, whose main interests are purely transactional. They straight-line answers and speed through surveys.
The biggest emerging problem, however, is the spread of fraudulent and ever more sophisticated bots that can game the CAPTCHA tests, deliver fake responses that mimic humans and render the research useless.
Easy-to-use survey software and other research tools, combined with an industry of easily accessible for-hire panels for quantitative research, have also inspired a client DIY market.
In fact, the online sample panel industry is estimated to be $1.5 billion, and demand is growing 10-30 percent annually, depending on the region.
The fact that only a handful of market research providers, agencies or brands, audit online samples for fraud – the same issue that has dramatically affected programmatic advertising performance – has spawned many dodgy survey sample providers who operate beyond the reach of accountability.
The top research firms have quality control programs, but no researcher or brand is immune from these structural industry challenges.
Besides grappling with engaging the right survey responders, the access to 100 percent identified, validated, relevant, timely and highly predictive demographic and behavioral data has also eluded the market research industry.
For example, even when surveyors reach the right target consumers, the data and insights gathered from quantitative surveys is reported data, which has huge limitations.
Responders often do not recall, are unable, or unwilling, to share how exactly they behave. Crafting closed-end questions in a survey places limits on the potential responses. And even crafting the best open-ended questions, it is very hard to understand why consumers say what they say or do what they report they do.
Methods that make a survey statistically valid and reliable do not address these inherent flaws. This can cast doubt on the results, and when accepted, lead to poor brand decisions.
This is by no means a critique of the dedicated, skilled professionals that serve the market research industry.
The industry challenge has always been gaining access to the right people with the right data.
Hopes that the advent of the Internet, where data is generated by humans at exponential rates, would bring an opportunity to access verifiable, relevant behavioral data, have been dashed by the fact that “walled gardens” such as Facebook, Google, Instagram and Amazon have exchanged “free” services for perpetual licenses to the richest, most relevant, most predictive consumer data being created.
These digital platforms have monopolized that data to use for their advertising-based business models. Brands and market researchers have effectively been shut out from the most comprehensive behavioral data ever created by human beings.
For example, customer segmentation analysis is one of the most important types of research objectives in the digital age.
Most advertising is now digital and is targeted and delivered by programmatic exchanges in a fraction of a second.
An accurate segmentation schema using demographic, psychographic, behavioral and geographic data is critical to serving up the right message, or offer, to the right audience, at the right time, over the right channel.
In today’s consumer insights world, access to legal, ethical data for creating segments, personas and audiences for ad targeting is essential.
Despite the fact that some ad agencies and exchanges have in many cases used illegal and unethical surveillance methods, targeting effectiveness is often still abysmal.
Empirical studies show that the accuracy of targeting one attribute in an online ad, gender, is 42 percent. For two attributes, gender and age, accuracy rate is 20 percent.
For three attributes – gender, age and income – accuracy goes down to almost zero. A brand team might as well toss dice or spin a roulette wheel to target digital ads.
The answer to both the research dilemma and the digital ad targeting dilemma is the same: engage with consumers legally, ethically, transparently and directly in the personal data value exchange process.
The best and most reliable path to consumer data access is in building a direct data relationship with the consumer.
Companies that create fair value for customers via rewards, benefits and personalization in return for access to consumer data will develop comprehensive, predictive data assets that will fuel every aspect of their marketing programs.
The radical innovation that is transforming the market research industry today is a legal framework and technology platform that empowers individual consumers to take control of their digital platform data – Facebook, Google, Amazon, Instagram and Twitter – create an asset that can be copyrighted, and licensed to ethical brands and their trusted market research partners, in a secure, encrypted platform, in exchange for benefits and rewards.
What is the future of quantitative surveys and qualitative research? These tried-and-true methods now become a solid, superb supplement, to the digital platform data insights, but are no longer the key drivers of highly predictive customer segmentation efforts.
Consumer analytics teams may run a survey, to probe more deeply with a subset of the panel, or a specific segment, and ask “why” questions, or test a new concept, or ad, as before. Qualitative research can be deployed for the same purposes.
The market research industry has served its clients well over its first 100 years.
AS WE BEGIN the industry’s second century, the true digital transformation of this critical industry, powered by legal and ethical personal data exchanges, is about to take off.
In 2021, as market research professionals take their new, highly influential and prestigious place at the table, they will be respected for empowering the marketing and sales teams to engage with highly predictive customer segments and audiences, armed with the most relevant custom content, offers, innovative new products and services far beyond what had been possible for a century.
In 2021 and beyond, market research innovators will be able to say they led the radical reinvention of their beloved industry.
Milton Pedraza is chairman of DataLucent and CEO of the Luxury Institute. Reach him at mpedraza@luxuryinstitute.com.