Market Research: now done by AI!
Not by humans...
If you are in advertising or marketing, here’s a fantastic HBR article that you need to read right now:
Using Gen AI for Early-Stage Market Research
(Full disclosure: one of the authors, Ayelet Israeli, is one of my brilliant professors… so I’m highly biased when reading her research!)
If you work at an Advertising Agency or you’re a Brand Manager, and you do any flavor of ‘Market Research’ or ‘Consumer Insight’ for your clients… then this may be quite exciting and/or a little uncomfortable for you.
The article, and their research (from a 2023 study), shows that you can use LLMs for early-stage market research. With the proper prompting and architecture, you can create "synthetic focus groups" to simulate customer responses. They outline that LLMs have notable limitations with understanding a product’s novel features, customer segmentation, and dynamic market conditions.
Fine-tuning is the last thing you should do.
Yes. It’s true that, “A fine-tuned model built on years of carefully structured survey data is far more useful than one relying on internet training alone.” So… be my guest. Go fine-tune.
You’re likely to regret it.
The most impactful finding in the paper is confirming that "fine-tuning" LLMs with proprietary customer data can boost performance, but that fine-tuning methods only work within the same category of interest. You cannot “fine-tuned using laptop surveys” and expect that model to work on market simulation for “tablets.”
In other words… folks… fine tuning is the last thing you should try!
There’s a better way…
So, what’s the better solution? The answer is simple:
Retrieval Augmented Generation (RAG) wrapped in an Agentic Pipeline can solve this.
If you build out the proper pipeline, then you can completely automate the consumer insight and early market research. This is not difficult to do. AND… “marketers must invest in internal data collection and governance to maximize the value of these tools.” Without this, it won’t work. You must have your own well-collected and high-quality data about your customers and products.
So even though the authors talk about ”Augmentation” and not “Replacement”… I respectfully disagree.
Right now… today… we can completely automate the majority of early-stage market research. Sure… you still want a “Human In The Loop" (HITL) to check the work… but this is going to be at the top of the organization, like the CMO… not a brand manager, product manager, or R&D researcher. Many of these tasks are about to be automated.
And there are companies who are doing some early-stage UX market research for you!
Check out WEVO.AI!
We had Nitzan Shaer, the CEO of WEVO join me for a presentation at HBAP | STARTUPS last year.