What Real Data Is Worth

Introduction

What do market research and carpenters have in common?

They both price on time and materials.

A market research proposal looks a lot like an estimate from a carpenter. Both are built from the bottom up, a function of materials needed for the job, time required to do it, and a markup. A carpenter’s estimate itemizes wood, hardware, and labor. It works for the carpenter because the materials are most of what the customer takes home: wood plus joinery equals a chair. It would be strange to price a chair any other way.

Research is priced and sold the same way, but it is not at all the same deliverable. The materials are the panel and data costs; the labor costs are the time humans spent programming, translating, project managing, and analyzing. Clients get the data, but that’s not what they seek. What they want is an understanding of the market and their customers. The operational factors are the raw ingredients — the line items on the invoice that describe how the work gets done. They are not the value the work produces. Nevertheless, this is the business model by which research agencies have been run for ages, mainly because the inputs have been genuinely expensive. Fieldwork at scale, be it a survey or a store audit, engenders real costs. Invoicing based on those costs is therefore logical and defensible.

The side effect of this business model is that the thinking and understanding that clients are buying has never actually been priced. It has instead been embedded in the margin. Clients, even though they were paying for the doing, got the thinking. Suppliers never had to have awkward conversations about what the thinking was worth because production costs were substantial enough to carry the price of the thinking. This has remained the case even through the digital age as technology reduced costs and made research more affordable.

AI breaks this in two ways.

The first is familiar. AI will reduce costs, albeit this time to an unprecedented degree. The research process is being fully automated — not just the programming, processing, and charting, but much of the design and analytical work as well. The cost of creating, running, and processing data is collapsing toward something close to compute costs. What the industry is not prepared for is the unavoidable impact on revenue. As has been the case with every previous technological advancement the industry has faced, the “savings” will be passed through to clients. It is, as the saying goes, just math.

The second is being marketed as if it solves the same problem. Synthetic panelists — by which I mean digital substitutes for humans whose demographics, behaviors, and attitudes are calculated or projected by a language model — are being pitched as a cost story: cheaper sample for cheaper questions, real fieldwork for the work that matters. The idea is reasonable on its face, and tracks with the way most buyers already think about research: you spend based on the stakes. A small decision does not warrant a large study. And even with an efficient production process, fieldwork is expensive. This is exactly how the decision to use synthetic panelists will be made by people who consider themselves prudent. Except that synthetic panelists are predictions of how people would act based on previous responses. In periods of stability, prediction is often close enough. But when the market does something unexpected, which by definition is something the model has not seen before, prediction fails.

This book is about how the research industry should think about its purpose and its pricing with both of these things happening at once. The impact of AI is more than just a story of cost compression. It forces us to ask questions about what a research firm sells, whether that product remains in touch with current reality, and what real data is worth.

Contents

• Introduction
• Part I: The Economics of Research
  • 1. How Research Creates Value
  • 2. Method and Moat
  • 3. Cost-Plus Pricing
• Part II: The Impact of AI
  • 4. Collapsing Execution Costs
  • 5. The Epistemic Threat of Synthetic Data
  • 6. The Price of Continuity
  • 7. Nutrition Labels
• Part III: What Real Data Is Worth
  • 8. Grossman-Stiglitz
  • 9. The Inseparability of Explanation and Real Observation
  • 10. What Remains to Sell
  • 11. A New Business Model
• Conclusion