Business & Organizations · Customers & marketing

What customer research can and cannot tell you

Pick research that fits the decision, see why a poll with 2.4 million replies called an election wrong, and tell a rise that follows a change from a rise the change caused.

4 short lessons · about 17 minutes · 9 sources · updated November 2026

What you will learn

  • Tell primary research apart from secondary research.
  • Choose a method that fits an exploratory question or a measurement question.
  • Identify a sampling problem in a survey of existing customers.
  • Recognize wording that steers a response.

Lesson 1 of 4 · 4 min

Pick the research that fits the decision

Before you readIn 1985 nearly 200,000 people took taste tests and preferred a new Coca-Cola formula. After Coca-Cola swapped it in, how long was it before the original was back?

Correct.Not quite. The answer is 79 days.

The original was back after 79 days. This lesson is about why huge research can still answer the wrong question.

In 1985 Coca-Cola had taste tests from nearly 200,000 people, and they preferred a new formula. The company swapped it in for the original. Drinkers protested by the thousands, and 79 days later the old formula was back.

The research was huge, and it answered the wrong question. The tests asked which drink tasted better. The decision was whether to take the original away.

So start with the decision and what you don't know about it. Say a museum wants more people at its evening events. "We need research" is too broad to act on. "We don't know why nearby residents skip Thursday evenings" tells you who to ask and what about.

The museum can use two kinds of data, sorted by why they were collected. Primary data is gathered for the question at hand, through new interviews or a new survey. Secondary data was collected earlier for some other purpose, like census figures or the museum's past ticket records.

Secondary data is usually cheap and quick to get. Its weakness is that nobody collected it with your question in mind, so it may not hold the answer. Primary data can be built to fit, but it costs more and takes longer.

Interviews let you dig into a few people's experiences. One resident might say the last bus leaves before the event ends. Another might assume the museum closes at five. You couldn't have put either answer on a survey form, because you didn't know to ask.

What interviews can't give you is a percentage. Ten conversations might turn up five different barriers. They can't tell you how many residents face each one. Finding the reasons and counting them are different jobs.

Check yourselfA museum studying why residents avoid its evening events analyzes last year's ticket records, which were collected to process sales. What kind of data are the records for this new question?

Correct.Not quite. The answer is Secondary data, because they were collected for another purpose.

Primary and secondary refer to the purpose the data were collected for. Who holds them and whether the analysis is new don't matter. The records existed before this question and were gathered to process sales, so they're secondary data. How useful they are depends on whether they fit the current question.

Counting is the job of a survey. It asks many people the same questions, so it can estimate how common each answer is. That only works if the right people are asked and the questions are fair.

The museum's ticket records have a blind spot. They show every purchase and nothing about the people who thought about coming and stayed home. Those are the people the museum wants to understand. So ask of any evidence whose behavior is missing from it.

Try to recallA museum wants to discover why residents avoid its evening events and then estimate what share of residents face each barrier. Which method fits each aim, and why can't the first method answer the second question?

Interviews fit the exploratory aim because they look at selected people's experiences in depth. A survey of a suitably sampled population fits the estimate. A few interviews with selected people can't establish proportions for the whole population.

Lessons in this topic

  1. Pick the research that fits the decision

    See why Coca-Cola's 200,000 taste tests answered the wrong question, and match interviews, surveys and existing records to what each one can tell you.

    Essentials · 4 min

  2. Why 2.4 million answers got an election wrong

    A 1936 poll with 2.4 million replies picked the losing candidate. See how who gets asked, who replies and how a question is worded decide what a percentage means.

    Essentials · 4 min

  3. Did the change cause the result?

    Ticket sales rose after a new poster. See why that proves little, and how a test that splits people by chance shows what a change really did.

    Deeper · 4 min

  4. How a small sample can speak for a whole town

    Tell a probability sample from a convenience sample, use stratified sampling so a small group is heard, and check how many of the people picked replied.

    Deeper · 5 min

Three quick stories from this topic

The Bing headline test

A Bing headline idea waited over six months for a test, then lifted revenue by 12%

Managers had ranked the 2012 idea a low priority. An engineer finally showed the new ad headlines to some users and the old ones to others. The gain was worth more than $100 million a year in the United States alone.

Why it took a test

Revenue moves every day for many reasons. Showing both versions over the same hours meant the headline was the one thing that differed between the two groups of users.

After is not because

Sales that rise after a change might have risen anyway. A same-time comparison shows what the change itself added.

The census knocks on doors

The 2020 US census planned visits to about 56 million addresses that had not replied

By July 2020, 61.8% of households had answered on their own. A survey can live with that and report on whoever replied. A census has to count everyone, so census takers were sent to every address that stayed silent.

Why research uses samples

Reaching everyone in a population is slow and expensive. Most research asks a small part of it and chooses that part carefully.

Silence is information

The people who don't reply may differ from the ones who do. A census chases them. A survey has to report how many it never heard from.

The 1936 Literary Digest poll

A 1936 poll with 2.4 million replies picked the loser of a 46-state landslide

The Literary Digest mailed 10 million ballots, mostly to names from telephone books and car registrations, and three in four went unanswered. It forecast a win for Alfred Landon. Franklin Roosevelt carried every state but Maine and Vermont.

Big is not representative

A sample speaks for a population only if its members were chosen to stand in for it. Millions of replies from the wrong mix of people repeat the same tilt millions of times.

Two flaws at once

The mailing list left out many voters, and only 24% of the people on it replied. Researchers still debate which flaw did more damage.

Sources

This topic is part of From a useful idea to a working business, a guided path through related topics.

  • What New Coke revealed about brands

    In 1985 Coca-Cola swapped its drink for one that won taste tests, and drinkers revolted. Work out what a brand adds beyond the product, and why that's so hard to measure.

    4 lessons · 20 min

  • The marketing mix

    Marketing is more than advertising. See how product, price, place and promotion fit into one offer, why it starts from the customer, and what services and critics add to the list.

    4 lessons · 16 min

  • Advertising campaigns and media buying

    Follow a campaign from the client's brief to the agency's idea to the bill for airtime. See how firms buy reach, why sales figures can't show an ad worked, and how they set the budget.

    4 lessons · 17 min

  • Negotiation

    What decides who gets the better deal. See why your best option without a deal sets your power, how to find trades that leave both sides better off, and what experiments say about the first offer.

    4 lessons · 16 min

  • Why good inventions spread slowly

    The first smartphone went on sale in 1994. See what separates an invention from its adoption, and what sets the pace of its spread.

    4 lessons · 20 min

  • Double-entry bookkeeping

    Every transaction is written down twice, once where value went and once where it came from. See how that rule keeps the books balanced, catches some errors, and grew out of medieval Italian trade.

    4 lessons · 16 min

All business & organizations topics · Take the quiz