Weighting and projection: turning a sample into a market
A panel measures the stores in it. Statements about a market come from projecting those measurements onto the universe of stores that were not measured, and that projection is a model rather than an observation.
Every syndicated retail figure a brand uses has been through this step, and the assumptions inside it are rarely visible in the output.
What does projection actually require?
A defined universe, a known or estimated size for it, and a weighting scheme that maps sampled stores onto it. If the universe is "independent convenience stores in a state," somebody has to know how many there are and what they look like.
The universe definition is the load-bearing part. Two providers projecting to differently defined universes will produce different market totals from identical store data.
How is weighting done?
Stores are grouped by characteristics believed to matter — format, region, size, urbanity — and each sampled store stands for a number of unsampled ones in its group. The weights are set so the weighted sample matches the known composition of the universe.
The assumption is that stores within a group behave alike. Where they do not, the projection carries an error that no amount of sample size fixes.
Why does sample size not fix bias?
Because bias is a systematic difference between the sample and the universe, and adding more of the same kind of store adds more of the same difference. Bias does not shrink as a sample grows; it compounds confidence instead.
That is the single most important sentence in this subject, and it is the one most often reversed in a sales conversation about large samples.
What makes a weighting scheme credible?
That the grouping variables are the ones that actually drive differences in the category being measured. Region and format are standard; for some categories, proximity to a particular kind of location matters more.
A provider that can explain why its grouping variables were chosen has thought about it. One that lists them without justification may have inherited them.
What should a buyer ask?
How the universe is defined and sized, what the weighting variables are, what the effective sample is in the cells that matter to them, and what the provider does when a cell is thin.
That last question matters most for a brand with a regional or format-specific interest. A national projection can be sound in aggregate and thin exactly where one brand needs it.
What is the honest limitation?
That a projection represents the universe it was built for. A panel of independent stores represents the independent channel, not every channel, and a provider that names what its data does not cover is telling you it understands sampling.
Presenting a channel projection as a total market figure is the error that follows, and it is usually made by the reader rather than the provider.
How should projected figures be reported?
With the universe named, the basis stated, and the uncertainty acknowledged. Projected totals carry more uncertainty than the sampled totals they come from, and reporting them to the same precision implies otherwise.
Frequently asked questions
Is a bigger panel always better?
Better for precision within its universe and no help at all against bias. Coverage of the right store types matters more than raw count.
How do providers know the universe size?
From government and commercial store counts, trade sources and their own fieldwork, with the specifics varying. Ask which sources are used and when they were last refreshed, because store universes change.
What happens when a cell has very few stores?
Estimates for that cell become unstable and providers typically suppress, combine or flag them. Silent projection from a handful of stores is the practice to ask about.
Can I get the unprojected data?
Sometimes, and it is worth asking. Sampled totals are what was actually measured, and for some analyses they are preferable to a projection.
Does projection affect share figures?
Less than it affects absolute totals, since projection errors partly cancel in a ratio. Shares are usually the steadier output and should be preferred where the question allows.
How often should weights be refreshed?
Whenever the universe changes materially, and on a regular cycle regardless. Stale weights describe a market that no longer exists.
Is this specific to retail?
No — it is standard survey sampling practice applied to stores rather than to people. The discipline is well developed and the questions are the standard ones.
What is the single best question to ask a provider?
What does your data not cover. The answer separates providers who have thought about their universe from those who have not.
### What does a projection error look like in practice?
A market total that disagrees with shipment data, or a regional figure that moves in a direction nobody in the field recognises. Neither proves the projection wrong and both are worth investigating.
The most productive response is to compare at a level where you have independent evidence. A brand with its own shipment and distribution data has a genuine check available and rarely uses it.
Should projected and sampled figures ever appear on the same chart?
Only with clear labelling, because they are different quantities. A chart mixing measured and projected series without distinguishing them invites a reader to compare two things that were built differently.
Working with retail data? See NRS Insights data solutions or contact NRS Insights.