Reading a data quality report
Most serious data providers publish something about the quality of each delivery: completeness, late files, corrections, coverage against the universe. Most buyers never open it, which means the one artefact designed to tell you when not to trust a number goes unread.
It takes minutes to read and it changes how a period is interpreted.
What should be in one?
Completeness — how many expected stores reported, and for how many of the period's days. Late arrivals. Corrections applied. Coverage against the universe by the cells that matter. And any known incidents.
Anything less than that is a status page rather than a quality report, and it is fair to ask for more.
What does completeness actually tell you?
How much of the panel is in the number. A period at partial completeness is a smaller sample with wider uncertainty, and if the missing stores are not random, it is also a biased one.
The second half is the part that matters. Ask whether non-reporting correlates with anything — region, size, system type — because if it does, the gap is not just noise.
How should a low-completeness period be treated?
With caution and an explicit note. It is usually still usable for direction and less reliable for level, particularly in small cells where a handful of missing stores changes the answer.
What should not happen is comparing a partial period against a complete one and reporting the difference as a market movement. That is a mechanical artefact and it is a common one.
What do corrections tell you?
That the provider is finding errors, which is good, and how large the errors were, which is the number to watch. A consistently small correction volume is reassuring; a period with a large one deserves a look at whether anything you have already reported is affected.
Corrections applied to prior periods are restatements and belong in the restatement policy rather than only in the quality report.
What is coverage drift?
A slow change in how much of the universe is measured, usually from panel churn. It moves totals for reasons unrelated to demand, and it is visible only if coverage is published period by period.
Plot it. A coverage line beside a sales line explains a surprising share of unexplained movements.
What should trigger a conversation with the provider?
A period with materially lower completeness, a large correction volume, a coverage change in a cell you rely on, or a reported incident. Any of those is a normal reason to ask what happened and what it means for your analysis.
Providers expect these questions from serious buyers. Asking them is part of using the data rather than a complaint about it.
How does this connect to your own reporting?
Whatever caveat applies to the source applies to everything built on it. A quality note that stops at the analyst and does not reach the deck is a caveat that has been lost exactly where it mattered.
Carry the note forward. One line on a chart is enough and it protects both the analyst and the decision.
What if no quality report exists?
Ask for one. Completeness and coverage are computed as part of any serious production process, so their absence from delivery usually means they are not being published rather than not being known.
Frequently asked questions
How low is too low for completeness?
There is no universal threshold, and it depends on the cell sizes you use. What matters is that you know the figure and treat a low period differently from a full one.
Does completeness affect shares as well as totals?
Less, because missing stores affect numerator and denominator together. Shares generally hold up better under incompleteness than absolute levels.
Should I re-run an analysis after a late file arrives?
If the analysis informs a live decision, yes. Otherwise note the version and move on; the register of decisions bound to figures is what tells you which is which.
What is a reported incident?
Anything that interrupted normal collection or processing — a system outage, a transmission failure, a batch error. Providers vary in how much they disclose, and more disclosure is better.
Can I audit the quality report?
Partly, by checking store counts and coverage against what you would expect and by comparing period to period. A full audit requires access most buyers do not have, which makes the provider's transparency the practical control.
Does a clean report guarantee a good number?
No. It tells you the collection process behaved; it says nothing about whether the universe is right or the weighting is sound. Those are separate questions.
How does this apply to internal data?
Identically. Internal pipelines fail quietly, and a completeness check on your own data is usually the fastest way to explain a surprising internal number.
Should the quality report be automated into a dashboard?
A completeness figure beside the headline is the single most useful integration, because it puts the caveat where the number is read. Statistical agencies such as the Census Bureau publish response rates alongside estimates for the same reason.
What is the one field to read if you read nothing else?
Completeness for the period, in the cells you use. It is the field most likely to change how you interpret the number in front of you.
### What should happen when a quality issue is disclosed?
Check whether it affects the cells and periods you actually used, then decide whether to restate or to note. Most disclosed issues turn out not to touch a given analysis, and establishing that takes minutes.
Where it does, the register of decisions bound to figures is what tells you whether anything downstream has to change. Without that register, the assessment is a guess.
Working with retail data? See NRS Insights data solutions or contact NRS Insights.