Data Quality for AI: Six Checks and a Worked Example
TL;DR
Test the records needed for one decision. A six-row catalogue shows how to find duplicates, missing prices, unsupported currencies and expired records.
Data is good enough for an AI application when it supports the specific decision the application must make. A complete spreadsheet can still contain outdated prices. A clean document library can still mix policies for different customers. A million records will not solve either problem if the system cannot identify which record is authoritative.
Start with one output, its source fields and the mistakes that would make it unusable. Then test a small, representative slice of the data against those requirements. This gives you a repair list before you spend money connecting a model to the full collection.
This guide includes a deliberately flawed, synthetic catalogue and a reproducible validation exercise. It is a test of data rules, not a benchmark of an AI model or a claim about a client's records.
Define what the AI must be able to answer
Consider an assistant that helps staff prepare product quotations. Before choosing a model, specify what a valid quotation needs: an unambiguous product identifier, a current price, a currency, the appropriate customer scope and a source record that an authorised employee can inspect.
Now ask what happens when one of those items is missing. Should the assistant ask a question, refer the quotation to a person or refuse to produce a price? Decide explicitly. Otherwise a model may produce a plausible number where the business needed an unresolved state.
The same principle applies to other workflows. A support assistant needs the policy that applies to this customer today. A document processor needs a field definition and evidence for the extracted value. A forecasting project needs a meaningful outcome label and a way to avoid learning from information that would not have been available at prediction time.
Six useful questions about the records
The Government Data Quality Framework distinguishes completeness, uniqueness, consistency, timeliness, validity and accuracy. Those dimensions help separate different problems. In particular, a populated field is not necessarily correct.
For the quotation example, translate them into concrete checks:
| Question | Check | What a failure means |
|---|---|---|
| Are required values present? | Price and currency are populated | The quote cannot be completed from this record |
| Is the item uniquely identified? | One current record per product and customer scope | The system needs a rule for duplicates or versions |
| Do sources agree? | Sales and catalogue records use the same effective terms | Someone must identify the authoritative source |
| Is the record current? | Effective and expiry dates cover the quote date | An old price must not silently become today's price |
| Is the format allowed? | Price is numeric; currency is supported | Normalise or reject before downstream use |
| Is the value true? | Compare with an approved price source | A valid number may still be wrong |
These are separate checks, not six percentages to average into an impressive overall score. A quotation with a perfect format and the wrong currency remains wrong.

A six-row catalogue with four different problems
Our synthetic input contains the following rows. The complete fixture and expected findings are included in the downloadable test pack.
| Row | SKU | Price | Currency | Current? | Deliberate issue |
|---|---|---|---|---|---|
| 1 | A100 | 12.00 | GBP | Yes | None in the tested fields |
| 2 | A100 | 12.00 | GBP | Yes | Duplicate of row 1 |
| 3 | B200 | blank | GBP | Yes | Missing price |
| 4 | C300 | 18.50 | EUR | Yes | Currency outside this GBP-only workflow |
| 5 | D400 | 8.00 | GBP | No | Expired price |
| 6 | E500 | 22.00 | GBP | Yes | None in the tested fields |
We ran deterministic validation against this fixture. It flags one repeated SKU, one missing price, one unsupported currency and one expired record. Those results show that the checks detect the deliberately inserted cases. They do not establish the accuracy of rows 1 and 6, prove that a real catalogue has the same error rate or show that an LLM can quote reliably.
Notice the distinction between a duplicate group and a duplicate row. A100 forms one group containing two records. Keeping one and removing the redundant copy leaves one usable candidate, provided the records really refer to the same product and scope. With conflicting prices, automatic deletion would hide the disagreement instead of resolving it.
Repair the source, or define the exception
Give every finding an owner and a decision. The missing B200 price might belong to the commercial team. The expired D400 record may require a current catalogue import. The EUR value may be legitimate for another market, in which case the correct response is to exclude it from this GBP workflow rather than overwrite it.
Record the original value, the corrected value, the reason and the applicable version. Avoid cleaning data only inside a prompt where the change cannot be traced. If multiple applications use the same flawed source, a prompt-level workaround also leaves the underlying problem in place for everyone else.
Some exceptions should remain exceptions. If the current price has not been approved, the system should ask for it. Turning every missing value into a guessed value can make a dataset look complete while making it less trustworthy.
Sample by the ways the work can fail
A random sample is useful, but it can miss a rare document type that causes serious errors. Start with the actual workload and deliberately cover different sources, record ages, languages, layouts and customer scopes where those differences matter.
Keep ordinary cases and challenging cases distinguishable. If you build a test pack mostly from known failures, its error rate is not an estimate of the entire production population. Conversely, a clean demonstration pack selected by hand will tell you little about messy daily work.
For a document workflow, retain the source page alongside the expected fields. For a retrieval system, retain the relevant passage and access scope. The worked RAG guide separates finding the right evidence from producing an answer; a data-quality check should preserve that distinction.
Prevent yesterday's answer leaking into tomorrow's test
If you tune prompts or train a model using a set of examples, reserve other examples for evaluation. Where the task has a time dimension, consider whether a chronological split is more representative than a random split. Records from the same customer, document or incident may also need to stay together to prevent near-duplicates appearing on both sides.
The important question is what information will really exist when the system makes its decision. A support ticket's final resolution cannot fairly be used as an input when testing how the system would classify the ticket at arrival. A later correction to an invoice cannot be treated as information the original processor already knew.
Document how the evaluation set was constructed. That lets a reviewer challenge the test rather than accepting a single accuracy number without context.
Separate access rules from content quality
A record can be accurate, current and forbidden to the person asking the question. Check customer boundaries, employee roles and source permissions independently of whether the text looks useful.
When access changes, test the entire route, including search indexes, cached results and downloadable exports. A successful permission change in the source application does not itself prove that every copied version has become inaccessible. The AI integration guide explains why the connection between systems needs explicit operating rules.
Data minimisation also makes debugging easier: the smaller the relevant input, the clearer it is which source supports a result. Do not export a whole customer database merely to test a narrow classification task.
Set a decision for the first pilot
Finish the assessment with a short decision record: which data may be used, which cases remain unsupported, which defects must be repaired and what happens when a new defect appears. Name the person who owns the authoritative source and the person who can pause the workflow.
The download includes the six-row fixture and a repair log template. The Python example below implements the four checks using only the standard library. Run it as a small example, then replace the rules with checks derived from your own task. A clean run means the selected rules passed; it is not a general certificate that the data is safe or accurate.
For an AI readiness audit, bring one representative export, the required output and a few examples that currently need manual judgement. That is enough to begin a useful discussion about the work needed before a pilot, without pretending the entire organisation needs to be perfectly cleaned first.
Run the four checks yourself
Copy this Python 3 example into a local file. It uses only the synthetic rows shown above and makes no network requests.
from collections import Counter
rows = [{'sku': 'A100', 'price': 12, 'currency': 'GBP', 'current': True}, {'sku': 'A100', 'price': 12, 'currency': 'GBP', 'current': True}, {'sku': 'B200', 'price': None, 'currency': 'GBP', 'current': True}, {'sku': 'C300', 'price': 18.5, 'currency': 'EUR', 'current': True}, {'sku': 'D400', 'price': 8, 'currency': 'GBP', 'current': False}, {'sku': 'E500', 'price': 22, 'currency': 'GBP', 'current': True}]
counts = Counter(r['sku'] for r in rows)
findings = {
'repeated_sku_groups': sum(n > 1 for n in counts.values()),
'missing_prices': sum(r['price'] is None for r in rows),
'unsupported_currencies': sum(r['currency'] != 'GBP' for r in rows),
'expired_records': sum(not r['current'] for r in rows)
}
assert findings == dict(repeated_sku_groups=1, missing_prices=1, unsupported_currencies=1, expired_records=1)
print(findings)