Expectation-Based Testing

Data Quality with Pandas checks a DataFrame with boolean masks written inline, as part of the transformation code. This article covers a different methodology: declaring what should be true about the data as its own artifact, separate from the logic that produces it.

Imperative checks vs. declarative expectations#

An imperative check is a line of code, living wherever someone happened to write it, doing one specific thing:

invalid_mask = df["amount"] < 0if invalid_mask.any():    print(f"Found {invalid_mask.sum()} negative amounts")

A declarative expectation states the same claim as data, not code — inspectable, listable, and shareable independent of any specific pipeline that happens to check it:

import great_expectations as gx
validator.expect_column_values_to_not_be_null("customer_id")validator.expect_column_values_to_be_between("amount", min_value=0, max_value=None)validator.expect_column_values_to_be_in_set("status", ["pending", "shipped", "delivered"])

Each line names a claim about the data rather than embedding the check's logic inline — Great Expectations is one widely used implementation of this pattern, though the methodology does not depend on any specific tool.

Why this separation matters#

A collection of expectations — an expectation suite — can be listed, reviewed, and generated into human-readable documentation without anyone reading the pipeline's source code at all. This is what makes expectations a practical implementation of Data Contracts: the contract's claims about a dataset become the literal expectations checked against it, in one place, rather than scattered across whatever code happens to touch that data.

Expectations still need to run somewhere#

Declaring an expectation is not the same as enforcing it. A suite defined once and never actually executed against real data is functionally identical to the unenforced, documentation-only contract Data Contracts warns about — the value is entirely in the checks that actually run, on a schedule, against real batches, not in the expectations existing as declared claims somewhere in a repository.

Common mistakes#

Expectations loose enough to never fail#

An expectation with a range wide enough to accept anything is a false negative waiting to happen — it exists, it runs, and it passes regardless of what the data actually looks like. A useful expectation is tight enough to meaningfully fail when something is actually wrong.

Expectations so strict they fail on legitimate variation#

The opposite failure: a threshold tuned to today's exact data distribution fails the moment normal variation — a slightly larger batch, a seasonal spike — pushes a value just outside it. Data Quality Testing Anti-Patterns covers the alert fatigue this produces once it happens often enough that failures stop getting investigated.

Writing an expectation suite that never actually runs in a pipeline#

An expectation suite committed to a repository and never wired into a scheduled job or a pipeline checkpoint checks nothing. The suite itself is not the safeguard — the scheduled execution against real data is.

Further reading#

• Great Expectations documentation — a full reference implementation of declarative, expectation-based data testing.
• GX Core overview — the conceptual model behind expectations and expectation suites.
• dbt data tests — a second, SQL-based implementation of the same declarative pattern.
• Data Contracts — the agreement an expectation suite is typically built to enforce.

See also#