Data Contracts

Data Ingestion Fundamentals introduced an informal version of this idea: a source and consumer need to agree on schema, freshness, ordering, and delivery. A data contract formalizes that agreement — explicit, checked, and owned — rather than left as something every consumer independently infers from watching how a table currently behaves.

What a contract actually specifies#

A data contract typically covers more ground than a schema alone:

• Schema — field names, types, nullability.
• Semantics — what a field actually means. A amount column typed as an integer is a valid schema and an ambiguous contract until it states whether that integer is dollars, cents, or something else entirely.
• Guarantees — freshness (service-level agreement), completeness expectations, ordering.
• A change policy — how a breaking change gets communicated and versioned before it lands, not discovered by a consumer's pipeline failing.

Shifting the failure left#

Without an explicit contract, a breaking change — a renamed column, a retyped field, a status value that quietly stops being emitted — surfaces as a downstream pipeline failure, discovered by whoever's job broke, usually well after the change actually shipped. With a contract checked at the producer's own deploy time, the same change fails there, before it ever reaches a single consumer:

Without a contract:  Producer ships change -> consumer pipeline breaks (later, elsewhere)                         -> someone traces the failure back to its source
With a contract:  Producer ships change -> contract check fails in CI -> caught immediately,                            before the change reaches any consumer

This is the same "shift the check as early as possible" principle behind type checking a program before running it, applied to a data pipeline's interface between teams instead of a compiler's interface between functions.

Contracts as code: the same enforcement Avro's schema registry provides#

Avro and Schema Evolution covers a schema registry enforcing backward/forward compatibility rules automatically, rejecting a producer's breaking change before it is even accepted. A data contract is the same enforcement idea, generalized past structural schema compatibility to include semantics and SLAs too — some teams implement it literally as a schema registry plus a separate set of checked assertions about meaning and freshness; others use a dedicated contract-testing tool. The mechanism varies; the principle — check the agreement automatically, at the point the producer changes something, not after a consumer notices — is the same one.

Who owns the contract#

A contract is what makes a dataset a genuine single source of truth rather than something every consumer independently reinterprets. It needs an owner on the producer side — the team whose changes could violate it, and therefore the team responsible for updating it deliberately rather than breaking it accidentally. Without a named owner, a contract degrades into unmaintained documentation nobody notices going stale, and consumers quietly return to reverse-engineering expectations from whatever the data currently looks like — exactly the fragile default a contract exists to replace.

Common mistakes#

Writing a contract with no enforcement behind it#

A contract that lives only as a document, never checked automatically against real schema or data, degrades into documentation someone forgets to update — indistinguishable, in practice, from having no contract at all. Expectation-Based Testing covers turning a contract's claims into checks that actually run.

Specifying schema without semantics#

A contract that states amount: integer without stating the unit invites exactly the kind of silent, type-check-invisible mismatch this article opened with. Write down what a field means, not only what type it is.

No named owner on the producer side#

A contract without an owner has no one accountable for updating it when the underlying data legitimately needs to change — leaving consumers to choose between blocking a needed change or breaking silently around an unmaintained contract.

Further reading#

• Defining data contracts to work everywhere — Great Expectations — a widely referenced definition and framing of the practice.
• Service-level agreement — the formal shape of the freshness and completeness guarantees a contract states.
• Avro and Schema Evolution — the schema-registry enforcement mechanism a data contract generalizes.
• dbt data tests — a common way a contract's schema and value claims get checked in practice.
• Data lineage — tracing a contract violation back to the specific change that caused it.
• Data Ingestion Fundamentals — the informal version of this same agreement, from the ingestion side.

See also#