Webneuron
Data Engineering

Data contracts work when they are boring

The idea is sound and the implementations are often elaborate. What actually holds is a small, dull agreement that somebody enforces.

January 19, 20276 min readBy Webneuron Engineering Team

Data contracts address a real and expensive problem: a producing team changes a field, and three downstream consumers break, discover it days later, and spend a week establishing why their numbers moved. The proposal is straightforward — make the interface between producer and consumer explicit, versioned and enforced.

The difficulty is that the idea attracts elaborate implementations. Rich schema languages, registries, governance councils, approval workflows. These programmes tend to produce impressive documentation and very little behavioural change, because the elaborate version is expensive to comply with and therefore routed around.

What is actually required

A contract needs to say four things: what fields exist and their types, what they mean in a sentence, how often the data arrives, and who to contact when it does not. That is nearly the whole of the value.

The semantic definition is the part most often skipped and the part that matters most. Type checking catches a string where a number was expected. It does not catch a field that used to mean orders placed and now means orders shipped, which is the failure that costs a quarter of misreported figures.

What makes them hold

  • Enforcement in the pipeline. A contract checked in continuous integration is a contract; one checked in a review meeting is a document.
  • Breakage that surfaces on the producer’s side. If violating the contract only hurts consumers, it will be violated, because the incentive sits with the wrong party.
  • Versioning with a real deprecation period. Consumers need time; producers need permission to change. Both are solved by versions and neither by argument.
  • Ownership by the producing team rather than a central function. A contract written by a governance group about somebody else’s data is a description, not a commitment.
  • A small number of contracts covering the data that actually matters. Attempting to cover everything guarantees that nothing is covered well.

Starting narrow

The successful pattern is almost always the unambitious one. Take the three datasets whose breakage has caused the most pain in the last year. Write plain contracts for them. Enforce them in the pipeline. Let the reduction in incidents make the argument for extending the practice.

The failed pattern is equally consistent: a platform-wide initiative, a rich specification, a registry, and eighteen months later a set of contracts that describe how the data behaved when they were written.

Let's build the system your business will run on next.

Tell us where it hurts. We'll bring the architects, engineers, and delivery model to fix it — and scale it.