Why your data team spends most of its time on plumbing
You hired analysts and they became integration engineers. That is not a discipline problem; it is what the org chart made inevitable.
A familiar frustration in organisations with a data function: expensive, capable people were hired to produce insight, and most of their week is spent moving data between systems, chasing a failed load, and reconciling two sources that ought to agree.
This is usually read as a maturity problem that better tooling will fix. More often it is a structural one, and the structure is visible on the org chart.
The incentive that produces plumbing
In most organisations the team that produces data bears no cost for how difficult it is to consume. A product team ships a feature, adds a table, changes a column, and moves on. The consequences land on the data team, who have no authority over the producing system and no influence over its roadmap.
The rational response is to absorb the mess: build a transformation layer that compensates for every upstream irregularity. That layer grows. Maintaining it becomes the job. And because it works, the upstream problem is never visible enough to fix.
What changes the ratio
- Move responsibility for data quality upstream, to the team that owns the system producing it. Nothing else addresses the cause.
- Make consumption difficulties visible to producers. A weekly figure showing which upstream changes broke which pipelines redistributes attention quickly.
- Treat data as a product with an interface, so that changing it is understood as changing an interface rather than editing a private table.
- Fund the platform work explicitly, rather than expecting it to be absorbed between analytical requests where it will always lose.
- Be honest in hiring. If seventy percent of the role is engineering, hiring analysts produces frustrated analysts and mediocre engineering.
The reasonable objection
Product teams will say, fairly, that they are not data engineers and have their own roadmap. That objection is legitimate, and the answer is not to transfer the whole burden to them. It is to give them a small, well-supported set of obligations — publish a contract, announce breaking changes, own the meaning of your fields — and to make the platform team responsible for making those obligations cheap to meet.
Where that balance is struck, the plumbing does not disappear, but it stops expanding. And the people hired to find things out get to spend a meaningful share of their week actually doing it.
Let's build the system your business will run on next.
Tell us where it hurts. We'll bring the architects, engineers, and delivery model to fix it — and scale it.