Building a grid asset data platform regulators can audit

The utility held asset data in nine systems: a geographic information system, two asset management systems inherited from an acquisition, a work management system, SCADA historians, a meter data platform, an outage management system, a finance ledger, and a substantial number of spreadsheets that had quietly become authoritative.

None of them agreed on what an asset was. The same transformer appeared under three identifiers with three different installation dates and two different capacity ratings. Analysts had learned which system to trust for which field, and that knowledge lived in people rather than anywhere queryable.

Regulatory submissions were assembled by hand each quarter. A team spent weeks reconciling figures across systems, and when the regulator asked how a number had been derived, answering meant reconstructing the reconciliation from email threads and working files.

Condition-based maintenance had been on the roadmap for three years. It kept failing at the same point: the telemetry was available and the asset register was available, but joining them reliably was not possible because the identifiers did not match and the telemetry arrived a day late.

A previous data warehouse project had delivered a set of reports rather than a platform. It solved the questions asked in its requirements document and could not answer the next one, so analysts had gone back to extracting to spreadsheets within a year.

The objectives: one governed source of asset truth, telemetry ingested continuously rather than daily, automated lineage from every reported figure back to source, regulatory reporting produced from the platform rather than assembled by hand, and a model analysts could extend without an engineering ticket.

The platform was built as a lakehouse with clearly separated layers: raw ingested data preserved exactly as received, a conformed layer where entity resolution and standard definitions are applied, and a serving layer of governed data products that reporting and analytics consume.

Preserving the raw layer unchanged mattered more than it might appear. It is what allows a figure to be traced back to precisely what the source system sent on a given day, which is the question a regulator actually asks, and it means a definition change can be reprocessed historically rather than only applied going forward.

Entity resolution for assets was the hardest engineering problem in the programme. A deterministic matching layer using identifiers, geospatial location, and installation attributes resolved the large majority automatically; the remainder went to a stewardship queue where subject matter experts adjudicated, and every adjudication was retained as a permanent matching rule.

Telemetry from SCADA historians and meter data was moved onto a streaming ingestion path rather than a nightly extract, bringing condition monitoring data to fifteen-minute freshness and making the condition-based maintenance use case viable for the first time.

Data definitions were made explicit and owned. Every field in the serving layer has a documented business definition, a named owner, and automated quality tests that run on every load. A failing test blocks promotion to the serving layer rather than surfacing later as a wrong number in a report.

Regulatory reporting was rebuilt as code against the serving layer. Each submission is reproducible: rerun it for a past period and it produces the same figures, with lineage showing exactly which source records contributed.