Industrial Data Archaeology: reconstructing analytics-ready operational context from brownfield railway telemetry archives

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Industrial operators accumulate decades of telemetry, events and diagnostic records in legacy systems designed for supervision and certification rather than analytics. Recovering analytics-ready information from these brownfield archives is an industrial information integration problem: signals are fragmented across heterogeneous schemas, sparsely and asynchronously sampled, and detached from the operational context required for engineering interpretation. We formalise Industrial Data Archaeology (IDA) as the upstream problem class of non-disruptively reconstructing analytics-ready operational context from brownfield archives, and demarcate it from generic ETL, lakehouse ingestion, physical retrofit and clean-data predictive maintenance. IDA is specified as a typed archive instance, a reconstruction operator and eight architectural invariants that any solution must jointly satisfy, together with structural guarantees—boundary confinement of fleet-specific knowledge, idempotent windowed reconstruction and schema-evolution safety—under which onboarding a new fleet reduces to configuration. The methodology is instantiated as a non-disruptive Medallion retrofit and evaluated on a multi-fleet railway deployment spanning seven rolling-stock archetypes over 2.5 years. Across 376,012 orchestrated task instances (November 2025–June 2026) the pipeline sustains 99.99% final task success (41 final failures; 99.76% recovery on retry) with no memory-related terminations, absorbs 3,379 schema-evolution events without downtime, and achieves 8.44× compression on continuous telemetry (4.19 TB → 496 GB) together with a 1940.8× serving-cardinality reduction. Event-context enrichment reaches 70.75%, and every post-seed fleet onboarding was completed through configuration alone. The formalisation and invariants are domain-independent; consumption of the resulting Gold products by predictive-maintenance and digital-twin workloads is the natural downstream stage.

This publication has 42 references indexed in Scilit: