Finance: demonstrable lineage for regulatory reporting

A regulated financial institution had to be able to demonstrably substantiate reports to its regulator: which source, which transformation, who had access. The existing environment was a patchwork of separate services, each managed, secured and audited on its own, which meant lineage was largely reconstructed by hand for every reporting cycle.

Consolidation on Microsoft Fabric with a strict silver/gold separation following the medallion pattern. Controlled gold reports run in the Warehouse (T-SQL, multi-table transactions) rather than the Lakehouse, so there are hard guarantees on consistency within a single reporting run. Microsoft Purview tracks lineage automatically from source to report. OneLake Security governs access at column and row level, so sensitive positions are visible only to those with a role for it. All changes to pipelines and models go through Git integration and Deployment Pipelines, executed by a service principal, never manually in production.

The regulator does not ask for speed, it asks for reproducibility: can you show that every figure in the report comes about the same way, every time. That is exactly what Git plus Deployment Pipelines delivers, every change to a pipeline or model is a pull request with review, not a click in a portal nobody can reconstruct. Choosing the Warehouse over the Lakehouse for the gold layer gave the multi-table transactions financial reporting requires.

Keeping the existing patchwork of services and only improving the reporting layer did not solve the core problem: lineage stayed fragmented across systems that each kept their own audit trail. Bolting a fully external lineage platform onto the existing services created a second source of truth alongside the systems themselves, exactly the kind of inconsistency the regulator wanted to see disappear.

• Reserved capacity for production is more expensive than pay-as-you-go, but gives the predictability a regulated environment requires.
• Strict separation between production and everything else means more environments to manage, not fewer.
• Every change going through a pull request slows down small tweaks; for a regulated institution that is a deliberate trade-off, not an unwanted side effect.

Microsoft Fabric (Warehouse, Lakehouse, Git integration, Deployment Pipelines), Microsoft Purview (lineage, classification), OneLake Security, Microsoft Entra ID, Azure Key Vault for customer-managed keys.

• Treat every change to production as a pull request with review, no exceptions, not even for "small" fixes.
• Choose the Warehouse for reports that need multi-table transactions and hard consistency guarantees; leave the Lakehouse for engineering.
• Let Purview lineage track automatically instead of documenting it by hand; manual documentation falls behind within a quarter.

The biggest time savings did not come from faster reports but from less time spent reconstructing how a figure came about, that work largely disappeared because lineage is now captured automatically. An underestimated hurdle: service principals initially got overly broad roles to "just get it working", which auditors rightly flagged as a risk. Applying least privilege retroactively to existing automation took more time than getting it right from day one would have.