Senior Data Quality Engineer, SDET with Databricks and Microsoft Fabric
- Experiência
- 5+ anos
- Inglês
- C1, obrigatório
- Práticas
- CI/CD, DevOps
Technology stack Python, SQL Server, Azure Databricks, Microsoft Fabric, Azure DevOps, dbt tests, Power BI Client Our client is a UK financial services group operating in a highly regulated environment, currently rebuilding its data and analytics capability as part of a wider platform modernisation programme. Project overview The project migrates data and reporting from a legacy estate onto a modern platform built on Databricks and Microsoft Fabric. Quality engineering is central to the project.
The business needs confidence that migrated data reconciles with the source, that pipelines behave as specified, and that reports produce consistent results. Position overview This is a data focused quality engineering role rather than an application testing role. You will build automated validation for data pipelines and data products, own reconciliation testing between legacy and target systems, and put quality gates into the delivery pipeline so that a failure blocks a release rather than producing a report that nobody reads.
Design and implement data testing and quality assurance frameworks. Build automated validation for data pipelines and data products. Own source to target reconciliation between legacy systems and the new platform.
Build repeatable regression packs that run unattended. Write unit tests that execute automatically within CI/CD, with quality gates that can block a release. Detect and handle schema drift and unexpected structural changes.
Validate