Senior Data Engineer – Data Platform & Modernisation

Analisi del mercato

Stack

Dettagli

Gratis, dopo l'accesso.

Candidati

Carica il tuo CV: datori di lavoro e recruiter di diversi paesi che cercano il tuo profilo potranno contattarti direttamente.

Descrizione

Job Responsibilities Lead the architecture, design and implementation of ETL processes, workflows and data pipelines for the ingestion, transformation and integration of data from multiple source agencies. Build and maintain efficient and reliable large-scale batch and real-time data pipelines, including dependency management, failure handling, recovery and operational monitoring. Develop and maintain modular, production-quality code for statistical production processes, including the modernisation of legacy processes while preserving existing business logic and reconciling outputs against current production results. Review complex business logic and provide technical direction for data migration and modernisation initiatives. Lead database and data model design, including schema design, storage patterns and performance optimisation for large-scale statistical datasets. Design and drive the adoption of reusable data engineering components, including common libraries, configuration frameworks, logging standards and utilities to streamline production workflows and promote consistent engineering practices. Design and implement CI/CD pipelines for data engineering workflows to automate testing and deployment, while establishing workflows across Development, UAT and Production environments. Lead the design and configuration of complex pipeline workflows and schedules using enterprise tools, including dependency management, failure handling, recovery and operational monitoring. Est

Descrizione completa, azienda e link per candidarti: dopo l'accesso. Accedi

I contatti del datore di lavoro (email/telefono/telegram) sono nascosti nell'anteprima pubblica. Invia il tuo CV così i datori di lavoro possono contattarti direttamente.