Senior Data Engineer - Python/SQL (Public Sector)

Analisi del mercato

Stack

Dettagli

Gratis, dopo l'accesso.

Candidati

Carica il tuo CV: datori di lavoro e recruiter di diversi paesi che cercano il tuo profilo potranno contattarti direttamente.

Descrizione

Role Overview The Senior Data Engineer sits within the DSAD Platform and Architecture Team and is responsible for designing, building, productionising and operating trusted data pipelines and reusable data products on the SWDA data platform using AWS and Databricks. The role will provide technical leadership across data ingestion, transformation, modelling, quality, lineage, orchestration, observability and performance, applying CRISPr principles of comprehensiveness, robustness and scalability so that data remains dependable for analytics, reporting and ML/AI workloads. Key Responsibilities Data Pipeline Architecture and Delivery: Lead the design and implementation of maintainable batch and streaming pipelines that ingest, validate, transform and publish structured, semi-structured and unstructured data across AWS and Databricks. Lakehouse and Data Product Engineering: Design Bronze, Silver and Gold data layers using Delta Lake or equivalent patterns; develop reusable domain data models, curated datasets and data products with explicit ownership, definitions and service expectations. Source Integration and Incremental Processing: Implement reliable ingestion from databases, files, APIs and event streams using full-load, incremental, change-data-capture and replayable processing patterns appropriate to each source. Data Quality and Contracts: Establish schema enforcement, data contracts, automated validation, reconciliation and anomaly checks covering completeness, validity,

Descrizione completa, azienda e link per candidarti: dopo l'accesso. Accedi

I contatti del datore di lavoro (email/telefono/telegram) sono nascosti nell'anteprima pubblica. Invia il tuo CV così i datori di lavoro possono contattarti direttamente.