Senior Data Engineer - Python/SQL (Public Sector)

Analityka rynku

Stack

Szczegóły

Bezpłatnie, po zalogowaniu.

Aplikuj

Prześlij CV — pracodawcy i rekruterzy z różnych krajów, którzy szukają takiego profilu, będą mogli napisać do Ciebie bezpośrednio.

Opis

Role Overview The Senior Data Engineer sits within the DSAD Platform and Architecture Team and is responsible for designing, building, productionising and operating trusted data pipelines and reusable data products on the SWDA data platform using AWS and Databricks. The role will provide technical leadership across data ingestion, transformation, modelling, quality, lineage, orchestration, observability and performance, applying CRISPr principles of comprehensiveness, robustness and scalability so that data remains dependable for analytics, reporting and ML/AI workloads. Key Responsibilities Data Pipeline Architecture and Delivery: Lead the design and implementation of maintainable batch and streaming pipelines that ingest, validate, transform and publish structured, semi-structured and unstructured data across AWS and Databricks. Lakehouse and Data Product Engineering: Design Bronze, Silver and Gold data layers using Delta Lake or equivalent patterns; develop reusable domain data models, curated datasets and data products with explicit ownership, definitions and service expectations. Source Integration and Incremental Processing: Implement reliable ingestion from databases, files, APIs and event streams using full-load, incremental, change-data-capture and replayable processing patterns appropriate to each source. Data Quality and Contracts: Establish schema enforcement, data contracts, automated validation, reconciliation and anomaly checks covering completeness, validity,

Pełny opis, firma i link do aplikacji — po zalogowaniu. Zaloguj się

Kontakty pracodawcy (e-mail/telefon/Telegram) są ukryte w publicznym podglądzie. Wyślij CV — a pracodawcy będą mogli napisać do Ciebie sami.