Senior Data Engineer

Markteinblicke

Stack

Details

Kostenlos, nach der Anmeldung.

Bewerben

Laden Sie Ihren Lebenslauf hoch — Arbeitgeber und Recruiter aus verschiedenen Ländern, die Ihr Profil suchen, können Sie direkt kontaktieren.

Beschreibung

Senior Data Engineer Work location- Paya Lebar The Senior Data Engineer sits within the DSAD Platform and Architecture Team and is responsible for designing, building, productionising and operating trusted data pipelines and reusable data products on the data platform using AWS and Databricks. Experience: Typically 6–10 years of relevant data or software engineering experience, including hands-on design and operation of production data pipelines and at least 2 years working with cloud-based data platforms. • Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing. • Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering. • Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis. • Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets. • Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery. • Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines. • Experience with workflow orchestration, Git-based development, CI/CD, infrastruct

Vollständige Beschreibung, Unternehmen und Bewerbungslink — nach der Anmeldung. Anmelden

Kontaktdaten des Arbeitgebers (E-Mail/Telefon/Telegram) sind in der öffentlichen Vorschau ausgeblendet. Senden Sie Ihren Lebenslauf — dann können Arbeitgeber Sie direkt kontaktieren.