Data Engineer

Job Description · We are looking for an experienced Data Engineer to join the delivery team for the EDW Reporting Modernisation project. · You will be responsible for designing and building the Gold layer Data Mart tables, transformation pipelines, and associated views on the Unified Data Platform (Databricks), working closely with the Tech Lead and Data Modeler to deliver a robust and scalable reporting data layer that caters to Power BI reports. · As the onshore Data Engineer, you will serve as the technical bridge between the Singapore-based design team and the offshore engineering squad, providing day-to-day technical direction, code review, and quality oversight across the full Data Mart build.

Key

• Design and Development:: o Develop Gold layer Data Mart tables and associated database views in Databricks in accordance with signed-off Source-to-Target Mapping (STTM) documents and design specifications o Implement Silver-to-Gold transformation pipelines using Databricks Workflows, Delta Live Tables (DLT), and PySpark o Create Delta table DDL in Unity Catalog, including schema definition, table properties, and column-level comments o Implement SCD Type 2 merge logic for dimension tables and aggregation logic for fact tables per the grain specification o Configure table partitioning, Z-ORDER clustering, and Liquid Clustering strategies for query performance optimisation o Develop and implement data quality expectations using Delta Live Tables DQ framework p