Data Engineer - Data Lakehouse Architecture (Public Sector)

Market insights

Stack

Apply

Upload your CV — we will connect you with the employer directly through our pool.

Description

1-year contract, renewable Government project Hybrid work arrangement Role Overview The Data Engineers will design, build and deploy a scalable and reliable data lakehouse platform on AWS. Their responsibilities will include developing end-to-end data pipelines, establishing data quality frameworks and delivering production-ready solutions to support the organisation's data infrastructure. The Senior Consultant will additionally lead complex engineering activities and provide technical guidance and implementation oversight. Key Responsibilities Data Pipelines and Architecture Design and implement scalable ETL/ELT pipelines using AWS Glue, Step Functions, Lambda and S3. Build data lakehouse solutions using Apache Iceberg or Amazon S3 Tables, incorporating schema evolution, partition evolution and ACID transactions. Optimise pipelines for performance, cost and reliability. Data Quality and Governance Develop and maintain automated data quality validation frameworks. Establish metrics and monitoring to ensure data accuracy, completeness and consistency. Implement data governance controls and ensure compliance with applicable standards. Application Development and Deployment Write production-quality code and deploy solutions on AWS. Implement CI/CD pipelines and infrastructure as code using tools such as Terraform or CloudFormation to support repeatable and auditable deployments. Collaboration and Handover Work with cross-functional teams and communicate technical designs clearly to technical and non-technical stakeholders. Prepare technical documentation and support handover to the Day 2 operations team. Qualifications and Experience A degree in Computer Science, Data Engineering, Information Systems or a related discipline. Five to eight years of experience in data engineering, ETL/ELT development or data platform roles. Strong hands-on experience developing and deploying production solutions using AWS services. Proficiency in Apache Iceberg or similar open table formats. Experience in a Government Commercial Cloud (GCC) environment is strongly preferred. Relevant AWS certifications are an advantage. Skills and Competencies Strong understanding of data lakehouse architecture, Apache Iceberg features and Amazon S3 Tables optimisation. Ability to design reliable pipelines and implement automated data quality and governance controls. Proficiency in production-quality coding, performance optimisation and cloud deployment. Ability to troubleshoot complex engineering issues and document solutions clearly. /Ref: T24-251

Employer contacts (email/phone/telegram) are hidden from the public preview — send your CV, and we will connect you directly.