Senior Data Engineer - Streaming and Data Platform (Contract) URGENT HIRING
- Methoden
- CI/CD
The client is a Singapore-based data consultancy building anon-premises data platform for a major data-centre operator across APAC. Each site produces telemetry from thousands of nodes at rates of up to millions of data points per second. This Senior Data Engineer will build the streaming pipelines that bring telemetry into the platform and establish the delivery tooling used across the wider data-engineering team.
This is an initial six-month contract. Any extension will depend on project requirements and is not guaranteed. Immediate Availability Immediate hiring priority: Priority will be given to Singapore Citizens and Permanent Residents who can start immediately or within two weeks.
is best suited to candidates between roles, nearing the end of a contract or project, or otherwise available at short notice. Extended notice periods are unlikely to meet the project timeline. Key
- Build and operate Kafka-to-Apache Iceberg streaming pipelines, beginning with alert history and documents and scaling across telemetry topics at three times current volume. · Design reliable Kafka consumers or Kafka Connect integrations with correct offset management, replay and failure recovery. · Run and tune Spark workloads in production across streaming and batch use cases. · Design and optimise Iceberg tables covering partitioning, file sizing, compaction and schema evolution. · Set up and maintain the team's data tooling, including dbt, Dagster or a comparable orche