Data Engineer - up to SGD15k
- Experience
- 5+ years
- Practices
- CI/CD
We are seeking a Senior Data Engineer . In this position, you will architect, develop, and fine‑tune high‑throughput data systems that power scalable analytics, network telemetry processing, and advanced AI/RAG solutions. You’ll work closely with data scientists, analysts, and system owners to translate complex business requirements into secure, automated, and resilient data pipelines.
Key
Build scalable ingestion, transformation, and serving capabilities across a hybrid cloud environment, ensuring reliability, security, and compliance with enterprise standards. Optimise batch and streaming workflows using Databricks and Kafka, applying best practices for performance, resilience, and maintainability. Deliver high‑quality, secure data to support reporting, machine learning, and GenAI use cases.
Lead the setup and operationalisation of knowledge bases and retrieval‑augmented generation stacks for agentic AI applications. Perform cleansing and transformation using PySpark or SQL based on business requirements. Monitor, troubleshoot, and ensure pipeline reliability, while owning production readiness, observability, incident triage, and release coordination.
Provide guidance on reusable components, code quality, deployment readiness, and production support practices. Connect diverse data sources (files, APIs, databases, streams), maintain metadata, and ensure transparency through documentation. Work with Microsoft Fabric, Databricks, Delta Lake, and other platfor