Build and optimise data pipelines using Java (preferred) or Python. Develop real-time streaming and batch data pipelines. Work with Kafka, Flink or Spark, with Kafka/Flink preferred.
Develop AWS-based data solutions using Kinesis, MSK, Lambda and Glue. Implement data governance, lineage, security and quality controls. Support CI/CD using Azure DevOps, GitHub Actions or Jenkins.
Use Terraform and Infrastructure as Code practices. Deploy containerised workloads using Docker, Kubernetes and AWS EK…