Description
Key
Responsibilities
•Design, develop and maintain scalable ETL/ELT data pipelines for large-volume structured and unstructured datasets. •Develop high-performance data processing solutions using Python, PySpark, Apache Spark, Spark SQL and Scala. •Build batch and real-time data pipelines using Kafka, Kinesis, Spark Streaming, AWS Glue and Airflow.
•Develop data ingestion frameworks integrating RDBMS, APIs, files, cloud storage and streaming platforms. •Design and implement data lakes, data warehouses and cloud-based data processing platforms. •Work with Databricks, Hadoop, Hive, Cloudera, Presto and Snowflake for large-scale data processing and analytics.
•Perform data modelling, data transformation, data quality, query optimisation and performance tuning. •Develop and optimise SQL solutions across Oracle, SQL Server, PostgreSQL, Teradata, MongoDB and cloud databases. •Design and implement data migration solutions involving large-scale enterprise datasets.
•Develop and support real-time and batch processing architectures for enterprise applications. •Integrate data platforms with REST APIs, GraphQL and enterprise applications. •Implement CI/CD and DevOps practices using Jenkins, Git, Docker, Kubernetes and OpenShift.
•Develop cloud-native data solutions using AWS and Azure, including S3, Glue, EMR, Redshift, Kinesis, Lambda, RDS and DynamoDB. •Develop AI/GenAI-enabled data solutions involving LLMs, NLP, RAG, Agentic AI and vector databases. •Integrate LLM services and AI pla
Employer contacts (email/phone/telegram) are hidden from the public preview —
send your CV, and we will connect you directly.