We are seeking an experienced Data Engineer with 8+ years of expertise in Big Data Engineering, Databricks, Apache Spark, PySpark, Scala, and SQL . The role focuses on building and delivering scalable data solutions using Databricks, Delta Lake, Lakehouse architecture, AWS, and Power BI in a fast-paced Agile environment.
The role involves designing, developing, and supporting modern data engineering solutions using Databricks and Apache Spark , with strong emphasis on PySpark, Scala, Python, SQL, and data modeling. The candidate will work with Delta Lake and Lakehouse architecture to build scalable distributed data processing solutions.
The position requires hands-on experience with AWS Big Data services , including S3, Glue, EMR, DMS, Athena, RDS, Redshift, and Kinesis, as well as experience managing EMR clusters and working with relational databases such as PostgreSQL, Oracle, and SQL Server.
The candidate will also contribute to data orchestration, streaming, CI/CD, source control, data cataloguing, and Power BI solutions. Experience with Agile at Scale, Continuous Integration, Test-Driven Development, and Infrastructure as Code practices will support successful delivery.