We are looking for a Data Engineer to design, build, and maintain scalable data pipelines on AWS and Databricks. You will combine data engineering with DevOps practices (CI/CD, infrastructure as code, monitoring) to deliver reliable, automated, and secure data platforms that support analytics, reporting, and machine learning. Key
Design and develop batch and streaming data pipelines using Databricks (PySpark, Spark SQL, Delta Lake) on AWS Build and optimize ETL/ELT workflows to ingest data from databases, APIs, files, and event streams Implement lakehouse architecture using Delta Lake, Unity Catalog, and the medallion model (bronze/silver/gold) Manage and orchestrate workflows with Databricks Workflows, Airflow, or AWS Step Functions Provision and manage cloud infrastructure using Terraform or CloudFormation Build and maintain CI/CD pipelines for data code and infrastructure (GitHub Actions, GitLab CI, Jenkins, or Azure DevOps) Implement monitoring, logging, and alerting for pipelines and platform health (CloudWatch, Datadog, etc.) Optimize Spark jobs and cluster configurations for performance and cost Enforce data quality, governance, security, and access control standards Collaborate with data scientists, analysts, and business stakeholders to translate requirements into technical solutions Document pipelines, architecture, and operational runbooks Required Skills and
Bachelor’s degree in Computer Science, Engineering, or a related field Stro