Data Engineer (Databricks)

Job Summary The Data Engineer will design, develop, and maintain scalable, reliable data pipelines on Databricks and cloud platforms, integrating diverse data sources to support analytics, reporting, and machine learning. Collaborate with cross-functional teams to enhance data platform governance, monitoring, and reliability.

Design, develop, and maintain ETL pipelines for centralized data storage systems such as Delta Lake to ensure efficient data ingestion and transformation Integrate data from databases, APIs, log files, streaming platforms, and external providers to support diverse analytics needs Develop data transformation routines to clean, normalize, and aggregate complex or inconsistent datasets for accurate analysis Apply data processing techniques to handle large-scale and varied data sources effectively Contribute to the development and enforcement of frameworks and best practices for code development and deployment to ensure quality and consistency Implement data governance policies aligned with company standards to maintain data integrity and compliance Collaborate with analytics and product teams to design and operationalize data pipelines that meet business requirements Work with infrastructure teams to advance cloud-based data platforms leveraging Azure and Databricks technologies Explore and evaluate new tools and techniques to enhance data platform capabilities on Azure, Databricks, or related platforms Monitor data pipelines continuously t