Lead Data Engineer

We are seeking a skilled and detail-oriented Data Engineer to design, develop, and maintain robust data pipelines and ETL solutions. This role involves working closely with cross-functional teams to ensure data quality, scalability, and alignment with business and technical requirements. Design, develop, test, and maintain scalable ETL pipelines to meet business, technical, and user requirements.

Collect, refine, and integrate new datasets. Maintain comprehensive documentation and data mappings across multiple systems. Create optimized and scalable data models that align with organizational data architecture standards and best practices.

Conduct code reviews and perform rigorous testing to ensure high-quality deliverables. Drive continuous improvement in data quality through optimization, testing, and solution design reviews. Ensure all solutions conform to big data architecture guidelines and long-term roadmap.

Implement robust monitoring, logging, and alerting systems to ensure pipeline reliability and data accuracy. Apply best practices in data engineering to design and build reliable data marts within the Hadoop ecosystem for planning, reporting, and analytics. Maintain and optimize data pipelines to ensure data accuracy, integrity, and timeliness.

Manage code in a centralized repository with clear branching strategies and well-documented commit messages. Coordinate with stakeholders to ensure smooth production deployment and adherence to data governance policies. Proacti