Spark / Data Lakehouse Developer

Обязанности: Design and implement data reconciliation mechanisms within the Operational Data Store (ODS) layer., Develop integration and synchronization logic between ODS and source systems based on MongoDB., Build, optimize and maintain data pipelines with Spark and distributed processing frameworks., Design and develop data layers in Lakehouse architecture (from Raw through Curated to Gold zones)., Implement and maintain solutions using Delta Lake, Apache Iceberg or Apache Hudi., Ensure data quality, reliability, and monitoring across pipelines and storage layers., Collaborate closely with data engineers, analysts, and architects to deliver end to end data solutions., Participate in continuous improvement of the data platform architecture and engineering standards., Note: Detailed project information will be shared during the recruitment process.

Опыт: Proven experience as a Data Engineer / Spark Developer in big data or cloud ecosystems., Strong knowledge of Apache Spark, especially in data processing and ETL pipeline development., Hands on experience with MongoDB and ODS type data store architectures., Practical experience in Data Lakehouse technologies such as Delta Lake, Apache Iceberg or Hudi., Proficiency in Python and/or Scala, including data manipulation and pipeline scripting., Solid understanding of data modeling, data reconciliation, and data governance concepts., Experience working with distributed data environments and cloud platforms (Azure, AWS, or GCP)., Fam