Senior Scala Engineer

About the team: As part of the Data & AI area, we implement projects based on the practical "data science" and "artificial intelligence" applications of an unprecedented scale in Poland. Data & AI is a group of over 150 experienced engineers organized into over a dozen teams with various specializations. Some of them build dedicated tools for creating and launching BigData processes or implementing ML models for the entire organization.

Others work closer to the client and are responsible for the implementation of the search engine, creating recommendations, building a buyer profile or developing an experimental platform. There are also research teams in the area whose aim is to find solutions to non-trivial problems requiring the use of machine learning. We are looking for Senior Scala Engineers who want to build a highly scalable and fault-tolerant data ingestion for millions of Allegro customers.

The platform collects 5 billion clickstream events every day (up to 150k / sec) from all Allegro sites and Allegro mobile applications. This is a hybrid solution using a mix on-premise and Google Cloud Platform (GCP) services like Spark, Kafka, Beam, BigQuery, Pubsub or Dataflow. We are looking for people, who: Are programming in languages such as Scala or Java, Python Have strong understanding of distributed systems, data storage, and processing framework like dbt, Spark or Apache Beam Have knowledge of GCP (especially Dataflow and Composer) or other public cloud environments lik