Snowflake Data Engineer (Airflow)
- Doświadczenie
- od 3 lat
- Angielski
- B2, wymagany
- Praktyki
- CI/CD
Develop and maintain data pipelines using Apache Airflow and Python. Build data ingestion processes from APIs, databases, and file-based sources into AWS S3 and Snowflake. Design and implement data transformations using dbt.
Optimize Snowflake queries, warehouse performance, and resource consumption. Implement pipeline monitoring, error handling, and alerting mechanisms. Ensure data quality, consistency, and reliability through automated testing and validation.
Maintain technical documentation and follow data engineering best practices.
3+ years of professional experience in Data Engineering. Strong expertise in Apache Airflow , including DAG development, operators, sensors, hooks, TaskFlow API, and workflow optimization. Advanced Python skills, including writing clean, maintainable, and testable code.
Advanced SQL knowledge, including complex queries, CTEs, window functions, and performance optimization. Hands-on experience with Snowflake , including warehouses, stages, external volumes, and data loading processes. Practical knowledge of dbt , including models, tests, macros, snapshots, and incremental transformations.
Experience with AWS S3 and Data Lake architecture, including data partitioning, file formats, and access management. Familiarity with Git and CI/CD practices. English proficiency at B2 level or higher.
Nice to Have Experience with Docker and Kubernetes. Knowledge of Apache Iceberg or Delta Lake. F