Data Engineer

Обязанности: Build and maintain multi-layer pipelines (Landing → Raw → Enriched → Curated) on Azure Databricks (Spark, Delta Lake), Orchestrate pipelines via Databricks Jobs / Workflows: dependencies, retries, alerting, Implement batch and API-based data ingestion from multiple source systems, Apply schema enforcement and handle schema drift; run data quality checks at ingestion, Work hands-on with dbt for pipeline observability and data governance tasks, Monitor job failures, data freshness, and pipeline health against SLA-based delivery targets, Set up alerts, logging, and retry strategies for production pipelines, Work within a controlled Dev/Test/Prod environment: access control, audit trails, pipeline documentation, traceability to requirements, Contribute to CI/CD and MLOps/AI pipelines on Databricks where the project needs it

Опыт: Strong hands-on experience with Azure Databricks: Spark, Delta Lake, Build and maintain multi-layer pipelines: Landing → Raw → Enriched → Curated, Schema enforcement and schema-drift handling, Data quality checks at ingestion, Orchestration via Databricks Jobs / Workflows: dependencies, retries, alerts, Batch and API-based data ingestion, Strong hands-on experience with dbt, Monitor job failures, data freshness, and pipeline health, SLA-based data delivery, Alerts, logging, and retry strategies, Access control models, Audit trails, lineage, and traceability, Logging and evidence generation, Segregation of duties, Data contracts, Pipeline doc