Senior Data Engineer

Análises de mercado

Stack

Grátis, depois de entrar.

Candidatar-se

Envie seu currículo: empregadores e recrutadores de vários países que procuram o seu perfil poderão falar com você diretamente.

Descrição

Accountabilities: - Design, develop, and optimize scalable ETL/ELT data pipelines for batch and real-time processing using Python, SQL, PySpark, Spark Structured Streaming, and Delta Live Tables. - Implement and maintain Medallion Architecture, organizing data into Bronze, Silver, and Gold layers to support data quality, performance, and analytical requirements. - Process terabyte-scale datasets using distributed computing techniques, ensuring pipeline reliability, scalability, and efficient resource utilization. - Manage Databricks environments, including workflows, clusters, Delta Lake, and related lakehouse components. - Implement Unity Catalog for metadata management, granular access controls, OAuth-based authentication, and enterprise-level data security and governance. - Develop and execute cloud migration strategies for legacy databases, including Oracle, Teradata, and SQL Server, to cloud-based data platforms and storage solutions such as Google Cloud Storage, Amazon S3, Azure Data Lake Storage, Snowflake, and Amazon Redshift. - Build and maintain CI/CD pipelines using Terraform and GitHub Actions to automate infrastructure provisioning, data pipeline deployments, testing, and monitoring. - Develop custom data processing scripts and data-driven APIs using Python and web frameworks such as Flask. - Use Docker to containerize data processing workloads and support portable, consistent deployment environments. - Evaluate emerging technologies and approaches for data-inten

Descrição completa, empresa e link para se candidatar: depois de entrar. Entrar

Os contatos do empregador (e-mail/telefone/telegram) ficam ocultos na prévia pública. Envie seu currículo para que os empregadores possam falar com você diretamente.