DATA ENGINEER

Markteinblicke

Stack

Details

Kostenlos, nach der Anmeldung.

Bewerben

Laden Sie Ihren Lebenslauf hoch — Arbeitgeber und Recruiter aus verschiedenen Ländern, die Ihr Profil suchen, können Sie direkt kontaktieren.

Beschreibung

Key Responsibilities Design, develop and maintain scalable data pipelines and data ingestion frameworks for large-volume datasets. Develop data transformation and processing applications using Apache Spark, PySpark, Scala and Python . Build and optimize data pipelines using Azure Databricks, Azure Data Factory, AWS EMR and related cloud services. Work with Hadoop, HDFS, Hive, Snowflake, Teradata and Data Lake environments. Develop batch and real-time data processing solutions using Spark Structured Streaming and Kafka . Perform data extraction, transformation and loading across heterogeneous source and target systems. Develop and optimize Spark SQL, HiveQL and SQL queries for performance and cost efficiency. Design data models, partitioning strategies and scalable data storage architectures. Build and manage workflow orchestration using Apache Airflow . Implement CI/CD pipelines and automated testing using tools such as Jenkins, Docker, GitHub Actions and pytest . Troubleshoot data pipeline, performance and production issues and implement sustainable solutions. Collaborate with business stakeholders, architects and technology teams to understand requirements and deliver data engineering solutions. Ensure data quality, reliability, security and operational stability across enterprise data platforms. Required Skills 6+ years of experience in Data Engineering / Big Data Engineering . Strong hands-on experience with Apache Spark / PySpark . Strong programming skills in Python and

Vollständige Beschreibung, Unternehmen und Bewerbungslink — nach der Anmeldung. Anmelden

Kontaktdaten des Arbeitgebers (E-Mail/Telefon/Telegram) sind in der öffentlichen Vorschau ausgeblendet. Senden Sie Ihren Lebenslauf — dann können Arbeitgeber Sie direkt kontaktieren.