Data Engineering Consultant

Markteinblicke

Stack

Details

Kostenlos, nach der Anmeldung.

Bewerben

Laden Sie Ihren Lebenslauf hoch — Arbeitgeber und Recruiter aus verschiedenen Ländern, die Ihr Profil suchen, können Sie direkt kontaktieren.

Beschreibung

Job Description We are looking for a highly skilled and experienced Senior Data Engineer to design, develop and optimize scalable data engineering solutions using Big Data, cloud and modern AI technologies. The successful candidate will be responsible for developing high-performance data pipelines, data ingestion frameworks and data processing solutions, while working closely with architects, business stakeholders, data scientists and engineering teams. Key Responsibilities Design, develop and maintain scalable batch and real-time data pipelines using Apache Spark/PySpark, SQL and Python. Develop and optimize ETL/ELT pipelines for large-scale data processing. Build data ingestion solutions using databases, APIs, files and streaming platforms. Work with AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda and DynamoDB. Develop and optimize data processing solutions using Hadoop, Hive, Spark, Kafka, Cloudera and Databricks. Perform SQL and Spark performance tuning and optimize large-scale data processing workloads. Design and implement data models, data warehouses and data lake solutions. Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes and OpenShift. Integrate Generative AI and NLP capabilities into enterprise data applications where required. Develop solutions using LLM frameworks, RAG, vector databases and AI APIs. Collaborate with solution architects, data scientists, software engineers and business stakeholders to deliver pr

Vollständige Beschreibung, Unternehmen und Bewerbungslink — nach der Anmeldung. Anmelden

Kontaktdaten des Arbeitgebers (E-Mail/Telefon/Telegram) sind in der öffentlichen Vorschau ausgeblendet. Senden Sie Ihren Lebenslauf — dann können Arbeitgeber Sie direkt kontaktieren.