Data Engineering Consultant

Analisi del mercato

Stack

Dettagli

Gratis, dopo l'accesso.

Candidati

Carica il tuo CV: datori di lavoro e recruiter di diversi paesi che cercano il tuo profilo potranno contattarti direttamente.

Descrizione

Job Summary We are seeking a Senior Data Engineer to design, develop, and optimize scalable data pipelines and processing solutions using Big Data, cloud, and AI technologies. Collaborate with architects, data scientists, and engineering teams to deliver high-performance, production-ready data systems. Responsibilities Design, develop, and maintain scalable batch and real-time data pipelines using Apache Spark/PySpark, SQL, and Python to support large-scale data processing Develop and optimize ETL/ELT pipelines for efficient data ingestion and transformation Build data ingestion solutions integrating databases, APIs, files, and streaming platforms to ensure reliable data flow Utilize AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda, and DynamoDB to deploy and manage cloud-based data solutions Develop and optimize data processing workflows using Hadoop, Hive, Spark, Kafka, Cloudera, and Databricks for high throughput and low latency Perform SQL and Spark performance tuning to enhance large-scale data workload efficiency Design and implement data models, data warehouses, and data lake architectures to support analytics and reporting needs Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes, and OpenShift to automate delivery and scaling Integrate Generative AI and NLP capabilities into enterprise data applications to enhance data insights and automation Develop solutions leveraging LLM frameworks, RAG, vector databases, and AI A

Descrizione completa, azienda e link per candidarti: dopo l'accesso. Accedi

I contatti del datore di lavoro (email/telefono/telegram) sono nascosti nell'anteprima pubblica. Invia il tuo CV così i datori di lavoro possono contattarti direttamente.