Data Engineer- Quantexa

We are seeking an experienced Data Engineer with excellent expertise in big data platforms, containerized environments, and DevOps practices. In this role, you will design, build, and maintain scalable data pipelines that process large-scale datasets supporting advanced analytics and machine learning solutions. You will primarily work on the Quantexa platform within the AML domain, ensuring high availability, performance, and reliability of data infrastructure deployed on OpenShift Container Platform (OCP).

Key

Data Engineering & Analytics Design, implement, and maintain data transformation, aggregation, and enrichment pipelines to support analytics and machine learning initiatives. Process and manage large-scale datasets using Hadoop, Spark, and related technologies. Ensure data quality, integrity, and consistency across the entire data lifecycle.

Implement data governance practices, including data lineage, metadata management, and compliance controls. Platform & Containerization Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques. Optimize data workflows for containerized deployment and efficient resource utilization.

Monitor, tune, and enhance data pipeline performance to ensure scalability and reliability. DevOps & Operations Collaborate with DevOps teams to streamline deployments and implement CI/CD pipelines. Implement monitoring and logging solutions to ensure health, av