Data Engineer

The Applications Development Programmer Analyst Citi is seeking a Applications Development Programmer Analyst 2 (C10) to join its Technology – Data Engineering function, focused on the PySpark / Spark-based Data Platform. In this role, you will design, develop, and maintain scalable data pipelines and analytical solutions, supporting large-scale data operations across enterprise systems. You will work within a full software development lifecycle (SDLC) environment spanning Development (NAM), UAT, and Production.

activities. Core Technical Duties Design, develop, test, and maintain PySpark and Apache Spark data pipelines for batch and streaming workloads. Manage and monitor Spark Streaming Jobs (running Sunday evening through Saturday afternoon) performing Eligibility and Data Quality (DQ) checks, publishing data to Kafka sender topics, and processing exception/XMEV requests.

Manage and monitor Spark Batch Jobs (running hourly or every 2 hours) that store audit, inbound, and outbound data into Hive tables (e.g., gfolyreg_managed.SWXPTT_audit). Respond to ITRS Alerts and system monitoring events (e.g., job failures, kills, or jobs not triggered within 30 minutes of scheduled time). Work with large-scale data sources across Oracle, Hadoop, Impala, and SAS Server.

Configure and manage JDBC connections from SAS, Python, and R to dedicated Sandbox/data platform environments. Collaborate on schema creation during code deployment; raise CMP tickets for permission grants and country-s