Data Engineer (Azure, Databrick)

Аналитика рынка

Стек

Откликнуться

Загрузите резюме — мы свяжем вас с работодателем напрямую через нашу базу.

Описание

Key Responsibilities Data Management & Transformation Design, develop and maintain data pipelines that extract data from business applications, databases and external systems. Implement data quality checks and validation processes to ensure data accuracy, consistency and completeness. Perform data cleansing, aggregation, filtering, transformation and enrichment. Establish and maintain governance frameworks for data and algorithms used in analytics, automated decision-making and analytical applications. Develop and maintain logical and physical data models, including data structures, relationships and constraints. Ensure data solutions comply with security, governance and organisational best practices. Data Platform Optimisation & Automation Develop and maintain CI/CD pipelines for data deployments and cloud resource provisioning. Optimise data pipelines and processing workflows for performance, scalability and efficiency. Optimise data models and algorithms for data quality, security, governance, performance and scalability. Monitor data warehouse and ETL platform capacity, including capacity planning and forecasting. Monitor and tune data platforms, troubleshoot performance bottlenecks and implement optimisation techniques such as caching and indexing. Monitor platform resource and credit consumption and perform regular housekeeping activities. Support the deployment, integration and maintenance of AI/ML solutions within the data platform. Collaboration & Business Enablement Partner with the Data Lead and business users to manage and maximise the value of data as a business asset. Work with stakeholders to understand data requirements and translate them into scalable technical solutions. Guide business users in the creation and maintenance of reports and dashboards. Collaborate with technology, data and business teams to improve data quality, accessibility and usability. Any other ad-hoc duties as assigned by supervisor Requirements 5+ years of experience in data management or data engineering, covering areas such as data integration, data modelling, data quality and platform optimisation. Hands-on experience designing and implementing data architectures using Azure and Databricks. Strong programming skills in Python, PySpark and SQL . Strong experience with relational and non-relational databases , including SQL and NoSQL databases. Hands-on experience with Azure DevOps and Databricks LakeFlow Jobs , including version control, CI/CD and pipeline deployment. Experience with data catalogue and governance tools such as Databricks Unity Catalog and Microsoft Purview . Experience supporting AI/ML model deployment, feature engineering and production inference is an advantage. Interested applicants, please email your resume to Lisa (Lisha) Zhang Email: [email] CEI Reg No: R25157448 EA Licence No: 99C4599 Recruit Express Pte Ltd

Контакты работодателя (email/phone/telegram) скрыты из публичного превью — отправьте резюме, чтобы мы связали вас напрямую.