Senior Data Engineer

Markteinblicke

Stack

Details

Kostenlos, nach der Anmeldung.

Bewerben

Laden Sie Ihren Lebenslauf hoch — Arbeitgeber und Recruiter aus verschiedenen Ländern, die Ihr Profil suchen, können Sie direkt kontaktieren.

Beschreibung

Key Responsibilities Data Management: Extract, combine, transform, load, manipulate, and cleanse data from various sources to ensure high-quality datasets for analysis. This also includes data migration from legacy systems into next generation systems Collaboration: Work closely with data modelers and business analysts to understand the data model and source-to-staging data mapping. Data Structures: Create and integrate data structures suitable for analysis, including developing DDLs to create various MS Cloud SQL Server tables and views in Pond (a subset of the Data Lake) DB. Databricks Development: Design and develop Databricks notebooks for extracting data from various input files, joining related entities to create Shared File Formats (SFFs). Ingestion Framework: Configure the Ingestion Framework (IF) tables to utilize the existing framework for loading staging and Data Lake Core (DLC) tables. Algorithm Development: Develop and deploy algorithms in performance-efficient code across analytical and data platforms. Data Pipelines: Build large-scale batch and real-time data pipelines using data processing frameworks in cloud platforms or on-premises environments. Continuous Integration: Leverage best practices in continuous integration and delivery to enhance data workflows. Data Governance: Track data consumption patterns, collaborating with data governance and information stewards, and participate in vetting and promoting content for the curated data catalog. Automation: R

Vollständige Beschreibung, Unternehmen und Bewerbungslink — nach der Anmeldung. Anmelden

Kontaktdaten des Arbeitgebers (E-Mail/Telefon/Telegram) sind in der öffentlichen Vorschau ausgeblendet. Senden Sie Ihren Lebenslauf — dann können Arbeitgeber Sie direkt kontaktieren.