Python Engineer – Web Crawling & Data Extraction (m/f/d)

Markteinblicke

Stack

Kostenlos, nach der Anmeldung.

Bewerben

Laden Sie Ihren Lebenslauf hoch — Arbeitgeber und Recruiter aus verschiedenen Ländern, die Ihr Profil suchen, können Sie direkt kontaktieren.

Beschreibung

About the Job We build and operate large-scale web crawling and extraction systems that turn unstructured, dynamic websites into clean, structured and correct data. You'll own crawlers end-to-end: discovery, resilient fetching against real anti-bot defenses, and turning HTML/PDF into trustworthy structured output. What you will do Build and scale crawlers that handle dynamic, large-scale sites, and keep them running as those sites change and fight back. Stay ahead of anti-bot measures: TLS/browser fingerprinting, proxies, session and rate-limit strategy, and change detection. Turn raw HTML and PDF into clean, correct structured data (including LLM-assisted extraction), where silently wrong output is worse than a crash. Design and operate pipelines for ingestion, deduplication, versioning and enrichment. Ship and run containerized apps in the cloud, with CI/CD. Shape our technical architecture and engineering culture. Our Tech Stack Python, Docker, GCP (Cloud Run jobs), Azure Blob Storage, MongoDB. Go services and a React frontend on the periphery. You don't need all of it on day one, but you do need to ramp up fast. Your profile 3+ years of professional software development (5+ for the senior track). Strong Python: real production code, async. Hands-on web scraping at scale, including firsthand experience with anti-bot defenses and the failure modes of large-scale collection. A data-correctness mindset: you care that extracted data is right, not just that the job finished.

Vollständige Beschreibung, Unternehmen und Bewerbungslink — nach der Anmeldung. Anmelden

Kontaktdaten des Arbeitgebers (E-Mail/Telefon/Telegram) sind in der öffentlichen Vorschau ausgeblendet. Senden Sie Ihren Lebenslauf — dann können Arbeitgeber Sie direkt kontaktieren.