Senior AI Quality & Evaluation Engineer

Analityka rynku

Stack

Szczegóły

Bezpłatnie, po zalogowaniu.

Aplikuj

Prześlij CV — pracodawcy i rekruterzy z różnych krajów, którzy szukają takiego profilu, będą mogli napisać do Ciebie bezpośrednio.

Opis

Avenga is an international engineering firm helping businesses operate with AI at the core. With 6,000+ experts worldwide, we combine engineering expertise with AI-native thinking to turn ambitious ideas into real-world impact across industries, technologies, and markets. At the core of the role In Bulgaria, are looking for a highly skilled Senior AQA Engineer to design and build automated testing frameworks for AI-powered applications, with a strong focus on LLM evaluation, AI safety, RAG and agent testing, and integrating quality checks into CI/CD pipelines. Core skills you’ll bring 5+ years in software quality or test engineering, with strong Python (C#/.NET is a plus). You write test frameworks, not only test cases. Hands-on experience evaluating LLM applications: evaluation frameworks (for example Promptfoo, DeepEval, Ragas, Langfuse or LangSmith), building datasets, designing judges and metrics. Working knowledge of RAG and agent architectures and how they fail: retrieval quality, tool-call errors, loops, hallucination, context loss. Experience with adversarial or security testing, of AI systems or in general. CI/CD and infrastructure-as-code literacy; comfortable running tests on at least one of Azure, AWS or GCP. Clear written reporting: you turn findings into decisions. Full working proficiency in English. Nice-to-have Familiarity with the EU AI Act, ISO/IEC 42001 or the NIST AI RMF. Performance and load testing of LLM-backed services. Benchmarking open-weight models

Pełny opis, firma i link do aplikacji — po zalogowaniu. Zaloguj się

Kontakty pracodawcy (e-mail/telefon/Telegram) są ukryte w publicznym podglądzie. Wyślij CV — a pracodawcy będą mogli napisać do Ciebie sami.