ML Platform Engineer, Distributed Training

Аналитика рынка

Стек

Бесплатно. Вход по почте.

Откликнуться

Загрузите резюме — мы свяжем вас с работодателем напрямую через нашу базу.

Описание

Accountabilities - Optimize the training platform: Partner with infrastructure teams to improve GKE-based training environments, including GPU node pools, job scheduling, queueing, resource quotas, and capacity management. - Diagnose platform failures: Investigate underlying causes of infrastructure and training issues, resolve systemic problems, and implement durable fixes rather than relying on temporary workarounds. - Tune GPU networking: Optimize multi-node GPU communication and training throughput using technologies and techniques such as RDMA, GPUDirect, NCCL, and GPU interconnect topology. - Run distributed training workloads: Execute, scale, profile, and debug multi-node training jobs using PyTorch-based frameworks such as FSDP, DeepSpeed, Megatron, or NeMo, alongside Ray Train. - Identify performance bottlenecks: Analyze compute utilization, memory consumption, input pipelines, and network communication to improve training efficiency and scalability. - Design performance benchmarks: Develop and run benchmarks that measure throughput, scaling efficiency, and cost per training run, using the results to guide engineering decisions. - Build model evaluation systems: Create and maintain evaluation harnesses that objectively measure the impact of changes to training recipes, models, and platform configurations. - Improve observability and testing: Instrument training workloads and platform components with telemetry and monitoring, and maintain end-to-end pipeline tests tha

Полное описание, компания и ссылка на вакансию — после входа. Войти

Контакты работодателя (email/phone/telegram) скрыты из публичного превью — отправьте резюме, чтобы мы связали вас напрямую.