Senior Machine Learning Engineer, SageMaker Inference

Are you interested in helping AWS customers by building ML-distributed systems to deploy Generative AI models for inferencing at scale? SageMaker is a fully managed service to prepare data and build, train, and deploy machine learning (ML) models for any use case with fully managed infrastructure, tools, and workflows. Specifically, the SageMaker Inference Platform enables customers to deploy ML models at scale to run high throughput, low-latency CPU/GPU/Inferentia workloads and monitor the quality of the models and data in production.

The team is currently focused on solving core infrastructure problems for Generative AI inference serving needs of both internal and external customers. We're moving fast, and this is a great team to join and make a huge impact on AWS especially in the Generative AI space. To learn more about the product, please visit [link] <br/><br/>Key job responsibilities<br/>- Design and implement kernel-level optimizations and inference pipeline improvements that reduce latency and increase throughput for LLM serving workloads.<br/>- Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains.<br/>- Write well-organized, maintainable code and participate actively in code reviews, providing constructive feedback to peers and mentoring newer engineers on the team.<br/>- Collaborate with applied scientists, ML e