Sglang Engineer Jobs

1 job found

Today

Amazon Development Center U.S., Inc. Santa Clara, CA

Santa Clara, CA

Today

$165k - $224k/yr Full time Artificial Intelligence AWS Aws Bedrock

Machine learning engineer on an AWS inference optimization team focused on improving large language model inference performance.

Develop model servers for large language OSS models (examples include Qwen, Gemma, gptoss, and others)
Build inference stack components, including CUDA kernel development in vLLM and SGLang
Create and iterate on services for GenAI optimization and benchmarking
Create and modify a mid-size application or service
Apply inference optimization techniques to improve performance, including:
- Speculative decoding
- Disaggregated Inference
- CUDA kernel tuning
Own price-performance for Amazon LLM inference on SageMaker, Hyperpod, and Bedrock
Deliver production-ready solutions for customer-facing products, using existing academic/industrial research and practical problem solving
Optimize inference performance using AWS inference stacks including vLLM and SGLang
Tune GenAI inference performance on Amazon SageMaker, Amazon Bedrock, and Hyperpod

3+ years of non-internship professional software development experience
2+ years of non-internship experience designing or architecting new and existing systems (design patterns, reliability, scaling)
1+ years of software development engineer or related occupational experience
1+ years building large-scale, multi-tiered, multi-threaded, embedded, or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl
Bachelor’s degree (Computer Science, Engineering, Mathematics, or related field) or foreign equivalent
Knowledge of Machine Learning and LLM fundamentals, including:
- Transformer architecture
- Training and inference lifecycles
- Optimization techniques

3+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
Bachelor’s degree in computer science or equivalent

Sign-on payments
Restricted stock units (RSUs)
Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line)
Flexible Spending Accounts
Adoption and Surrogacy Reimbursement coverage
401(k) matching
Paid time off
Parental leave
Workplace accommodation or adjustment during the application and hiring process (including support for interview or onboarding)

Work safely and cooperatively with employees, supervisors, and staff
Adhere to standards of excellence despite stressful conditions
Communicate effectively and respectfully to ensure exceptional customer service
Follow all federal, state, and local laws and company policies
Criminal history may have a direct, adverse, and negative relationship with some material job duties, including the abilities to follow policies, exercise sound judgment, manage stress, work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the company’s reputation
Under the Los Angeles County Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered

Location: Santa Clara, CA (onsite)

Compensation: USD 165,200 - 223,600 per year

New Sglang Engineer Jobs, daily.