Kv Cache Engineer Jobs

1 job found

Today

Systems Software Engineer

Micron Richardson, TX

Richardson, TX

Today

Full time C/c++ Cuda Gpu Computing

Apply Now

Job Description

Micron’s Systems Software Engineering team is hiring a Principal Systems Software Engineer within the ITP Pathfinding organization’s Storage and Memory Systems (SMS) group. In this role, you will architect, design, and build software that helps validate and communicate the impact of future memory and storage technologies across AI applications, from hardware/software interfaces to inference workflows.

Role Summary

As a Principal Systems Software Engineer, you will enable proof-of-concept hardware capabilities by designing and evolving AI inference software stacks. You will also develop methods to quantify system feature value, lead architecture and development efforts, and contribute to research that identifies memory and storage system challenges and opportunities for AI workloads.

Responsibilities

Design and modify AI inference software stacks to enable proof-of-concept hardware capabilities.
Develop software to quantify and demonstrate the value of proposed system features.
Lead software architecture and development efforts independently and as a technical lead for small teams.
Research emerging AI software topics to identify memory and storage system challenges and opportunities.
Develop and modify Linux applications and libraries using C, C++, Python, Rust, CUDA, and/or SYCL for advanced heterogeneous computing systems.
Develop and use emulators to predict system-level performance and functional impacts before hardware availability.
Debug system-level issues across applications, libraries, operating systems, kernels, system configuration, networks, and hardware/software interfaces.
Use emerging AI tools to improve productivity.
Collaborate across Micron organizations and with external partners.
Mentor interns and earlier-career engineers.
Present technical findings, insights, and innovations to internal or external customers.

Required Qualifications

Bachelor’s or Master’s degree with 10+ years of experience, or a PhD with 7+ years of experience, in Computer Science, Software Engineering, Computer Engineering, or a related field.
Working knowledge of end-to-end AI inference software stack architectures, with hands-on development experience across layers such as frameworks, inference engines, ML compilers, compiler infrastructure, hardware runtimes, or hardware abstraction layers.
Experience debugging issues spanning applications, libraries, operating systems, kernels, networks, and hardware/software interfaces.
Proficient development skills in C, C++, and Python, with Linux-specific programming experience including multithreading, IPC, ioctl, GDB, and interaction with /proc and /sys filesystems.
Experience programming GPUs or other accelerators using CUDA, SYCL, OpenCL, or a comparable programming model.
Demonstrated ability to lead technical initiatives without supervisory authority.
Familiarity with LLM serving internals, including KV-cache management, prefill and decode phases, KV-cache offloading, and disaggregated serving.
Familiarity with inference engines and distributed serving frameworks such as SGLang, vLLM, NVIDIA Dynamo, or TensorRT-LLM.
Strong understanding of heterogeneous compute architecture concepts, including memory hierarchies, NUMA, caching, accelerator architectures, and interconnects such as PCIe, CXL, NVLink, or UALink.

Technologies

C, C++, Python, Rust
CUDA, SYCL, OpenCL
Linux, GDB
/proc, /sys
Multithreading, IPC, ioctl
KV-cache management, prefill and decode phases, KV-cache offloading, disaggregated serving
SGLang, vLLM, NVIDIA Dynamo, TensorRT-LLM
NUMA, PCIe, CXL, NVLink, UALink
Emulators, ML compilers

Preferred Qualifications

Technical leadership experience in a systems-focused pathfinding or research organization.
Technical knowledge of HBM, DRAM, and emerging memory technologies.
Technical knowledge of NVMe SSDs, NAND, and next-generation storage systems.

Location and Experience

Richardson, TX (onsite). Minimum experience: 7 years.

View full job page →

Get Kv Cache Engineer Jobs alerts

New Kv Cache Engineer Jobs, daily.