Senior Infrastructure Security Engineer
Job Description
As a Senior Infrastructure Security Engineer, you will secure fal.ai’s compute platform at layers below the application, spanning Linux kernel, hypervisors and sandboxes, GPU drivers and firmware, and host systems. The role focuses on hardening tenant isolation, evaluating vulnerabilities, and shaping the host and isolation security roadmap in collaboration with GPU vendors and upstream maintainers.
Responsibilities
- Build and maintain a threat model for running code on shared GPU hosts, covering containers and microVMs down to PCIe, IOMMU, and GPU passthrough.
- Design and validate GPU memory and device-state scrubbing between tenants.
- Support hardware-backed isolation initiatives where applicable, including confidential computing and attestation.
- Harden kernel builds and host images by applying configuration minimization, LSMs, seccomp, and namespace and cgroup policy, along with secure and measured boot.
- Reduce attack surface across GPU drivers, CUDA runtimes, and interconnect fabrics such as NVLink and NVSwitch, plus BMC and firmware components.
- Assess and triage security issues in the kernel, hypervisors, GPU drivers, and firmware, including judgment of exploitability and real-world impact, severity assignment, and driving fixes or mitigations.
- Monitor vendor bulletins and upstream disclosures, and perform kernel and driver patching that keeps pace without disrupting the fleet.
- Conduct vulnerability research on the dependencies and components relied upon.
- Contribute to the host and isolation security roadmap and evaluate security tooling.
- Partner with Detection & Response on telemetry and serve as the escalation point for host, kernel, and driver incidents.
- Review designs from Infra, Platform, and ML teams, and help establish the secure implementation as the default path.
- Work with GPU vendors, cloud providers, and upstream maintainers to coordinate disclosure and fixes.
Required Qualifications
- 5+ years in systems or security engineering, including deep hands-on work in the Linux kernel, hypervisors, or device drivers.
- Fluency in kernel C, with comfort using Rust, Go, or Python for tooling.
- Strong understanding of virtualization internals including KVM, VFIO, IOMMU/DMA protection, and tradeoffs among containers, sandboxes, and microVMs.
- Experience shipping hardening work that operates in production at fleet scale.
- Proven ability to find, triage, or fix real vulnerabilities in low-level code and explain the exploit path.
- Ability to communicate risk clearly to both kernel engineers and executives.
Technologies
- Linux kernel, hypervisors, microVM
- PCIe, IOMMU, GPU passthrough
- LSMs, seccomp, namespace and cgroup policy, secure and measured boot
- GPU drivers, CUDA runtimes, NVLink, NVSwitch
- BMC, kernel C
- Rust, Go, Python
- KVM, VFIO, IOMMU/DMA protection
- containers, sandboxes
- LLMs, agents, AI coding tools
- NVIDIA GPU stack, open kernel modules, GSP firmware, MIG, vGPU
- GPU confidential computing
- AMD SEV-SNP, Intel TDX, attestation
- confidential computing
Location and Employment
- Location: San Francisco, CA (onsite)
- Employment type: Full time
- Department: Engineering
Compensation
- Salary range: USD 180,000 - 250,000 per year
- Equity: Offers Equity
- Compensation range: $180K - $250K • Offers Equity
- Additional notes: $180,000 - $250,000 + equity + comprehensive benefits package
Benefits
- Health, dental, and vision insurance (US)
- Visa sponsorship and assistance relocating to San Francisco
- Regular team events and offsites
- Interesting and challenging work
- A lot of learning and growth opportunities
Nice-to-Haves / Qualifications
- AI fluency: uses LLMs, agents, and AI coding tools daily to review code, build tooling, and accelerate vulnerability research.
- NVIDIA GPU stack experience: open kernel modules, GSP firmware, MIG, vGPU, or GPU confidential computing.
- Confidential computing: AMD SEV-SNP, Intel TDX, attestation.
- Published CVEs, talks, or write-ups on kernel or hypervisor security.
- Prior work at a GPU cloud, hyperscaler, or chip vendor.
Similar Jobs
S
S
S
S