Staff, Robotics ML/Data Engineer

Persona AI Inc is seeking a Staff-level Robotics ML/Data Engineer to architect and scale multimodal robotics data pipelines. The role focuses on transforming raw, in-the-wild egocentric video and dense sensor streams into high-fidelity training assets that directly support foundation model development.

Responsibilities

• Design cross-modal validation systems that verify agreement between video, proprioception, force/haptic signals, and language annotations. Examples include reprojecting robot state into the image plane to confirm video-state consistency, and using VLM-assisted checks to assess whether instructions align with observed behavior.
• Orchestrate hand-tracking, segmentation, depth estimation, 3D reconstruction, and pose-tracking components. Retarget human demonstrations into robot trajectories, and run simulation-in-the-loop validation using kinematic feasibility, physics replay, and motion-consistency filtering to ensure synthesized data is physically grounded rather than only visually plausible.
• Implement robust data augmentation methods to expand expert trajectories and improve learning robustness, including spatial transformations, temporal scaling, synthetic viewpoints, and sensor noise injection.
• Create unified state-action representations across different embodiments, coordinate frames, rotation conventions, gripper and hand parameterizations, and sampling rates. Apply per-dimension validity masking and per-source normalization so that onboarding a new robot or sensor becomes a configuration update instead of a code rewrite.
• Build dataset tooling that enables researchers to query, visualize, and audit data, including clip browsers, trajectory viewers, and annotation review user interfaces. Translate model-failure analysis into new curation rules and targeted re-collection requests.
• Architect end-to-end ingestion pipelines that convert raw, unstructured sources into indexed, queryable, training-ready datasets. Include temporal segmentation of long recordings into action clips, metadata and scene-graph extraction, embedding-based retrieval, and language annotation workflows.

Requirements

• M.S. or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, or a related field.
• Deep expertise in Python with extensive experience using PyTorch, especially for custom dataloaders for multimodal datasets.
• Experience analyzing and processing complex time-series data from force-torque sensors, load cells, or tactile arrays, with careful alignment to visual frames.
• Strong knowledge of video processing pipelines and libraries including OpenCV, FFmpeg, and Decord, including managing I/O bottlenecks for terabyte-scale video datasets.
• Solid robotics and 3D geometry knowledge covering coordinate frames and transforms, rotation representations, camera intrinsics and extrinsics, forward/inverse kinematics, and URDF.
• Proven ability to implement programmatic and generative data augmentation for computer vision and time-series data.

Technologies

Python, PyTorch, OpenCV, FFmpeg, Decord, URDF, Ray, Apache Spark, Open X-Embodiment, DROID, AgiBot World, EgoDex, SAM-family, MANO, SMPL, Omniverse, MuJoCo, NVIDIA robotic software stack, NVIDIA's robotic software stack, VLM

Benefits

• Competitive compensation
• Performance-based bonus
• 99% employer covered medical benefits
• Early-stage equity
• Competitive PTO
• Company-wide paid winter break between December 24th and January 2nd
• Full access to advanced tools

Bonus Skills

• Experience with NVIDIA’s robotic software stack (Open X-Embodiment, DROID, AgiBot World, EgoDex, or similar).
• Comfort using modern perception tools as a user, including segmentation (SAM-family), monocular depth, hand/body pose estimation (MANO/SMPL), and 6-DoF object pose tracking and point tracking. Experience integrating and evaluating these components in a pipeline is expected.
• Familiarity with distributed data processing systems such as Ray and Apache Spark.
• Background in generating or utilizing synthetic robotic data through simulation using Omniverse and MuJoCo.
• Experience integrating spatial awareness or tactile data representations (for example, Fourier encoding) into visual pipelines.

Job Details

• Department: Software
• Reports To: Teleoperations Lead
• Employment Type: Full-Time
• Location: Houston, TX (onsite); Houston, TX or Pensacola, FL