Senior Data Engineer

The Senior Data Engineer at Absentia Labs will own the design and ongoing evolution of a biomedical data platform that supports scientific and machine learning workflows. This hybrid role in New York focuses on scalable, reliable systems, schema-driven modeling, and production-grade pipelines.

Responsibilities

• Architect and lead end-to-end data systems for large-scale biomedical datasets, including chemical, biological, toxicology, omics, assay, clinical, and experimental data.
• Define and evolve schema-driven data models that reconcile noisy, semi-structured, and heterogeneous sources into coherent, interoperable representations.
• Set standards for data quality, validation, provenance, lineage, and versioning tailored to scientific and ML workflows.
• Build and maintain cloud-native data infrastructure, including data lakes, warehouses, object storage, and streaming systems, with emphasis on scalability and reliability.
• Design pipelines that support both batch and streaming access for ML training, evaluation, and inference.
• Work closely with ML engineers, scientists, and product leads to translate research requirements into durable data abstractions.
• Make principled production trade-offs involving performance, cost, flexibility, and correctness.
• Provide technical leadership through design reviews, architectural guidance, and mentorship.
• Identify and proactively address systemic risks in data integrity, scalability, and operational complexity.

Requirements

• 5+ years of experience in data engineering, platform engineering, or ML infrastructure roles, with demonstrated ownership of production systems.
• Proven experience designing and operating large-scale, production-grade data pipelines.
• Strong proficiency in Python and data-centric software engineering practices.
• Deep experience with cloud platforms including AWS, GCP, or Azure, covering storage, compute, and security primitives.
• Familiarity with distributed data processing and orchestration tools such as Spark, Beam, Ray, Airflow, and Dagster.
• Experience supporting ML/AI workloads, including dataset generation, feature pipelines, and reproducible training workflows.
• Strong architectural judgment and the ability to communicate technical decisions clearly across disciplines.

Technologies

• Python
• AWS
• GCP
• Azure
• Spark
• Beam
• Ray
• Airflow
• Dagster

Benefits

• Flexible remote or hybrid work arrangements.
• A deeply technical, low-ego culture focused on learning and rigor.

Bonus If You Have

• Competitive compensation with meaningful equity participation.
• Prior work with biomedical or life-science data such as omics, assays, molecular representations, clinical, or toxicology data.
• Experience with streaming platforms including Kafka, Pub/Sub, or Kinesis.
• Exposure to ontology-aware data modeling or schema evolution in scientific domains.
• Infrastructure-as-code and systems experience with Terraform, Docker, or Kubernetes.
• Experience in early-stage startups or research-heavy environments.
• Open-source contributions or technical publications.

Additional Role Notes

• Focus on systems and interfaces, not only pipelines.
• Comfort owning ambiguous problems and converging on robust solutions through iterative design.
• Understanding that biomedical data is rarely clean, with schema design, normalization, and semantics treated as first-order engineering problems in AI-driven settings.

What You’ll Help Deliver

• Architect the data backbone of an AI-driven biomedical platform.
• Enable direct impact on translating scientific data into machine intelligence.
• Operate with high autonomy, high trust, and ownership over critical systems.

Compensation and Location

• Location: New York, NY (hybrid)
• Salary: USD 115,000 - 200,000 per year
• Minimum experience: 5 years

How to Apply

• Submit your resume and a short note on why the role resonates with you.
• GitHub links, technical writing, or relevant project links are encouraged.
• Compensation range: $115K - $200K