Duckdb

Duckdb Engineer Jobs

3 jobs found

14d ago

Staff Data Engineer

Storm3 San Francisco, CA 🇺🇸 Remote

San Francisco, CA

14d ago

$185k+/yr Full time Senior Apache Airflow Data Architecture

Apply Now

Job Description

Storm3 is hiring a Staff Data Engineer to help build PHI and healthcare data pipelines in a remote-first setup in San Francisco. You will work alongside a highly technical engineering team with direct partnership with the CTO and strong ownership over production systems. The role comes with a $185,000 base plus quarterly bonus and equity opportunity.

What you’ll bring

Experience working with patient-level healthcare data under HIPAA regulations
Strong Python expertise building production-grade data systems
Advanced SQL skills, including query optimization and troubleshooting
Hands-on experience with orchestration tools such as Dagster, Airflow, or Prefect (with Dagster strongly preferred)
Experience building and supporting ingestion pipelines, validation frameworks, audit processes, and failure recovery workflows
Experience deploying and operating workloads on Kubernetes
A strong observability and troubleshooting mindset with the ability to own business-critical systems
Comfort using AI-assisted development tools while maintaining full ownership of technical decisions and output quality

What you’ll own

Design, develop, and operate PHI data pipelines built in Python using Ibis, DuckDB, Pandas, Dagster, and Kubernetes
Lead patient identity resolution and record-linkage workflows to align healthcare data accurately
Own the data-ingestion lifecycle from file intake through validation, enrichment, masking, audit, and delivery
Automate data operations and recovery processes with enterprise-grade observability, governance, and retry mechanisms
Act as the escalation point for critical production issues impacting client-facing healthcare data pipelines
Partner closely with the CTO on key engineering decisions within the organization’s lakehouse architecture

Tools you’ll work with

Python, Ibis, DuckDB, Pandas, Dagster, Kubernetes, SQL, Airflow, Prefect

Benefits

$185k base + quarterly bonus + equity opportunity
Remote-first environment working alongside a highly technical engineering team
Direct partnership with the CTO and high ownership over critical systems

View full job page →

1mo ago

Data Engineer

Cyvl Somerville, MA

Somerville, MA

1mo ago Closed

$125k - $195k/yr Full time Apache Airflow Big Data Bigdata

This position is no longer accepting applications (closed Oct 30, 2026).

Job Description

Cyvl is seeking a Data Engineer to build and scale the pipelines and data platform that transform raw LiDAR, imagery, and GPS inputs into client-ready geospatial deliverables. The role emphasizes production data engineering, automated data quality, and cost and performance improvements on AWS.

Role Overview

Every Cyvl drive captures LiDAR, imagery, and GPS data that must become trustworthy outputs for city engineering teams. As a Data Engineer, you will develop the data infrastructure behind Cyvl’s Infrastructure Intelligence Platform, taking sensor data through processing, AI inference, and QA, and delivering reliable geospatial results for clients.

Location

Somerville, MA (hybrid)
Remote option for candidates based in San Francisco, CA, with in-person time in Somerville for key moments

Compensation

Salary: USD 125,000 - 195,000 per year
Equity: included

Key Responsibilities

Design, build, and operate pipelines that move sensor data from ingest to client-ready geospatial outputs.
Build and maintain a data platform so product, AI, and analytics teams can locate and use data quickly.
Create automated data quality checks and monitoring to detect issues before customers are impacted.
Partner with Data Operations and GIS teams to automate manual workflows and increase throughput.
Improve cost and performance for large-scale processing on AWS.

Requirements

3+ years building production data pipelines or data platforms.
Strong Python and SQL.
Experience with cloud data infrastructure (AWS preferred) and workflow orchestration (Airflow, Dagster, Prefect, or similar).
Experience making large, messy datasets reliable.
Comfort owning systems end to end, from design through on-call.

Technologies

Python, SQL
AWS
Airflow, Dagster, Prefect
PostGIS, GDAL, GeoParquet, H3
Iceberg, Delta, DuckDB, Spark
Docker, Kubernetes, Terraform

Benefits

Equity
Health, dental, and vision (Cyvl covers most of the premium for you and your family)
401k with a real match, vested day one
Generous PTO and Paid Parental leave
Learning stipend
Paid to live close (monthly bonus for living near the Boston office)
Team events

Nice to Have

Geospatial data experience (PostGIS, GDAL, GeoParquet, H3, or other spatial indexing).
Sensor data experience: LiDAR point clouds, imagery, GPS/IMU.
Lakehouse formats and engines (Iceberg, Delta, DuckDB, Spark).
Docker, Kubernetes, and Terraform.
Experience supporting ML training and inference pipelines.

About the Team

High ownership and fast iteration, with the understanding that some bets will not land.
On-site presence in Somerville is expected about 90% of the time.
Cyvl emphasizes values including Real, All Out, Accelerating, Reliable, and Empowering.

Examples of Work

How Atlanta cut downtown potholes by 60%
How Buffalo turned a campaign promise into a paving plan
How to rebuild US infrastructure smarter

View full job page →

2mo ago

Lead Data Engineer

Navanta, LLC Alpharetta, GA

Alpharetta, GA

2mo ago Closed

Full time Apache Airflow Data Architecture Data Build Tool

This position is no longer accepting applications (closed Sep 30, 2026).

Job Description

Navanta, LLC builds governed data foundations that support public Call Report data and secure bank-core ingestion as client environments are stood up. As the Lead Data Engineer, you’ll own the lakehouse backbone, working closely with AI/ML, Security, and platform teams to enable reliable, reconcilable data products for Navanta AI platforms.

What you’ll work on

Design and operate the lakehouse using Apache Iceberg (or similar) on object storage, including a catalog for table management and per-bank isolation, plus dbt models and a query engine.
Build secure, least-privilege ingestion from bank systems, using log-based CDC where permitted, with query-based and batch/SFTP fallbacks, and an in-bank collector pattern.
Own data modeling for the semantic and metric layer, covering deposits, concentration, uninsured exposure, asset quality, and peer groups.
Ensure correctness and resilience by handling schema drift, data quality, and reconciliation, while making ingestion observable and recoverable.
Partner across teams on the structured-query path with AI/ML and on PII classification at landing with Security, aligned with regulatory data-handling requirements.
Document for audit readiness by maintaining data lineage, transformation logic, and access controls to support exam and audit processes.
Define and enforce data contracts, including quality thresholds and alerting for pipeline failures.

Core expectations

This is a full-time role that combines ongoing pipeline operations with initiative-based lakehouse build-out and new bank onboarding. You will collaborate closely with AI/ML, platform engineering, and Security, including participation in an on-call rotation for data pipeline reliability.

What you bring

8–12+ years in data engineering with end-to-end ownership from ingestion through serving, including 2+ years in a lead or senior role.
Strong Python and expert SQL, with rigorous data modeling for analytics.
Hands-on lakehouse experience (Iceberg/Delta/Hudi or equivalent) and modern transformation tooling.
Experience building reliable pipelines from messy operational and transactional source systems.
Comfort with CDC mechanics and extracting data from databases you do not control.
Bachelor’s degree in computer science, mathematics, information systems, or a related field, or equivalent hands-on experience.
Financial services or regulated data environment experience strongly preferred.

Key KPIs

Data freshness and pipeline reliability: SLAs met for data ingestion and bank-core feeds.
Data quality score across key metrics versus source reconciliation.
Time to onboard a new bank’s data environment from kickoff to a queryable lakehouse.
PII classification coverage at landing and zero unauthorized data-access incidents.
Semantic layer adoption: percentage of assistant queries resolved via governed metrics versus ad hoc SQL.

Technologies you’ll use

Python, SQL, Apache Iceberg (plus Polaris / Nessie / Lakekeeper), dbt, Trino / Presto / DuckDB, Debezium, Kafka / Redpanda, Dagster (or Airflow), and storage such as S3 / MinIO. Source and modeling include SQL Server and PostgreSQL, plus pgvector (or equivalent).

Nice to have

Experience with financial or core-banking data, including FFIEC / Call Report data specifically.
Strong SQL Server familiarity.
Experience with data contracts, lineage, and governance practices.

Location and work environment

Alpharetta, GA (onsite). Typical office environment with up to 20% travel time may be required.

View full job page →

Duckdb Engineer Jobs

Staff Data Engineer

Data Engineer

Lead Data Engineer

Get Duckdb Engineer Jobs alerts