Change Data Capture

Change Data Capture Engineer Jobs

6 jobs found

Today

Data Engineer Sr

Oshkosh Corporation Oshkosh, WI

Oshkosh, WI

Today

$104k - $174k/yr Full time Application Security Automation Azure Data Engineer

Apply Now

Job Description

Oshkosh Corporation’s Advanced Analytics team is looking for a senior Data Engineer in Oshkosh, Wisconsin to help deliver an end-to-end data analytics strategy. This role focuses on acquiring, processing, and applying engineering and service data to support preventive maintenance and intelligent vehicle control, with an emphasis on building reliable CDC and ETL solutions.

You will work on scalable data processing pipelines that feed analytics platforms, partnering across disciplines and business segments while supporting governance, performance tuning, and documentation that enables architectural consistency and delivery planning.

Responsibilities

Design and develop scalable Change Data Capture (CDC) and ETL solutions to move structured and unstructured data from source systems to analytics platforms, supporting both batch and streaming.
Use tools and platforms including Terraform, Spark, Databricks, Snowflake, Git, and Azure infrastructure management (including ADLS Gen2, Azure Compute, Azure Data Factory, AI Foundry, AKS, VPC’s, and Storage Accounts, among others).
Assist with data governance across a data lake and an enterprise data warehouse, including quality and security practices across business segments.
Tune performance for data processes and troubleshoot data processing issues.
Collaborate, coordinate, and communicate across departments, disciplines, and segments.
Develop rapid prototyping and design processes to enable fast delivery to the business.
Maintain reference architecture and documentation to support architectural governance and application roadmaps.
Assist in educating others on data best practices, including data modeling, database design, ETL design, and job scheduling and monitoring.
Support or direct feasibility studies and project estimates for proposed R&D initiatives, including manpower, budget development, and timelines.
Follow supervisory direction on a mutually agreed schedule with minimum mentoring, providing feedback on technical hurdles, progress, and priority alignment to business needs.

Requirements

Bachelor’s degree in computer science, data science, or a related field, plus 4+ years working as a data engineer, ETL developer, and/or data warehouse DBA.
3-5 years of prior experience in a Data Engineering or Analytics organization in a technical role.
2+ years of software development experience ( Python/Go preferred ).
3+ years of major cloud provider experience (AWS/Azure/GCP).
Strong proficiency with SQL, Bash, and Python (including experience with Anaconda distribution).
2+ years operating in Unix/Linux and Windows server environments.
2+ years operating in an Agile / Scrum environment.
Experience with cloud-based data services and solutions including Azure Synapse / Data Lake, AWS RedShift, Snowflake, and GCP Big Query.
Experience partnering with Analytics and Data Science teams to build production-grade GenAI/ML solutions.
Familiarity with Change Data Capture solutions such as Qlik Replicate, Striim, and Debeezium.
Experience with big data tools and architectures including Cloudera Hadoop, HDFS, Hive, and Spark.
Working knowledge of telematics interfaces and streaming solutions including MQTT, NiFi, and Kafka.
Automotive or heavy duty on or off-road vehicle experience, including digital data bus knowledge such as Ethernet or Controller Area Network (CAN).
Highly organized with strong critical thinking, analytical, and problem-solving skills; capable of handling multiple tasks in a fast-paced environment.
Excellent interpersonal skills, including ability to present information and respond to leadership and peer questions.

Technologies

Change Data Capture (CDC), ETL
Terraform, Spark, Databricks, Snowflake, Git
Azure infrastructure management, ADLS Gen2, Azure Compute, Azure Data Factory, AI Foundry, AKS, VPC’s, Storage Accounts
AWS, Azure, GCP; Azure Synapse, Data Lake, AWS RedShift, GCP Big Query
GenAI/ML
Qlik Replicate, Striim, Debeezium
Cloudera Hadoop, HDFS, Hive
MQTT, NiFi, Kafka
Ethernet, Controller Area Network (CAN)
SQL, Bash, Python, Anaconda

Pay Range

USD 104,100 - 173,900 per yearly

Location: Oshkosh, WI (onsite)

View full job page →

1mo ago

Data Engineer

BioAgilytix Durham, NC

Durham, NC

1mo ago Closed

Full time Backend Developer Analytics API

This position is no longer accepting applications (closed Sep 18, 2026).

Job Description

BioAgilytix is seeking a hands-on Data Engineer technical lead to design, build, and support its Enterprise Data Platform. This onsite role in Durham, NC focuses on delivering trusted, governed, production-ready data pipelines, dimensional and semantic models, and certified data products that support laboratory operations, analytics, sponsor reporting, regulatory compliance, and AI initiatives.

What you will help deliver

Scalable enterprise data platforms that enable high-quality, governed data products for analytics, scientific operations, sponsor reporting, regulatory compliance, and AI initiatives.
Production-grade ELT pipelines integrating data from laboratory information systems (LIMS), ERP, CRM, APIs, sponsor systems, cloud applications, and other enterprise sources.
Modular, reusable transformation frameworks using modern ELT practices, including automated testing, documentation, lineage, version control, and deployment automation.
Dimensional data models, semantic models, and governed datasets based on established enterprise business definitions.
Validated, traceable, and auditable pipelines supporting regulated laboratory operations and enterprise reporting with data integrity, reproducibility, lineage, and compliance with GxP, GLP, HIPAA, and 21 CFR Part 11, aligned with enterprise data governance standards.
Automated data validation and reconciliation, data quality controls, audit logging, monitoring, observability, and operational alerting to help ensure reliability.
Platform improvements across performance, scalability, security, governance, and operational efficiency.
Operational support including production monitoring, incident resolution, root cause analysis, and continuous reliability enhancements.
Sponsor-facing data delivery through harmonization, transformation, validation, lineage, and regulatory reporting processes.
Certified, governed, and AI-ready data products that support enterprise analytics, machine learning, semantic search, and generative AI initiatives.
Collaboration with Laboratory Operations, Quality teams, IT, and business stakeholders to deliver scalable, reusable, governed solutions.

Core requirements

Bachelor’s degree in computer science, Information Systems, Engineering, Mathematics, Data Science, or a related field (Master’s preferred).
5+ years of experience in Data Engineering, Data Management, Software Engineering, Business Intelligence, or related technical disciplines, preferably within life sciences, biotechnology, pharmaceuticals, CROs, healthcare, or other regulated industries.
3+ years hands-on experience designing, developing, and supporting enterprise-scale data engineering solutions in production environments.
3+ years hands-on experience architecting, developing, and administering enterprise solutions using Snowflake, including performance optimization, security, governance, workload management, and operational support.
Strong hands-on experience with dbt Cloud or dbt Core for modular transformations, automated testing, documentation, lineage, and deployment.
Demonstrated expertise in enterprise dimensional data modeling (star schemas, conformed dimensions, slowly changing dimensions, snapshot fact tables, and analytical data warehouse design).
Experience designing semantic models, enterprise business vocabularies, ontology-driven data products, or knowledge graph concepts.
Strong proficiency in SQL and Python.
Experience with enterprise integration using ETL/ELT platforms such as Talend, Fivetran, or equivalent tools.
Experience integrating enterprise applications via REST APIs, GraphQL APIs, file-based interfaces, Change Data Capture (CDC), and event-driven messaging platforms.
Experience with AWS (S3, Lambda, ECS, Glue) and/or Azure.
Experience designing, validating, and maintaining certified enterprise data products with documented business definitions, transformation logic, lineage, ownership, and lifecycle management.
Experience implementing least-privilege security, RBAC, data masking, row-level security, encryption, secrets management, and secure data sharing.
Experience supporting enterprise production data platforms, including incident management, root cause analysis, operational monitoring, performance tuning, release management, and platform reliability engineering.
Experience working with Laboratory Information Management Systems, bioanalytical data, sponsor deliverables, and regulated laboratory environments is strongly preferred.
Expert proficiency in SQL and Python, including Snowflake architecture such as Snowpark, Dynamic Tables, Streams, Tasks, and data sharing/security/governance/workload management.
Strong DataOps and engineering practices including Git, GitHub Actions, CI/CD, Infrastructure-as-Code, automated testing, and observability.
Excellent communication, documentation, and collaboration skills, with the ability to work independently on complex assignments and escalate decisions when needed.
Ability to navigate a fast-paced, evolving data landscape, demonstrating resilience and flexibility.

Technologies you will work with

Snowflake, Snowpark, Dynamic Tables, Streams, Tasks, data sharing
dbt Cloud, dbt Core (models, snapshots, macros, tests, semantic models, documentation, lineage)
ELT, dimensional modeling, semantic modeling
Talend, Fivetran, REST APIs, GraphQL APIs, CDC, event-driven messaging
AWS (S3, Lambda, ECS, Glue) and/or Azure
RBAC, Power BI, Sigma, Tableau, semantic reporting platforms
Git, GitHub Actions, Infrastructure-as-Code, DataOps

Benefits

Medical Insurance (HDHP with HSA; PPO)
Dental Insurance
Vision Insurance
Flexible Spending Account (medical; dependent care)
Short Term Disability | Long Term Disability
Life Insurance
Paid Time Off (4 weeks per year)
Parental Leave
Paid Holidays (9 scheduled; 5 floating)
401k with Employer Match
Employee Referral Program

Additional position details

Full-time role; some flexibility in hours with availability during core work hours per the BioAgilytix Employee Handbook.
Occasional weekend, holiday, and evening work needed.
No supervisory responsibilities.
Reports to the Data Management Lead; works independently on complex engineering initiatives with infrequent supervision and instructions.
Preferred credentials: Master’s degree.

Physical demands

Ability to work in an upright and/or stationary position for up to eight (8) hours per day.
Occasional mobility needed; occasional crouching and stooping with frequent bending and twisting of upper body and neck.
Light to moderate lifting and carrying up to 20 pounds, including luggage and a laptop computer.
Ability to use office equipment and computer software; ability to communicate information and ideas effectively.
Regular and consistent attendance; ability to perform under stress and multi-task.

View full job page →

1mo ago

Senior Data Engineer

Versant Media North Hollywood, CA

North Hollywood, CA

1mo ago Closed

$140k - $160k/yr Full time Senior Big Data Bigdata

This position is no longer accepting applications (closed Sep 11, 2026).

Job Description

Build and scale a modern data platform that powers analytics, data science, and data products across Versant brands in a hybrid role based in North Hollywood.

Responsibilities

Design and implement lakehouse architecture with Delta Lake, including medallion pipeline patterns (Bronze/Silver/Gold), schema enforcement, and time travel
Build and operate batch and real-time ingestion pipelines using Databricks Auto Loader, Structured Streaming, and Change Data Capture (CDC) patterns
Implement data governance and security with Unity Catalog and RBAC, following compliance-driven practices for sensitive environments
Optimize performance and manage costs using FinOps strategies, including cluster sizing, workload tagging, Spark tuning, and Photon acceleration
Create and maintain CI/CD pipelines and orchestration workflows using Databricks Workflows, Delta Live Tables, and tools such as Airflow
Partner with Data Science teams on ML workflows using MLflow, feature store integration, and model lifecycle management
Ensure data quality, observability, and lineage across media-focused datasets such as streaming logs, ad impressions, and audience metrics
Provide technical mentorship through code reviews, pairing, and ongoing knowledge sharing

Requirements

Bachelor’s degree in Computer Science, Data Engineering, or equivalent practical experience
5+ years building production-grade data pipelines in cloud environments using Spark-based platforms (e.g., Databricks, EMR, Dataproc, open-source Spark)
Expertise in PySpark and SQL, with experience operating scaled pipelines in production
Hands-on experience building batch or streaming production ingestion pipelines using distributed processing frameworks (e.g., Spark, Flink) and query engines such as Presto
Proficiency with orchestration tools such as Apache Airflow or Dagster, including hands-on CI/CD, monitoring, alerting, and production data quality
Experience with modern data architectures, including event-driven and distributed systems
Proficiency with Git and collaborative development workflows
Hands-on experience building and operating batch and real-time pipelines using Spark-based batch and streaming patterns (e.g., Structured Streaming, CDC), including experience on Databricks or comparable platforms
Solid understanding of infrastructure, networking, and data security fundamentals

Technologies

Delta Lake, Delta Live Tables, Databricks Auto Loader, Structured Streaming, Change Data Capture (CDC)
Unity Catalog, RBAC
Databricks Workflows, Airflow, PySpark, SQL, Spark, Flink, Presto, Dagster
FinOps, Photon acceleration
MLflow, feature store integration, CI/CD, Git
Lakehouse, medallion pipeline patterns (Bronze/Silver/Gold), schema enforcement, time travel
Spark tuning, cluster sizing, workload tagging

Benefits

Medical, dental and vision insurance
401(k)
Paid leave
Tuition reimbursement
Company sponsored benefits and a variety of other benefits and perks

Preferred Qualifications

Experience building Lakehouse platforms and medallion pipelines in Databricks
Familiarity with Unity Catalog, data governance, and compliance frameworks (e.g., PCI)
Hands-on experience with CI/CD pipelines, orchestration tools, and infrastructure-as-code
Experience with Lakeflow Spark Declarative Pipelines (SDP), MLflow, feature stores, and MLOps practices
Background in media and entertainment data (e.g., video metadata, ad tech, audience analytics)
Experience building data platforms within the media industry, including audience analytics
Experience working with large-scale analytical datasets such as event logs, clickstream data, and audience metrics
Comfort using AI-assisted development tools (e.g., ChatGPT)
Databricks or cloud certifications (e.g., Databricks Certified Data Engineer)

Additional Information

Location: North Hollywood, CA (hybrid). Generally in the office a minimum of three days per week from the LA office.
Salary: $140,000 - $160,000 per year
Candidates may be required to attend an in-person interview with a VERSANT employee in New York as part of the selection process.
Equal employment opportunity policy applies without regard to protected characteristics.
Reasonable accommodation: email candidateaccessibility@versantmedia.com for support during the application and recruitment process.
Fair pay transparency statement applies; actual compensation may vary based on skills, qualifications, experience, and location.
For LA County and City Residents Only: qualified applicants with criminal histories may be considered consistent with applicable legal requirements.
VERSANT Media is not accepting unsolicited assistance from search firms; resumes submitted without a valid written Statement of Work may be treated as VERSANT’s sole property and no fee will be paid.

View full job page →

1mo ago

Data Engineer II

Jack Henry & Associates Allen, TX 🇺🇸 Remote

Allen, TX

1mo ago Closed

Full time Analytics Automation Change Data Capture

This position is no longer accepting applications (closed Sep 6, 2026).

Job Description

The Data Engineer II role sits within the Data & Analytics organization on the Warehousing & Pipelines team. The position focuses on building and evolving enterprise data integration solutions that support operational reporting, analytics, machine learning, and broader data initiatives.

Team Collaboration

Work with the Data & Analytics team in the Warehousing & Pipelines function.
Collaborate with DBAs, Data Architects, BI Developers, Data Scientists, Software Engineers, and business stakeholders.

Core Responsibilities

Design, develop, test, deploy, and maintain batch and streaming data pipelines for payment processing systems with strict availability and performance requirements.
Build data integration solutions using SQL, C#, Python, PowerShell, and related automation technologies.
Develop and support event-driven architectures using Change Data Capture (CDC), Kafka, messaging platforms, and stream-processing technologies.
Maintain and modernize legacy data integration solutions, including SQL Server Integration Services (SSIS), and transition workloads toward code-first, version-controlled development practices.
Create and maintain CI/CD pipelines for data integration, analytics, and data platform solutions.
Implement monitoring, logging, alerting, and observability for data platforms and pipelines.
Partner with Data Architects and enterprise data governance teams to populate and maintain enterprise data warehouse, lakehouse, and analytical data platform solutions.
Tune SQL queries and database interactions to reduce impact on source systems and improve application performance.
Optimize data pipelines to improve throughput, scalability, resiliency, and cost efficiency.
Produce and maintain technical documentation, architectural diagrams, operational runbooks, and deployment procedures.
Use AI-assisted development tools responsibly to improve productivity and code quality, while independently validating, reviewing, and testing all generated code.
Participate in on-call rotations and provide after-hours support as required.
May perform other job duties as assigned.

Required Qualifications

Bachelor’s degree in Computer Science, Information Technology, or Engineering.
Minimum 3 years of experience in data engineering, software engineering, or database development.
Experience developing solutions on relational database platforms such as SQL Server and/or MySQL.
Experience developing applications or services using C# or similar object-oriented programming languages.
Experience developing and supporting ETL/ELT and data integration solutions.
Experience working with source control systems such as Git, including code reviews and pull request workflows.
Experience with containerization technologies such as Docker.

Preferred Qualifications

Experience modernizing legacy analytics platforms, OLAP solutions, or analytical workloads.
Experience with cloud services and data platforms in GCP, AWS, and/or Azure.
Experience with Infrastructure as Code (IaC) using Terraform, Pulumi, Bicep, Ansible, or similar technologies.
Experience building and supporting Kubernetes-based applications and services.
Experience building and supporting data lakes and lakehouses, including open table format solutions such as Apache Iceberg, Delta Lake, Hudi, and Parquet.
Experience operating under PCI DSS, SOC, financial services, healthcare, or other regulatory frameworks.

Technology Stack

SQL, C#, Python, PowerShell, Change Data Capture (CDC), Kafka, SQL Server Integration Services (SSIS), CI/CD, Git, Docker, MySQL, ETL/ELT

Location and Interview Requirements

Remote position based in the Allen, TX 75013 area.
Candidates must live within approximately a 70-mile radius of office locations in Allen, TX; Lenexa, KS; Springfield, MO; or Monett, MO; and Louisville, KY; Troy, MI; or Birmingham, AL.
All positions, regardless of location, may require an onsite interview or in-person onboarding requirement to verify identity.

View full job page →

1mo ago

Senior IT Data Engineer

System One Washington, DC

Washington, DC

1mo ago Closed

Contract Senior Apache Airflow AWS

This position is no longer accepting applications (closed Sep 2, 2026).

Job Description

System One is hiring a Senior IT Data Engineer (contract) to support the Federal Reserve Board’s Division of Research & Statistics (R&S) DATA section in Washington, DC (onsite). In this role, you will help expand and optimize data and data pipeline architecture, strengthen ETL/ELT workflows, and deliver enterprise data platform capabilities for economists and research teams working on economic policy and analysis.

Responsibilities

Expand and optimize data and data pipeline architecture for economic policy and research use cases.
Optimize data flow and collection to improve how multiple teams access and use data.
Design and develop delivery architecture to support next generation R&S data initiatives.
Improve existing data systems and/or build new systems from the ground up.
Optimize or redesign the R&S division’s data architecture to support evolving data initiatives.
Support economists and technical experts with a service mindset and a self-directed approach.
Design and communicate enterprise information architecture at conceptual, logical, and physical levels.
Develop and communicate enterprise architecture that supports data needs across teams and systems.

Requirements

Bachelor’s degree in computer science, Information Technology, Engineering or a related technical field, plus at least 7 years of related experience (advanced degree preferred).
Advanced working knowledge of SQL and experience with relational database platforms including PostgreSQL, Microsoft SQL Server, and MySQL.
Advanced working knowledge of Python, R, and other scripting languages used for data engineering and analytics.
Experience working with large-scale data systems, including distributed computing, scalable data processing, data storage architecture, and optimization of high-volume workloads.
Experience designing, developing, and automating ETL/ELT workflows and data integration pipelines.
Experience building, optimizing, and maintaining scalable databases, data pipelines, and data processing frameworks.
Experience with workflow orchestration and pipeline automation tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions.
Experience migrating workflows and data pipelines between on-premises and cloud environments.
Experience processing, analyzing, and integrating structured and unstructured data sources.
Experience developing in Linux and using source control platforms such as GitLab and/or GitHub.
Experience performing root cause analysis on internal and external data and business processes to answer business questions and identify improvement opportunities.
Ability to design and communicate enterprise information architecture at conceptual, logical, and physical levels.
In-depth experience designing and implementing database, data lake, and enterprise data platform solutions.
Strong hands-on software engineering experience, including development, testing, and deployment of data applications and services.
Excellent oral and written communication skills with a strong customer service orientation.
Exceptional analytical, problem-solving, and troubleshooting skills.
Understanding of time series data and related analytical and forecasting techniques.
Experience in a research environment and/or with economic or financial data.
Experience with NoSQL and graph database technologies.
Experience developing, training, deploying, and maintaining machine learning models.
Working experience with AWS, Microsoft Azure, and Snowflake.
Experience implementing data warehouses utilizing Change Data Capture (CDC) methodologies.
Experience implementing and maintaining CI/CD pipelines and DataOps platforms.
Working knowledge of additional programming and scripting languages such as Java, Scala, JavaScript, or Perl.

Technology

SQL, PostgreSQL, Microsoft SQL Server, MySQL, Python, R, Apache Airflow, Prefect, Dagster, AWS Step Functions, Linux, GitLab, GitHub, NoSQL, graph database technologies, machine learning models, AWS, Microsoft Azure, Snowflake, Change Data Capture (CDC), CI/CD pipelines, DataOps, Java, Scala, JavaScript, Perl

Benefits

Eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans
Participation in a 401(k) plan

View full job page →

2mo ago

Senior Data Engineer

Everforth San Francisco, CA

San Francisco, CA

2mo ago Closed

$160k - $230k/yr Full time Senior Big Data Change Data Capture

This position is no longer accepting applications (closed Aug 4, 2026).

Job Description

Everforth seeks a Senior Data Engineer to design and operate scalable, real-time data pipelines and data models that enable clinical monitoring and machine learning workflows. The role emphasizes integrations with electronic health records, medical devices, and hospital systems, deployment in AWS with Snowflake and MySQL, and governance in a HIPAA-compliant environment.

Location

San Francisco, CA onsite

Compensation

USD 160,000 to 230,000 per year

Responsibilities

Create and sustain scalable data pipelines that ingest information from EHR systems, medical devices, and hospital information systems
Develop integrations using APIs, webhooks, and change data capture ingestion patterns
Transform and harmonize data for downstream stores such as MySQL and Snowflake
Build high-throughput, dependable pipelines supporting both real-time streaming and batch processing
Handle core healthcare data including ADT events, clinical data, and vital signs
Collaborate with internal teams and customers to onboard data and ensure reliability
Architect data models to support both application functionality and analytics workloads
Tune performance and scalability across Snowflake and MySQL
Deploy and operate data platforms within AWS
Implement comprehensive monitoring, logging, and alerting for production pipelines
Provide ongoing support for production systems and progressively improve reliability
Operate within a HIPAA-compliant environment with team support
Partner with ML teams on feature pipelines and data quality initiatives

Requirements

Minimum five years of experience in data engineering or backend engineering
Proficient in Python and SQL
Proven ability to deliver production-grade data pipelines end-to-end
Hands-on experience with Snowflake and MySQL
Experience with APIs, webhooks, or CDC-style ingestion
Experience operating in AWS or equivalent cloud environments
Experience working with sensitive or regulated data is a plus

Technologies

Python
SQL
Snowflake
MySQL
AWS

Benefits

100% company-paid medical, dental, and vision
401(k) with company match
Bonus and equity opportunities
Generous paid time off
Professional development budget
Company-paid annual retreat

Nice to Have

Exposure to HL7 or FHIR
Experience supporting ML pipelines
Experience with real-time or streaming systems

Why This Role

Direct impact on patient outcomes
Real-time, high-scale data challenges
High ownership within a small, capable team
Opportunity to help build out the San Francisco team

View full job page →

Change Data Capture Engineer Jobs

Data Engineer Sr

Data Engineer

Senior Data Engineer

Data Engineer II

Senior IT Data Engineer

Senior Data Engineer

Get Change Data Capture Engineer Jobs alerts