Data Management

Data Management Engineer Jobs

353 jobs found

Today

Fabric and Data Engineer

East West Bank Pasadena, CA

Pasadena, CA

Today

$30 - $34/hr Full time Analytics Application Security Automation

Apply Now

Job Description

The role focuses on designing, implementing, and supporting enterprise-grade data solutions using Microsoft Fabric and Azure. The position emphasizes secure, scalable, automated, and highly reliable data platforms that support analytics, reporting, and AI initiatives.

Responsibilities

Design, deploy, and support Microsoft Fabric environments, including workspaces and solutions.
Build and maintain Lakehouses, Warehouses, data pipelines, notebooks, and data transformation processes.
Develop scalable ETL/ELT solutions using SQL, Python, PySpark, and Spark.
Integrate Fabric with Azure services, databases, APIs, SaaS applications, and on-premises data sources.
Implement security, RBAC, data governance, lineage, and compliance controls.
Set up Git, CI/CD, and automated deployment practices across development, test, and production environments.
Monitor platform health, capacity, performance, pipelines, and production workloads.
Troubleshoot incidents, perform root-cause analysis, and optimize performance.
Create technical standards, documentation, operational procedures, and runbooks.
Coordinate with Cloud, Infrastructure, Security, Data, Application Development, and business stakeholders.

Requirements

5+ years of experience in data, cloud, platform, or database engineering.
Hands-on experience with Microsoft Fabric and Azure data services.
Strong SQL skills and experience with Python and/or PySpark.
Experience building ETL/ELT pipelines, data lakes, warehouses, and Lakehouse architectures.
Experience with Git, CI/CD, automation, and production support practices.
Working understanding of cloud security, identity and RBAC, data governance, and monitoring.
Strong troubleshooting, documentation, and communication skills.
Legal authorization to work in the United States. No visa sponsorship is offered.

Technologies

Microsoft Fabric, Azure
SQL, Python, PySpark, Spark
Lakehouses, Warehouses, notebooks
ETL/ELT, Git, CI/CD
RBAC, data governance, lineage
Azure Data Factory, OneLake
Fabric Lakehouse, Fabric Warehouse
Power BI, Delta Lake, Terraform

Preferred

Microsoft Fabric production experience and related Microsoft certifications (DP-600, DP-700).
Experience with OneLake, Fabric Lakehouse and Fabric Warehouse, Data Factory, Spark, Power BI, and Delta Lake.
Experience using Infrastructure-as-Code tools such as Terraform.
Experience supporting data platforms in financial services or another highly regulated environment.

Compensation

The base pay range for this position is USD $30.00/Hr. - USD $34.00/Hr.

Location

Pasadena, CA (onsite)

View full job page →

1d ago

Senior Databricks Data Engineer

CGI Group, Inc. Miami, FL

Miami, FL

1d ago

$92k - $204k/yr Full time Senior Azure Azure Databricks

Apply Now

Job Description

CGI Group, Inc. is hiring a Senior Databricks Data Engineer in Miami, FL (hybrid) to lead end-to-end delivery of scalable enterprise data integration solutions.

Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Apache Spark, Python/PySpark, and SQL
Build batch and incremental data integration pipelines from files, databases, and enterprise data sources
Create reusable, maintainable data processing solutions
Optimize Spark workloads for performance, scalability, reliability, and cost efficiency
Implement data quality controls including reconciliation, monitoring, and error handling
Own the complete engineering lifecycle: requirements analysis, solution design, technical review, implementation, and deployment
Prepare and present technical solution designs for architecture and engineering review
Apply review feedback and implement approved designs
Make technical decisions and proactively resolve engineering challenges
Maintain accountability for deliverables, quality, timelines, and production readiness
Develop complex SQL transformations for cleansing, enrichment, and aggregation
Design and maintain curated datasets, dimensional models, and reporting views
Support enterprise analytics, reporting, and downstream data consumers
Implement and maintain pipeline orchestration using Airflow, Databricks Workflows, or comparable tooling
Apply CI/CD practices for version control, automated deployment, and release management
Troubleshoot pipeline failures and production performance issues
Collaborate with data architects, integration teams, engineering teams, and business stakeholders
Document technical designs, implementation decisions, and operational procedures

Requirements

Minimum 5 years of relevant experience in data engineering, data integration, or related disciplines
Strong, recent hands-on development experience with Databricks
Proven experience designing, building, and implementing production ETL/ELT pipelines
Advanced proficiency in Apache Spark, PySpark, and Python with demonstrated hands-on engineering experience
Strong SQL skills including complex joins, window functions, data transformations, and query performance tuning
Experience with at least one major cloud platform: AWS, Azure, or GCP
Experience with workflow orchestration: Apache Airflow, Databricks Workflows, or equivalent platforms
Experience with CI/CD, source control, and automated deployment of data pipelines
Ability to independently own the full development lifecycle including solution design, technical review, implementation, and production support
Ability to evaluate requirements, make sound technical design decisions, and implement approved solutions
Strong problem-solving skills with a track record of independent execution, accountability, and ownership in fast-paced environments

Technologies

Databricks
Apache Spark
Python
PySpark
SQL
Airflow
Databricks Workflows
CI/CD
AWS
Azure
Google Cloud Platform (GCP)
Delta Lake
Unity Catalog
REST API

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field

Compensation

USD 91,800 - 203,800 per year (estimated U.S. range)

Benefits

Competitive compensation
Comprehensive insurance options
Matching contributions through the 401(k) plan and the share purchase plan
Paid time off for vacation, holidays, and sick time
Paid parental leave
Learning opportunities and tuition assistance
Wellness and Well being programs

Nice to Haves

Calling external APIs from Databricks notebooks (REST API integration, authentication, request handling, ingestion of API responses)
Streaming and event driven data processing
Delta Lake and medallion architecture
Unity Catalog, governance and lineage
Metadata driven and reusable pipeline frameworks
Enterprise scale Databricks implementations
Experience with complex enterprise data integration environments
API clarification: consuming/calling APIs from Databricks notebooks qualifies; building/exposing APIs is not required, and candidates without API experience remain eligible
Strong analytical skills, independent execution, technical documentation, collaboration across teams, and experience working in Agile delivery environments
Spark performance tuning and cost optimization
Batch and incremental data processing
Data quality, reconciliation, monitoring, and observability
Data warehousing, fact/dimension modeling, and curated data layers
Production troubleshooting and operational reliability
Technical documentation and design review participation
Cross functional communication and Agile delivery

CGI anticipates accepting applications for this position through November 13, 2026.

View full job page →

1d ago

Data Engineer

D2 Technologies Herndon, VA

Herndon, VA

1d ago

$100k - $150k/yr Full time Big Data Bigdata Cloud Data Engineering

Apply Now

Job Description

D2 Technologies is building enterprise-scale data capabilities and is seeking an experienced Data Engineer to design, develop, and optimize data solutions on the Databricks Lakehouse Platform. This hybrid role in Herndon, VA supports large-volume data processing through scalable ETL/ELT pipelines and performance-focused analytics engineering using Apache Spark and SQL.

What You’ll Do

Design, develop, and maintain scalable data engineering solutions using Databricks and Apache Spark
Build and optimize high-performance ETL/ELT pipelines for both structured and unstructured data
Develop, tune, and optimize complex SQL queries for data transformation and analytics
Translate business requirements into scalable technical solutions
Troubleshoot production issues and optimize data pipeline performance
Ensure data quality, integrity, security, and governance across enterprise data platforms
Collaborate with data architects, software engineers, analysts, and business stakeholders
Participate in code reviews and follow software engineering best practices
Create and maintain technical documentation for solutions, deployments, and operational support
Support production releases and provide operational support as needed

What You Bring

U.S. Citizenship is required
Top Secret (TS) Security Clearance or Public Trust Clearance preferred
Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience)
Hands-on experience developing and implementing data engineering solutions using Databricks
Strong, hands-on experience with the Databricks Lakehouse Platform
Strong programming experience with Apache Spark (PySpark/Spark SQL)
Expert-level SQL skills, including writing, optimizing, and troubleshooting complex queries
Strong analytical, critical thinking, and problem-solving skills
Experience designing, developing, and supporting ETL/ELT pipelines
Strong understanding of data modeling, data warehousing, and Delta Lake
Experience using Git or other version control systems
Excellent verbal and written communication skills

Additional Preferred Qualifications

Experience with Azure, AWS, or Google Cloud Platform
Knowledge of CI/CD pipelines and DevOps practices
Experience with Databricks Delta Live Tables (DLT) and Unity Catalog
Nice to have: PL/SQL

Technologies: Databricks, Databricks Lakehouse Platform, Apache Spark, PySpark, Spark SQL, SQL, ETL, ELT, Delta Lake, Git, Azure, AWS, Google Cloud Platform, Databricks Delta Live Tables (DLT), Unity Catalog, PL/SQL, CI/CD

Location: Hybrid remote in Herndon, VA 20192
Salary: $100,000 - $150,000 per year (negotiable)

Application screening: Federal Project requires US Citizenship. Are you US Citizen?

View full job page →

1d ago

Data Engineer

Delta Defense West Bend, WI

West Bend, WI

1d ago

Full time AWS Big Data Cloud

Apply Now

Job Description

Delta Defense is seeking a Data Engineer to help transform complex information into actionable insights. In this onsite role in West Bend, WI, you will build dependable data solutions and support the Enterprise Data Warehouse through transformation, data quality practices, KPI and reporting enablement, and close collaboration with Data Architecture, while maintaining engineering best practices.

What you’ll do

Work with data collection and processing systems to produce clean, tested, and reusable datasets for analysis.
Apply established standards for data transformations and ensure format and attribute requirements are followed.
Support data quality by maintaining and improving data profiling methodologies.
Maintain engineering best practices including CI/CD, unit testing, and performance testing and optimization.
Partner with the Data Architecture team to support the quality and consistency of the data model.
Use existing data infrastructure to complete data requests, including data housekeeping, data cleansing, normalization, hashing, and implementing required model changes.
Analyze data to identify anomalies, trends, and correlations across similar datasets.
Troubleshoot data-related issues, identify potential solutions, and resolve accordingly.
Maintain awareness of PII best practices and ensure adherence to Risk policies and standards set by the organization.
Perform other duties as assigned.

What you bring

High school diploma or equivalent.
Associate or bachelor’s degree preferred in Computer Science, Information Management, Data Science/Analytics, statistics, or equivalent relevant experience.
2-5 years of experience in data engineering or business intelligence roles.
3-5 years of SQL experience.
Ability to work efficiently with AWS cloud technologies such as S3 and EMR (EMR preferred).
1+ years experience in Agile SDLC.
1+ years dbt experience preferred.
Solid understanding of ETL/ELT methodologies.
Python and scripting language proficiency preferred.
Solid understanding of industry standard analytical modeling techniques.
Demonstrates the Core Values of Delta Defense, LLC.

Technologies

SQL, AWS, S3, EMR, dbt, Python
ETL/ELT, CI/CD, Agile SDLC

Location and sponsorship

This position is based at the company’s HQ in West Bend, WI. While some hybrid flexibility is available, the goal is for the selected candidate to be onsite. Delta Defense does not provide sponsorship for this role now or in the future.

View full job page →

3d ago

Associate Director, Pharmaceutical Data Engineer

Merck Rahway, NJ

Rahway, NJ

3d ago

$129k - $203k/yr Full time Backend Developer Manager Analytics

Apply Now

Job Description

Merck is modernizing digital operations with a data foundation built for analytics-ready intelligence and automation. In this onsite role in Rahway, NJ, you will help shift from traditional BI toward scalable data pipelines, custom data products, and agentic AI workflows that support pharmaceutical clinical and commercial supply chain needs.

What you’ll do

Design, build, and implement modern data foundations and pipelines to address business requirements.
Use innovation and technology such as Databricks, Python, and Agentic AI to manage data and improve the way advanced insights are generated.
Develop WebApps for Proof-of-Concept solutions, with an eye toward industrializing them later in the React Platform.
Streamline operations by reducing manual work through advanced automation and machine learning.
Support the Data Strategy and Strategic Roadmap, enabling a transition to future-ready systems and intelligent data architectures.
Build and optimize scalable data pipelines and architectures using Databricks, Python, Dataverse, and advanced SQL, transforming raw SAP, MES, LIMS, and other operational data into accessible analytics-ready foundations.
Bridge business and technology by translating stakeholder needs into clear business requirements and precise technical specifications, working with supply chain planners, quality teams, and manufacturing stakeholders across drug substance and drug product functions.
Design, test, and implement agentic AI workflows and machine learning models to automate decision-making and generate proactive business insights.
Create and deploy custom, user-friendly WebApps to support business operations, data entry, and interaction with underlying data models.
Develop in modern environments using VS Code and AI coding assistants such as Cline, with GitHub for version control of scripts, applications, and data models.
Partner with data architects to ensure scalable, secure, and efficient data models that support both analytical and transactional workloads.
Document processes, code, data models, and system architectures, and provide training and technical support for end-users as new technologies are rolled out.

What you bring

Strong expertise in Python and advanced SQL.
Hands-on experience with Databricks Lakehouse, Delta Lake, and Unity Catalog for robust data governance.
Experience building custom data products and rapid-prototyping front ends (including Streamlit and React).
Familiarity with SAP (or similar ERP/manufacturing systems) as a primary source for complex supply chain and manufacturing data.
Proven ability to gather complex requirements from non-technical stakeholders and translate them into actionable technical roadmaps.
Familiarity with or strong interest in Agentic Modeling, Large Language Models (LLMs), and integrating AI agents into workflows (for example, Copilot Studio).
Experience working in Agile/Scrum using Jira Cloud and maintaining technical documentation with Confluence.
Comfort working in VS Code and using AI-assisted coding tools to accelerate development.
Strong experience with GitHub including PR-based collaboration and CI/CD.
Strong analytical and problem-solving skills to tackle complex data challenges.
Ability to build trust-based relationships across technical and non-technical stakeholders and drive change.
Excellent written and verbal communication skills across global audiences.
Independently lead projects with a proactive approach to delivering results.

Preferred domain knowledge

Familiarity with pharmaceutical drug substance or drug product supply chain operations, including concepts such as process orders, batch genealogy, MRP/MPS planning, tech transfer, quality management, or global distribution.
Experience with SAP (MM/PP/QM/SD) or related manufacturing execution systems (MES/LIMS) is a strong plus.

Benefits

Medical, dental, vision healthcare and other insurance benefits (for employee and family)
Retirement benefits including 401(k)
Paid holidays
Vacation
Compassionate and sick days

Location: Rahway, NJ (onsite)
Work arrangement: Hybrid
Salary range: USD 129,000 - 203,100 per year
Employee status: Regular
End date: 11/13/2026
Requisition ID: R424311
Minimum experience: 5 years
Education: M.S. degree in Computer Science, Data Science, Engineering, Supply Chain Management, Operations Research, or a closely-related field

View full job page →

3d ago

Data Engineer, AI Governance

FIS Global Jacksonville, FL

Jacksonville, FL

3d ago

Full time Artificial Intelligence Data Engineer Data Governance

Apply Now

Job Description

FIS Global is hiring a Data Engineer, AI Governance in Jacksonville, FL for an onsite role. In this position, you will design, build, and maintain data foundations that enable responsible, trusted, and scalable AI across the enterprise.

You will focus on backend data pipelines, governance, lineage, data quality, metadata management, and auditability, partnering with business and technology stakeholders to keep data reliable and compliant for both AI and analytics use cases.

Responsibilities

Design, develop, and maintain scalable backend data pipelines, integrations, and data platforms that support AI governance and analytics initiatives.
Consolidate and transform data from multiple enterprise sources into trusted, governed datasets for AI and business consumption.
Establish and maintain end-to-end data lineage to provide visibility into data origins, transformations, and consumption.
Collaborate with business, technology, and governance teams to define data ownership, stewardship, and accountability models.
Implement data quality controls, monitoring frameworks, and observability practices to identify and resolve data issues proactively.
Develop metadata management processes to improve discoverability, traceability, and governance compliance.
Support data access governance by helping define, automate, and monitor data access controls and permissions.
Build and maintain data validation, reconciliation, and audit capabilities to strengthen trust in enterprise data assets.
Work with AI, machine learning, and analytics teams to ensure governed data is available for model development and operational use cases.
Recommend and implement automation opportunities that improve data reliability, efficiency, and operational excellence.

Requirements

Experience as a Data Engineer, Data Platform Engineer, Data Integration Engineer, or similar role supporting large-scale data environments.
Strong experience building and supporting backend data pipelines using modern data engineering practices.
Experience with data integration, transformation, cleansing, and consolidation across multiple data sources.
Strong understanding of data governance, data quality, metadata management, and data lifecycle management.
Experience establishing and documenting data lineage, data flows, and data dependencies.
Proficiency with SQL and data modeling concepts in enterprise data environments.
Experience with cloud-based data platforms, data warehouses, and data lake architectures.
Knowledge of data security, access management, and governance principles.
Strong analytical, troubleshooting, and problem-solving skills with attention to detail.
Passion for creating high-quality, trusted, well-governed data products.

Preferred Qualifications

Experience supporting AI, machine learning, or responsible AI governance programs.
Familiarity with data catalog, metadata management, data lineage, or data observability platforms.
Experience implementing enterprise data quality and monitoring frameworks.
Experience within financial services, fintech, payments, banking, or other highly regulated industries.

Benefits

Internal career growth opportunities
Opportunities to innovate in fintech
Inclusive and diverse team atmosphere
Professional and personal development
Resources to contribute to your community

Type of hire: Experienced (relevant combination of work and education).
Position type: Full time.

View full job page →

3d ago

Senior Data Engineer

Teal Health United States

United States

3d ago

$90 - $125/hr Contract Senior 3d Design Tools Aws Data Engineer

Apply Now

Job Description

Senior Data Engineer (contract) at Teal Health, focused on owning the data warehouse and delivering trusted, analysis-ready models through an Omni semantic layer.

Responsibilities

Own Teal’s data warehouse, including architecture, performance, reliability, and documentation
Manage and extend Fivetran ingestion across source systems; troubleshoot connector and schema issues
Build and maintain a dbt transformation layer using a staging reporting approach from raw sources to business-ready models
Support forecasting and reporting use cases across marketing, HubSpot, AWS, PostHog, and inventory
Own the Omni data layer: define relationships, joins, and modeled fields for cross-system questions
Partner with the analytics lead to ensure dashboards sit on accurate, consistent, and well-defined data layers
Take on early integration work: connect new data sources and support partner data flows
Write dbt tests and documentation to improve trust and understanding of warehouse data
Set standards for data modeling, testing, and documentation, and help shape team growth
Execute work from a ticketed backlog ( Jira or similar), including scoping requests and keeping priorities visible
Handle sensitive health data in line with HIPAA and Teal’s data governance practices

Requirements

5+ years of experience in data engineering or analytics engineering, including ownership of a production data warehouse
Strong SQL skills, including window functions, date logic, and complex joins across marketing, product, and operational data
Fluency with Postgres/Redshift/Snowflake-style SQL (for example: ::type casts, date_trunc(), and similar)
Hands-on experience with dbt-core, including:
- Models and sources
- dbt_project.yml and packages.yml configuration
- Jinja templating
- Custom macros (example provided: generate_schema_name)
- dbt’s testing and docs framework
Experience designing and developing a data warehouse using a layered staging reporting architecture
Experience working with Fivetran-landed data and the ability to dig into each source’s raw schema details
Working knowledge of common dbt packages such as dbt_utils and dbt_date (including surrogate keys, date spines, etc.)
Strong habits for documentation and organized, ticket-driven execution

Technologies

SQL
Postgres
Redshift
Snowflake
dbt-core
Jinja
Fivetran
dbt_utils
dbt_date
Omni
HubSpot
PostHog
AWS
HIPAA
Jira

Location

United States (hybrid)
Preferred: Hybrid or Full-time in Office (San Francisco)
Open to Remote-US

Employment Type

Contract
~5–20 hours per week (project volume fluctuates)

Reporting Line and Department

Department: Engineering
Reports to: Head of Engineering

Contract Compensation

Expected hourly rate: $90–$125 per hour

Preferred

Experience with Omni or a similar semantic/modeling layer (Looker, LookML, etc.)
Python for scripting, integrations, or pipeline tooling
Experience with healthcare or EHR integrations
Deeper Jinja experience, including control flow and variables within dbt models
Experience at an early-stage or fast-growing company where you owned a data function with limited support