Data Transforms

Data Transforms Engineer Jobs

42 jobs found

Today

Data Engineer

First Dakota National Bank Sioux Falls, SD

Sioux Falls, SD

Today

Full time Analytics Automation Developer Azure Data Engineer

Apply Now

Job Description

First Dakota National Bank is seeking a Data Engineer (any First Dakota location) to help develop, run, and continuously improve the Bank’s enterprise data platform. In this onsite role, you will design and maintain scalable ETL/ELT pipelines and data products using Microsoft Fabric and related technologies to support analytics, reporting, and data governance.

What you’ll do

Design, develop, test, and maintain data pipelines that extract, transform, and load data into Microsoft Fabric.
Connect and integrate data from core banking systems, third-party applications, APIs, files, and cloud-based platforms.
Build batch and near real-time processing solutions using SQL, Python, Spark, and related technologies.
Create reusable frameworks for ingestion, transformation, orchestration, and monitoring.
Transform and enrich data using business rules, auditing, validation, and data quality controls.
Develop fact, dimension, and analytical models to support Power BI and enterprise reporting.
Implement monitoring, alerting, error handling, and operational documentation for data platform processes.
Support workflow automation and orchestration initiatives using Fabric, SharePoint, Power Apps, Power Automate, and related technologies as appropriate.
Collaborate with data analysts, business stakeholders, and technology teams to deliver reliable, scalable data products.
Participate in data governance, security, metadata management, and documentation efforts.
Troubleshoot production issues and optimize for performance, reliability, and maintainability.
Perform other duties as assigned.

Requirements

Bachelor’s degree in computer science, Information Systems, Data Engineering, or a related field (or equivalent work experience).
3+ years of experience as a Data Engineer, ETL Developer, Data Warehouse Developer, or similar role.
Expert proficiency in SQL, ETL/ELT processes, and data integration.
Expert proficiency in dimensional modeling (Kimball / Star Schema).
Deep experience with one or more of: Microsoft Fabric, Databricks, Snowflake, Azure Data Services.
Professional experience in Python and Spark or other distributed data processing platforms.
Professional experience with Git and source control concepts, plus CI/CD and DevOps practices.
Experience with one or more of: dbt, Azure Data Factory, Kafka or streaming technologies, and REST APIs and application integrations.
Strong understanding of data quality and testing practices, enterprise data governance, security and access controls, and data lineage and documentation.
Experience with financial institutions, banking systems, or Jack Henry platforms is preferred.
Demonstrated positive attitude and good work ethic, with strong attention to detail and ability to maintain strict confidentiality.
Ability to process tasks accurately and efficiently, comply with Bank policies and procedures, and communicate effectively with clientele, coworkers, and departments.
Strong organizational skills and ability to work well in a team environment; punctuality is a must.

Location and work environment

Location: Sioux Falls, SD (onsite)
Office environment: moderate noise level and interaction with others in an office setting

Technologies you’ll work with

Microsoft Fabric
Python, SQL, Spark
Power BI
dbt
Azure services
SharePoint, Power Apps, Power Automate
Databricks, Snowflake
Git, CI/CD, DevOps
Azure Data Factory, Kafka
REST APIs, Jack Henry

Physical demands

Light physical demands, including bending and sitting; occasionally lifting up to 10 pounds with or without reasonable accommodation.

Requirements of all employees

Adaptability
Attendance and punctuality
Career apparel compliance
Customer service
Dependability
Design and attention to detail
Initiative, innovation, and interpersonal respect
Judgment, motivation, communication, and organization support
Planning and problem solving
Professionalism, quality, and safety/security procedures
Team work
Technical skills and ongoing knowledge building

View full job page →

Today

Senior Associate - Analytics Engineer

New York Life New York, NY

New York, NY

Today

$100k - $143k/yr Full time Analytics Analytics Engineer Aws Cloud

Apply Now

Job Description

New York Life is hiring an Analytics Engineer to strengthen the Data Services team within NYL Enterprise Intelligence. In this hybrid role based in New York, you will help build reliable analytics foundations that support enterprise-scale intelligence and actionable decisioning. This position offers a salary range of $100,000 to $143,000 per year, with discretionary bonus eligibility and overtime eligible: Exempt.

What you’ll do

As a Senior Associate in Tech Data AI Ventures, you will contribute to the development and upkeep of analytics-ready data assets. Your work will combine data transformation, modeling, testing, and reporting to ensure metrics are accurate, accessible, and dependable for enterprise use.

Develop data transformation logic and maintain existing data models using dbt and SQL.
Write and optimize SQL queries for transformation and analysis to support accurate, actionable data.
Assist with implementing and maintaining data quality checks and testing procedures.
Contribute to documentation of data models, transformations, and business logic to improve consistency, reliability, and maintainability.
Build and maintain reports and dashboards using visualization tools to surface insights.
Support delivery of business and operational metrics in partnership with Analytics and senior team members.
Collaborate with team members and stakeholders to achieve shared outcomes.
Participate in cross-functional meetings to understand business requirements and translate them into data needs.
Build strong relationships across engineering and analytics teams, supporting the continued development of enterprise intelligence capabilities.

What you bring

1–3 years of experience in analytics engineering, data engineering, or a related field.
Proficiency in SQL, including writing and optimizing queries for data transformation and analysis.
Familiarity with data warehousing concepts and analytics best practices.
Strong attention to detail and commitment to data quality, accuracy, and reliability.
Ability to work collaboratively in a fast-paced, team-oriented environment across engineering, analytics, and business teams.

Tools and technologies

dbt, SQL
GCP, AWS
Databricks, BigQuery
Looker, Tableau

Requisition ID: 95191
Department / Job Function: Tech Data AI Ventures
Location: New York, NY (Hybrid, 3 days per week)
Job Level: LEVELPF3
Overtime eligible: Exempt
Discretionary bonus eligible: Yes
Sales bonus eligible: No

View full job page →

Today

Senior Data Engineer

ConnectOne Bank Englewood Cliffs, NJ

Englewood Cliffs, NJ

Today

$160k - $180k/yr Full time Senior Apache Airflow AWS

Apply Now

Job Description

ConnectOne Bank is building dependable, secure data foundations to support advanced analytics and critical business decision-making. In this hybrid role based in Englewood Cliffs, NJ, the Senior Data Engineer will act as a primary architect and builder of the bank’s data infrastructure within an AWS ecosystem.

You’ll design and maintain scalable pipelines and data models, ensure data quality through testing and alerting, and help standardize engineering practices across the data team.

Role focus

Design, develop, and maintain scalable data pipelines, robust data models, and a secure data warehouse on AWS
Enable reliable acquisition, transformation, and storage of critical business data for analytical use

Responsibilities

Support the Bank’s People First focus and rules of engagement by maintaining a professional demeanor, serving as an active member of the CNOB team, and providing excellent client service while helping make CNOB “A Better Place to Be”
Architect, build, and maintain data pipelines using Apache Airflow, including seamless data movement across AWS S3 and Amazon Redshift
Design and implement scalable data models using dbt Labs to transform raw data into clean datasets optimized for analytical querying
Help set up new data sources by designing secure, efficient ingestion into the data lake/warehouse from third-party APIs, transactional databases, and external platforms
Troubleshoot bottlenecks and optimize Redshift query performance by maintaining strong understanding of the underlying data model, while implementing rigorous data quality testing and alerting
Act as a subject matter expert on data architecture, establishing best practices for code reviews, version control, and CI/CD workflows within the data engineering team

Requirements

People First interest and ability
Comprehensive understanding of cloud computing and cloud-native data architectures, specifically within AWS
Deep knowledge of modern data warehousing concepts, including schema design and data modeling methodologies
Understanding of data governance, data privacy regulations, and enterprise security best practices
Expert-level proficiency in SQL and Python
Hands-on mastery of dbt Labs, Apache Airflow, Amazon Redshift, and AWS S3
Strong proficiency with Git/GitHub and command-line interfaces
7+ years of progressive, dedicated experience in Data Engineering, Data Architecture, or a heavily overlapping software engineering field
Ability to analyze complex pipeline failures, identify root causes in underlying data, and implement permanent solutions
Aptitude to listen to non-technical stakeholders, understand analytical needs, and translate them into technical data structures
Ability to thrive in a remote work environment by managing time effectively and driving large-scale projects from conception to deployment with minimal oversight
Meticulousness focused on absolute data accuracy and system reliability

Technologies

Apache Airflow
AWS S3
Amazon Redshift
dbt Labs
SQL
Python
Git
GitHub
CI/CD

Compensation

USD 160,000 - 180,000 per year

Benefits

World class health, vision, and dental benefits on day one
401k with employer match
Hybrid work from home (depending on role)
Employee appreciation events (team building, softball games, food truck days, etc.)
Employee assistance programs (EAP)
Wellness programs (flu shot, preventive care, health programs and services discounts, etc.)
Tuition reimbursement
Employee Discount perks
CNOB Community Service Events

Bonus points for

AWS Certified Data Engineer, AWS Certified Solutions Architect, or official dbt Certification (preferred but not required)

View full job page →

1d ago

Lead Cloud Data Engineer

Baptist Health South Florida Miami, FL

Miami, FL

1d ago

$129k - $167k/yr Full time Backend Developer Apache Airflow Application Security

Apply Now

Job Description

Baptist Health South Florida is seeking a Lead Cloud Data Engineer to partner with multiple teams on designing, building, and maintaining data systems and applications. In this Florida onsite role, you will help deliver scalable, reliable, automated data pipelines and provide production support with adherence to defined service levels, with a strong focus on healthcare data security and compliance.

What you will do

Collaborate closely with cross-functional teams to design, build, and maintain data systems and applications.
Analyze data storage structures and implement performant automated pipelines that are reliable and scalable in a fast-growing data environment.
Apply data modeling principles and modern data platform knowledge to design and implement data pipeline solutions.
Provide production support while adhering to defined SLA(s).
Test and validate deliverables using industry best practices and communicate effectively with peers in technology teams.

Required experience and qualifications

Self-starter with strong organization, problem-solving, and learning skills.
Strong technical skills and demonstrated attention to detail.
Knowledge of data security, compliance, and policies in healthcare services.
Experience testing and validating according to industry best practices, with effective communication to technology peers.
Strong interpersonal skills plus written and oral communication ability across all levels of the organization.
Bachelor’s degree in computer science or related.
At least 10 years with a Bachelors, and at least 8 years with a Masters (recent experience) in data engineering and end-to-end automation of data pipelines.
Strong understanding of relational and NoSQL databases and data structures.
Understanding of data science and business intelligence workflows.
Programming experience ideally in Python and Java.
Experience with large-scale data warehousing and analytics projects, including AWS technologies.
Proven track record of successful written communication, with technical depth and business awareness.
Experience with machine learning toolkits.
Experience in production support and troubleshooting.
10+ years hands-on with modern data platforms (including Snowflake, Redshift).
10+ years hands-on in data warehouse design/architecture with AWS and Data Vault 2.

Additional preferred and platform experience

Experience with dimensional or dimensional modeling, orchestration tools like Airflow and Control-M, and containerization (Docker and Kubernetes).
Experience with CloudFormation and Terraform, plus AWS services such as S3, AWS Glue, Athena, Lake Formation, DynamoDB, DMS, and RDS.
Agile methodology and DevOps process experience.
Modern data integration tools experience such as dbt or Informatica Power Center, including AWS Glue or similar.
3+ years of data modeling, with familiarity with AI/ML and BI tools (a plus).
Big Data Engineering, Python, and SQL.
Healthcare dimensional modeling experience designing star and snowflake schemas for healthcare analytics, clinical quality measures, operational efficiency, and revenue cycle reporting.
Medical ontologies and semantics experience, including mapping internal hospital codes to standardized vocabularies such as ICD-10, SNOMED-CT, LOINC, and RxNorm.
Google Cloud Platform production engineering experience, including GCS, Dataproc, BigQuery, Cloud Run, Cloud Run Functions, and Cloud Composer.
Cloud security and identity experience, including role-based access control with Cloud IAM and securing credentials with GCP Secret Manager, with observance of strict HIPAA rules for PHI.
Python and PySpark experience, including tuning distributed jobs, handling partition skew, and optimizing Spark resource allocation.
dbt experience using dbt Core and/or dbt Cloud, including reusable SQL models, automated testing, and data dictionary documentation.
Workflow orchestration experience using Apache Airflow (Cloud Composer) and enterprise schedulers like Control-M.
Enterprise data governance experience integrating pipelines with Collibra for business glossaries, technical dictionaries, and automated lineage tracking.

Salary and role details

Location: Florida (onsite)
Estimated salary range: USD 128,671 - 167,273 per year, depending on experience
Experience level: 10+ years

Technologies

SQL, Python, Java
AWS: S3, Hadoop, Hive, Redshift, AWS, AWS Glue, Athena, Lake Formation, DynamoDB, DMS, RDS
Snowflake, Data Vault 2
Airflow, Control-M
Docker, Kubernetes, CloudFormation, Terraform
dbt, Informatica Power Center
Healthcare standards: ICD-10, SNOMED-CT, LOINC, RxNorm
GCP: Google Cloud Platform, Google Cloud Storage (GCS), Dataproc, Spark, PySpark, BigQuery, Cloud Run, Cloud Run Functions, Cloud Composer, Cloud IAM, GCP Secret Manager
Security/compliance: HIPAA, PHI
Governance: Collibra

Benefits

Career growth and development opportunities, with clear pathways and ongoing support
Comprehensive health and wellness resources beyond traditional benefits
A wellness program that can help employees eliminate their medical plan deductible, reducing out-of-pocket healthcare costs
Tuition reimbursement to support continued learning and advancement

View full job page →

2d ago

Lead Data Engineer with Databricks & DBT Exp

Realign LLC Irvine, CA

Irvine, CA

2d ago

$120k+/yr Full time Manager Apache Airflow Automation

Apply Now

Job Description

Realign LLC is seeking a Lead Data Engineer to support enterprise-scale data engineering initiatives from an onsite base in Irvine, CA. In this role, you will lead a Data Engineering Tower/Pod focused on end-to-end data delivery, platform stability, and consistent SLA adherence, with coordination across onshore, nearshore, and offshore teams.

You will bring hands-on platform and pipeline experience to drive scalable data engineering, strengthen reliability through monitoring and production support readiness, and align technical execution with portfolio management and technology leadership stakeholders.

Key Responsibilities

Lead and manage the Data Engineering Tower, overseeing multiple teams and delivery pods.
Own end-to-end data platform delivery, operational excellence, and SLA adherence.
Drive development of scalable data pipelines, focusing on data quality and platform reliability.
Manage cross-functional dependencies, communicate with stakeholders, and escalate issues as needed.
Ensure high availability through monitoring, alerting, and production support readiness.
Collaborate with business, portfolio management, and technology leadership teams.
Define and implement best practices for platform governance, operational readiness, and performance optimization.

Required Qualifications

Strong experience with Databricks, DBT, Apache Spark, and performance tuning.
Expertise with Apache Airflow and/or Astronomer for workflow orchestration and DAG design.
Deep understanding of data engineering, ETL/ELT, and large-scale data platforms.
Experience with CI/CD, DevOps, Infrastructure as Code (IaC), and automation.
Proven platform management, monitoring, and incident management experience.
Strong leadership and stakeholder management skills.

Technologies

Databricks, DBT, Apache Spark
Performance tuning
Apache Airflow, Astronomer
ETL/ELT
CI/CD, DevOps, Infrastructure as Code (IaC), automation
Monitoring, incident management

Preferred Domain Experience

Asset Management
Investment Management
Capital Markets
Portfolio & Market Data Platforms

Location: Irvine, CA (onsite)
Compensation: USD 120,000 per year

View full job page →

2d ago

Data Engineer

Community Financial System, Inc. Syracuse, NY

Syracuse, NY

2d ago

$66k - $109k/yr Full time Apache Airflow Azure Data Engineer Cloud

Apply Now

Job Description

Design, build, and maintain data pipelines and data infrastructure for data-driven initiatives in Syracuse, NY.

Responsibilities

Design and implement data pipelines to extract, transform, and load (ETL) data from multiple sources into a central repository or data lake (Snowflake)
Build and maintain scalable, robust data infrastructure, including relational databases, data lakes, and distributed computing systems for large data volumes
Collaborate with data analysts to understand metrics and data needs, providing access to required datasets and data systems
Implement data validation and cleaning processes to maintain data quality and monitor pipelines for errors or anomalies
Optimize data storage and retrieval for improved performance and reduced latency using methods such as indexing, partitioning, and caching
Apply data security and privacy measures, including encryption, access controls, and compliance with data governance policies and regulations
Support additional key initiatives and related Data & Analytics projects as needed
Maintain big-picture understanding while producing detailed operational workflows, processes, and requirements
Develop and implement data models and support data modeling and warehousing requirements across the organization
Demonstrate ongoing compliance with applicable laws, regulations, policies, procedures, internal controls, and required training
Serve as an integral member of the Data & Analytics Department by providing assistance wherever needed to help the department and the bank achieve goals

Requirements

Bachelor’s Degree in Computer Science, Information Systems, or another related field
3+ years as a data engineer (minimum 3 years of proven experience)
Experience with Snowflake and DBT
Experience with ELT tools, preferably Astronomer and Apache Airflow
Experience with BI tools, preferably Power BI
Strong working knowledge of Python, SQL, and Git
Familiarity with data modeling and data warehousing concepts, preferably with modeling tools such as DBT or DataIKU
Working knowledge of cloud data warehousing, preferably Snowflake, Azure, or similar cloud infrastructure
Excellent verbal and written communication skills in a technical and user-friendly manner; ability to adapt to changing priorities
Ability to collaborate and work effectively in a team environment
Prior experience/knowledge of AI/ML practices is given preference, including leveraging advanced analytics techniques using AI and ML
Banking experience preferred

Technologies

Snowflake, DBT, Astronomer, Apache Airflow
Power BI
Python, SQL, Git
DataIKU
Azure
ETL, ELT

Compensation

USD 66,000 - 108,804 per year

Benefits

11 paid holidays
Paid vacation
Medical, Vision & Dental insurance
401K with generous match
Pension
Tuition Reimbursement
Banking discounts

Physical Requirements

May be required to stand, walk, or sit
May require use of hands and fingers, handle or feel, reach with hands or arms, and speak and hear
Occasionally may be required to lift or move up to 25 pounds
Specific vision abilities required: close vision and the ability to focus

Location: Syracuse, NY (onsite)

View full job page →

3d ago

Senior Data Engineer

Optum Eden Prairie, MN 🇺🇸 Remote

Eden Prairie, MN

3d ago

$92k - $164k/yr Full time Senior Apache Airflow Api Integration

Apply Now

Job Description

Optum is hiring a Senior Data Engineer to design and run governed data platforms and pipelines for healthcare data solutions.

Responsibilities

Lead the design and delivery of scalable, governed data engineering solutions, including technical direction, implementation planning, risk management, and production readiness for data products used by business, analytics, and AI/ML stakeholders
Develop and manage ETL/ELT pipelines to ingest, transform, and integrate data from heterogeneous sources (relational databases, APIs, file shares, and end-user-computing assets)
Create Python and PySpark solutions for distributed processing, automation, integration, and reusable transformation logic
Improve data platform performance and manage cloud consumption using SQL tuning, Spark job optimization, storage format selection, partitioning, cluster configuration, and workload scheduling
Build and optimize workflows on Azure Cloud services and Databricks, including medallion (bronze/silver/gold) layering and orchestrated pipelines
Migrate business logic from legacy and end-user-computing systems into documented, testable transformations over governed tables
Define and implement data quality practices, including validation, reconciliation, and parity testing to ensure migrated pipelines are accurate, reliable, auditable, and consistent
Establish data lineage, ownership, and cataloguing, including documented rollup and allocation rules when sources operate at different grains
Implement security and compliance controls for PHI/PII and confidential business data in accordance with HIPAA or similar regulations, including de-identification and masking for controlled sharing
Build and improve CI/CD, version control, and automated testing for data pipelines
Implement production monitoring, alerting, logging, and operational runbooks for reliability, timely issue detection, incident response, and continuous improvement
Support AI/ML and advanced analytics by provisioning trusted data, scalable infrastructure, and governed consumption patterns for data scientists, analysts, and partners
Design and publish reusable data products, data services, APIs, and consumption layers so downstream applications and analytics/AI-ML use cases consume trusted data consistently
Partner with product owners, business stakeholders, architects, data scientists, analysts, and engineering teams to translate requirements into scalable solutions and communicate technical tradeoffs
Design, develop, and deploy AI-enabled data solutions for complex business challenges and responsible AI use, leveraging AI-powered development tools to improve productivity and delivery quality
Mentor data engineers and analysts through design reviews, code reviews, and best-practice guidance

Requirements

6+ years overall experience in data engineering, data warehousing, distributed data processing, or enterprise data platform delivery
6+ years of experience with SQL, data modeling, and performance tuning for large-scale systems
3+ years with ETL/ELT frameworks, pipeline orchestration, and heterogeneous source-system integration
3+ years with Python for data processing, automation, and integration
3+ years with Databricks, Spark/PySpark, and Delta Lake
3+ years with public cloud platforms such as Azure or GCP
1+ year integrating data from heterogeneous source systems
1+ year in data quality, validation, reconciliation, observability, and production support for large-scale data pipelines
1+ year with Git, CI/CD platforms, automated testing frameworks, release management, and deployment practices for production data pipelines
1+ year with AI-powered development tools used responsibly to improve engineering productivity and delivery quality

Technologies

Python, PySpark, SQL, Databricks, Spark, Delta Lake
Azure, GCP
ETL, ELT
Git, CI/CD
HIPAA
Azure Data Factory, Airflow, dbt
Unity Catalog, Microsoft Purview
APIs

Benefits

Comprehensive benefits package
Incentive and recognition programs
Equity stock purchase
401k contribution

Work Model

Remote within the U.S.
For hires in the Minneapolis or Washington, D.C. area: required to work in the office a minimum of four days per week

Preferred Qualifications

Experience with batch and near-real-time patterns, including event-driven ingestion, streaming pipelines, or comparable technologies
Experience with Azure Data Factory or comparable orchestration tooling such as Airflow, dbt, or similar
Experience migrating legacy or end-user-computing (spreadsheet- or macro-based) business logic into governed data pipelines
Experience with data governance, lineage, and catalog tooling (Unity Catalog, Microsoft Purview, or similar)
Familiarity with AI/ML frameworks and supporting infrastructure for data science workflows
Experience working with PHI/PII and restricted client data in compliance with HIPAA or similar regulations, including de-identification and masking
Healthcare payer, actuarial, pricing, or financial-modelling domain experience

Pay

$91,700 - $163,700 per year, based on full-time employment and several factors

Application Details

Posted for a minimum of 2 business days or until a sufficient candidate pool has been collected
Posting may come down early due to volume of applicants