Azure Data Lake Storage

Azure Data Lake Storage Engineer Jobs

14 jobs found

10d ago

Lead Data Engineer / Data Architect

Glint Tech Solutions LLC Charlotte, NC

Charlotte, NC

10d ago

Full time Azure Data Factory Azure Data Lake Storage Azure Synapse Analytics

Apply Now

Job Description

This on-site opportunity in Charlotte, NC focuses on shaping enterprise data strategy and modernizing platforms across architecture modernization, migrations, and cloud-native microservices.

Responsibilities

Design and own enterprise data architecture to support high-volume retail eCommerce operations.
Develop and maintain ETL and ELT pipelines using Azure Data Factory, Synapse Analytics, and Databricks with PySpark.
Model conceptual, logical, and physical data structures for efficient storage and retrieval.
Implement data warehousing approaches such as dimensional modeling, star schemas, Data Mesh, Delta Lakehouse, and Data Vault.
Lead database migrations and microservices migrations toward unified, cloud-native solutions.
Integrate data from databases, APIs, and external providers via API integrations and message queues.
Embed metadata management and governance within cloud-based data platforms.
Guide and mentor cross-functional engineering teams, shaping technical strategy and stakeholder communications.

Requirements

8+ years of overall experience with at least 1 year in an Architect-level role.
3+ years of hands-on Azure Data Factory and Azure Synapse Analytics experience.
Deep Microsoft Azure expertise including Data Lake Storage, Databricks, PySpark, Cosmos DB, and PostgreSQL.
Strong SQL proficiency for complex queries, transformations, and cloud performance tuning.
Experience with Azure MI, Cosmos DB, Analysis Services, and Informix.
Proficiency in Python and Python-based ETL tools.
Shell scripting experience — Bash, Unix, or Windows.
Proven ability to lead teams, define technical strategy, and communicate across organizational levels.

Technologies

Azure Data Factory
Azure Synapse Analytics
Databricks
PySpark
Data Lake Storage
Cosmos DB
PostgreSQL
SQL
Python
Bash
Unix
Windows
Azure MI
Analysis Services
Informix
Elasticsearch
Docker
Kubernetes
Delta Lake
Delta Lakehouse
Data Mesh
Data Vault

Nice-to-have skills

Elasticsearch experience
Docker and Kubernetes familiarity
Performance tuning and pipeline optimization expertise
Azure, Databricks, or relevant cloud certifications

Company overview

Glint Tech Solutions is a women-owned global staffing and IT recruiting firm connecting top technical talent with leading enterprise clients across the United States and Canada.

View full job page →

12d ago

Data Engineer

Asylon Norristown, PA

Norristown, PA

12d ago

$120k - $160k/yr Full time Python Azure Data Lake Azure Data Lake Storage

Apply Now

Job Description

Asylon is seeking an experienced Data Engineer to design and sustain real-time data streams from a fleet of robots for live monitoring and ML model training.

Responsibilities

Architect and optimize high-throughput, low-latency data pipelines leveraging Azure cloud services and on-prem components, tailored for streaming and time-series workloads.
Capture, manage, and process robot telemetry data such as position, velocity, and battery status.
Build scalable ingestion, storage, and retrieval solutions for high-resolution video and Lidar point cloud data.
Provide streamlined data access to R&D teams for analysis and experimentation.
Partner with software engineers to specify data formats and transmission protocols from robots to the cloud.
Implement resilience for intermittent connectivity and guarantee data completeness across pipelines.

Requirements

Minimum of 3 years of hands-on Data Engineering experience with a strong focus on real-time or streaming data.
Experience with data storage services, including cloud-based options such as Azure Data Lake and self-hosted solutions.
Expert-level proficiency in Python and data query languages.
Experience with streaming data processing frameworks.
Experience with time-series databases.
Bachelor’s degree in a related field.
Experience in Robotics, IoT, or Autonomous Systems domains: 5 years (Required).

Technologies

Azure
Azure Data Lake
Python
Kubernetes

Benefits

401(k)
Dental insurance
Health insurance
Paid time off
Vision insurance

Job Details

Location: Norristown, PA (onsite)
Work arrangement: In person
Employment type: Full-time
Salary: USD 120,000 - 160,000 per year
Schedule: Monday to Friday; weekends as needed

Application Question

Are you a U.S. Person as defined by 22 C.F.R. §120.62. This includes U.S. Citizens, lawful permanent residents, refugees, or asylees. Yes or No

View full job page →

17d ago

Azure Data Engineer – AI & Data Platform

Intone Networks Remote 🌍 Remote

Remote

17d ago

Full time Artificial Intelligence Azure Azure Data Factory

Apply Now

Job Description

Intone Networks offers a remote position that centers on cloud data platforms, scalable pipelines, and AI driven solutions. You will work with Azure data services to boost data engineering productivity and deliver insights that inform business decisions.

Benefits and Culture

This fully remote role enables collaboration across cross-functional teams and hands-on work with Azure data services and AI technologies to deliver scalable data platforms that drive business insights.

Responsibilities

Data Engineering & Platform Development: Design, develop, and maintain scalable data ingestion, transformation, and ETL/ELT pipelines using Azure services. Build and optimize data solutions with Azure Data Factory, Azure Synapse Analytics, Microsoft Fabric, Azure Databricks, and Azure Data Lake Storage. Develop batch and real-time data processing solutions using Spark, SQL, and Python. Implement data modeling solutions aligned with enterprise standards and business requirements.
AI & Generative AI Enablement: Develop AI-assisted solutions for automated data pipeline generation, code acceleration, and notebook creation. Integrate Azure OpenAI and Azure AI Services into data engineering workflows. Build intelligent data quality monitoring, anomaly detection, and predictive analytics solutions. Design and implement AI powered agents for operational monitoring, user notifications, and workflow automation. Evaluate and implement emerging AI technologies to improve engineering efficiency.
Data Governance & Quality: Implement data quality frameworks, validation rules, and monitoring mechanisms. Ensure compliance with security, privacy, and governance requirements. Establish metadata management, lineage tracking, and data cataloging processes. Support audit, compliance, and regulatory reporting requirements.
Operational Support: Monitor and troubleshoot data pipelines and platform issues. Perform root cause analysis and implement preventive measures. Support production deployments, release management, and operational excellence initiatives. Collaborate with cross-functional teams to resolve data and platform issues.

Requirements

SQL
Azure Data Factory (ADF)
Azure Synapse Analytics
Microsoft Fabric
Azure Databricks
Azure Data Lake Storage (ADLS)
Python
PySpark
Data Modeling and Data Warehousing
Azure DevOps (ADO), CI/CD
Git
Version Control
REST APIs and Data Integration
AI & Advanced Skills
Azure OpenAI Service
Azure AI Services
Generative AI and Large Language Models (LLMs)
AI Agents and Workflow Automation
Retrieval-Augmented Generation (RAG)
Prompt Engineering
Intelligent Monitoring and AI-driven Operations

Technologies

SQL
Azure Data Factory (ADF)
Azure Synapse Analytics
Microsoft Fabric
Azure Databricks
Azure Data Lake Storage (ADLS)
Spark
Python
PySpark
Azure DevOps (ADO)
Git
REST APIs
Azure OpenAI Service
Azure AI Services
Generative AI
Large Language Models (LLMs)
AI Agents
Retrieval-Augmented Generation (RAG)
Prompt Engineering
Intelligent Monitoring
AI-driven Operations

Role Overview

We are seeking an experienced Azure Data Engineer with expertise in cloud data platforms, modern data engineering practices, and AI driven solutions. The candidate will be responsible for designing, building, and optimizing scalable data pipelines, integrating enterprise data sources, and leveraging Azure AI and Generative AI technologies to improve data engineering productivity and business insights.

Key Competencies

Problem Solving & Analytical Thinking
Stakeholder Management
Collaboration & Communication
Innovation & Continuous Improvement
Operational Excellence
AI Adoption & Automation Mindset

View full job page →

23d ago

Data Analytics Engineer

Franciscan Health Mishawaka, IN 📍 Indiana Remote

Mishawaka, IN

23d ago

$83k - $114k/yr Full time Analytics Azure Data Factory Azure Data Lake Storage

Apply Now

Job Description

Franciscan Health seeks a Data Analytics Engineer to design and assemble data structures within data marts, data lakes, and data warehouses, while administering ETL processes to deliver analytics-ready data to stakeholders. The role includes mentoring peers and fostering a data driven culture across clinical and operational teams. This position is based in Mishawaka, Indiana with remote work options.

Responsibilities

Design and build tables across data marts, data lakes, and data warehouses, handling all related extraction, transformation, and loading tasks
Collaborate across diverse environments to develop systems that collect, manage, and convert raw data into usable information for analytics developers and business analysts
Aim to make data readily accessible to support performance evaluation and optimization efforts
Develop and validate architectures that enable data extraction and transformation for predictive and prescriptive modeling
Expand and optimize data and data pipeline architectures, improving data flow for cross functional teams
Take direction from lead data engineers and department leaders to advance the team’s data and analytics capabilities
Mentor junior data engineers and guide users across the organization to promote data literacy and a data driven culture in clinical and business operations
Consult with data and analytics experts to enhance system functionality and coordinate with data systems management to align with broader needs
Under leadership guidance, design, implement, and continuously expand data pipelines, perform ETL activities, and maintain optimal pipeline architecture
Assemble large, complex data sets that meet functional and non-functional requirements, while identifying opportunities for process improvements
Gather requirements and business process knowledge to transform data in ways that serve end users
Present data storage options including lakes, marts, and warehouses based on storage needs
Ensure the data architecture remains scalable and maintainable
Investigate data for potential ETL issues, notify end users, and propose effective solutions
Conduct preliminary testing of warehousing and database environments prior to data extraction
Extract data and migrate it to warehousing environments, other databases, or files
Test the system after data migration to verify integrity and performance
Provide ongoing maintenance and support, maintain documentation and code comments, and perform additional duties as assigned

Technologies

Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Databricks
Python
PySpark
SQL
SQL Server
Oracle
Microsoft Fabric
SSIS
Informatica
AWS
GCP
Epic Cogito

Qualifications

Bachelor's Degree
Minimum 2 years of related experience

Travel

Travel is rarely required for this role.

Compensation

Annual salary range: USD 82,932 to 114,031.

View full job page →

26d ago

Senior Data Engineer

Deloitte Arlington, VA

Arlington, VA

26d ago

$95k - $150k/yr Full time Senior Azure Azure Data Factory

Apply Now

Job Description

Senior Data Engineer at Deloitte in Arlington, VA onsite, focusing on designing and optimizing end-to-end data pipelines with Azure Databricks, ADF, and PySpark across development, UAT, and production environments.

Responsibilities

Maintain ongoing collaboration with Engagement Managers, project teams, and stakeholders across functional and technical groups, escalating issues to engagement management when needed.
Design, develop, and optimize ETL and ELT pipelines using Azure Data Factory and Databricks.
Develop and tune PySpark and Spark SQL notebooks to perform large-scale data transformations.
Architect comprehensive data solutions spanning development, UAT, and production environments with Unity Catalog.
Lead design discussions with client architects and other counterparts to align on technical direction.
Work with multiple teams to establish data contracts and agreed-upon schemas.
Lead the design and optimization of high-volume data pipelines.
Define and enforce data engineering standards, including naming conventions, partitioning strategies, cluster configurations, and Spark tuning.
Drive performance improvements through AQE tuning, dynamic clustering considerations, broadcast joins, and shuffle partition management.
Design Databricks cluster policies, autoscaling rules, and cost optimization strategies.
Investigate and resolve production incidents via root-cause analysis and implement durable fixes.
Mentor junior and mid-level engineers through code reviews and pair programming.
Assess new technologies and recommend adoption, such as Delta Live Tables, Auto Loader, serverless compute, or event hubs.

Requirements

Proficiency in Python, PySpark, Spark SQL, and SQL Server.
Experience with Azure services including Data Factory, Data Lake Storage Gen2, Key Vault, and Monitor.
Hands-on use of Databricks features such as Delta Lake, Unity Catalog, and Workflows.
Familiarity with Apache Airflow for workflow orchestration.
Version control and CI/CD using Git and Azure DevOps.
Deep knowledge of Spark internals, including DAG optimization, spill analysis, and skew handling.
Experience with advanced Delta Lake capabilities (time travel, deletion vectors, predictive I/O).
Unity Catalog governance skills, including row and column security, external locations, and system tables.
Infrastructure as Code experience with Terraform and Azure Resource Manager templates.
Bachelor's degree in Computer Science, Information Technology, Computer Engineering, or a related IT discipline, or equivalent experience.
Limited immigration sponsorship may be available.
Ability to travel approximately 10 percent, depending on client assignments.

Technologies

Python, PySpark, Spark SQL
SQL Server
Azure Data Factory
Azure Data Lake Storage Gen2
Key Vault
Azure Monitor
Databricks (Delta Lake, Unity Catalog, Workflows)
Delta Live Tables, Auto Loader, Serverless Compute
Azure Event Hubs
Git, Azure DevOps
Terraform, Azure ARM templates
Delta Lake advanced features

Team

The AI and Engineering group leverages advanced engineering practices to design, deploy, and operate integrated solutions across software, data, AI, networking, and hybrid cloud infrastructures. The team focuses on transforming mission-critical operations and enabling clients to modernize technology and data platforms while delivering client-centered outcomes.

Additional information

Accommodation for applicants with a need for assistance is available at the Deloitte accessibility page: https://www2.deloitte.com/us/en/pages/careers/articles/join-deloitte-assistance-for-disabled-applicants.html

View full job page →

27d ago

Senior Data Engineer (FTE)

Cittabase Solutions Houston, TX

Houston, TX

27d ago

$80k - $110k/yr Full time Senior Analytics Azure Data Factory

Apply Now

Job Description

Cittabase Solutions is expanding its engineering team with a Senior Data Engineer who will design and deploy modern data pipelines and cloud-based platform solutions for enterprise clients. The role emphasizes Microsoft SQL Server, Microsoft Fabric, and Databricks within an on-site Houston environment, contributing to data architecture, analytics readiness, and scalable data workflows.

Overview

We are seeking a talented Senior Data Engineer to join our team with experience across Microsoft SQL Server, Microsoft Fabric and Databricks. You’ll be part of a team delivering modern data pipelines and cloud platform solutions for enterprise clients.

Responsibilities

Strong track record designing data schemas, modeling data, and building pipelines
Experience delivering ETL/ELT pipelines to ingest data from multiple sources
Proficiency with Microsoft SQL Server and Azure tools: SSIS, SSAS, Microsoft Fabric, Azure Data Factory, Azure DevOps, Power BI, among others
Experience crafting cloud-based data architectures to support pipelines and analytics, ideally using Microsoft Fabric
Assess user requirements for reports, forms, queries, and data extraction
Create and deploy end-user data extraction and query tools aligned with business processes
Leverage Azure services for storage, processing, and analytics
Utilize Azure Data Lake for big data processing and analytics
Analyze data using various tools to provide insights for business stakeholders
Build Power Automation Scripts to enhance automation and efficiency in data processing
Maintain a deep understanding of data architecture and pipeline design

Requirements

5+ years in data warehouse and data pipeline development
Experience with SQL Server/Azure Fabric, SSIS, ADF and Power BI
Experience with Medallion Architecture
Familiarity with Azure services for data management
Ability to analyze complex datasets and present findings effectively
Certification in the Microsoft Fabric tech stack
Only US Citizens/Residents apply

Technologies

Microsoft SQL Server
Microsoft Fabric
Databricks
SSIS
SSAS
Azure Data Factory
Azure DevOps
Power BI
Azure Data Lake
Power Automation Scripts
Medallion Architecture

Job Details

Job Type: Full-time
Salary: USD 80,000 - 110,000 per year
Location: Houston, TX onsite
Ability to Commute: Houston, TX 77001 (Required)
Ability to Relocate: Relocate before starting work (Required)
Work Location: In person

View full job page →

1mo ago

Senior Data Engineer

Deloitte Boston, MA

Boston, MA

1mo ago Closed

$95k - $150k/yr Full time Senior Azure Azure Data Factory

This position is no longer accepting applications (closed Aug 4, 2026).

Job Description

Senior Data Engineer role with Deloitte in Boston, onsite, focused on delivering end-to-end data pipelines and solutions using Azure Data Factory, Databricks, and PySpark, along with client-facing collaboration and mentorship.

Responsibilities

Maintain regular collaboration with Engagement Managers, project teams, and cross-functional stakeholders; escalate issues to engagement management when needed.
Design, develop, and optimize ETL and ELT pipelines using Azure Data Factory and Databricks.
Develop and tune PySpark and Spark SQL notebooks for large-scale data transformations.
Architect end-to-end data solutions across development, UAT, and production environments with Unity Catalog governance.
Lead design discussions with client architects and counterpart teams.
Collaborate with multiple teams to establish data contracts and agreed schemas.
Direct the design and optimization of high-volume data pipelines.
Define and enforce data engineering standards, including naming conventions, partitioning strategies, cluster configurations, and Spark tuning.
Drive performance enhancements through adaptive query execution tuning, clustering strategies, broadcast joins, and shuffle partition management.
Design Databricks cluster policies, autoscaling configurations, and cost optimization strategies.
Conduct root cause analysis on production incidents and implement durable fixes.
Mentor junior and mid-level engineers via code reviews and pair programming.
Evaluate new technologies and recommend adoption, including Delta Live Tables, Auto Loader, Serverless Compute, DABs, and Event Hubs.

Requirements

Proficiency in Python, PySpark, Spark SQL, and SQL Server.
Experience with Azure components such as Data Factory, Data Lake Storage Gen2, Key Vault, and Monitor.
Experience with Databricks including Delta Lake, Unity Catalog, and Workflows.
Familiarity with Apache Airflow for workflow orchestration.
Git or Azure DevOps for version control and CI/CD.
Deep knowledge of Spark internals, including DAG optimization, spill analysis, and skew handling.
Delta Lake advanced features such as time travel, deletion vectors, and predictive I/O.
Unity Catalog governance including row/column security, external locations, and system tables.
Infrastructure as Code experience with Terraform and Azure ARM templates.
Bachelor's degree in Computer Science, Information Technology, Computer Engineering, or a related IT discipline, or equivalent experience.
Limited immigration sponsorship may be available.
Ability to travel approximately 10 percent, varying by client engagements.

Technologies

Python
PySpark
Spark SQL
SQL Server
Azure: Data Factory, Data Lake Storage Gen2, Key Vault, Monitor
Databricks: Delta Lake, Unity Catalog, Workflows
Delta Live Tables (DLT)
Auto Loader
Serverless Compute
DABs
Event Hubs
Apache Airflow
Git
Azure DevOps
Terraform
Azure ARM templates

Team

AI and Engineering leverages cutting-edge capabilities to build and operate integrated data, software, AI, network, and hybrid cloud solutions. The team focuses on transforming mission-critical operations and modernizing clients' technology and data platforms through tailored delivery models.

Additional Information

Accommodation information for applicants requiring assistance: https://www2.deloitte.com/us/en/pages/careers/articles/join-deloitte-assistance-for-disabled-applicants.html