Structured Streaming Engineer Jobs

1 job found

Today

Sr Databricks Data Engineer

Deloitte Pittsburgh, PA

Pittsburgh, PA

Today

$116k - $229k/yr Full time Apache Airflow Automation Big Data

Apply Now

Job Description

Join Deloitte's AI & Data practice on site in Pittsburgh to help modernize data platforms, enable analytics and AI, and drive measurable business outcomes. This role offers a competitive compensation range of USD 116,200 to 229,100 per year, plus a discretionary annual incentive and a benefits package aligned with the Core Talent Model. You will work in a collaborative, data-driven environment that prizes technical excellence and leadership, with opportunities to shape cloud based data engineering solutions on Databricks.

Benefits

Discretionary annual incentive program
Benefits package aligned with Core Talent Model
Competitive salary range: USD 116,200 – 229,100 per year

Responsibilities

Champion best practices by documenting and promoting superior data architecture, integration, and modeling approaches
Own data pipelines and architectures to support large scale enterprise data needs
Lead efforts to improve data quality, operational efficiency, and process scalability
Evaluate, pilot, and integrate new big data and analytics technologies; lead and develop teams of data engineers and architects
Design and implement governance, security, and compliance strategies for modern cloud data ecosystems
Communicate technical concepts and business value to executives, business leads, and technology teams
Oversee CI/CD practices with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell to streamline deployments and operations

Requirements

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience focusing on Databricks on AWS, Azure, or GCP
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources
Ability to travel up to 50% on average based on the work and client needs
Limited immigration sponsorship may be available

Preferred qualifications

Master's degree in Computer Science, Engineering, or a related field
Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services
Experience tuning and optimizing performance in Databricks and Apache Spark environments
Experience with Databricks Lakeflow
Experience with artificial intelligence and machine learning solutions

Technologies

Databricks
AWS, Microsoft Azure, Google Cloud Platform (GCP)
Apache Spark, Delta Lake, Unity Catalog
Delta Live Tables, Autoloader, Structured Streaming
Databricks Workflows, Apache Airflow
Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
PySpark, Databricks Lakeflow

View full job page →

Get Structured Streaming Engineer Jobs alerts

New Structured Streaming Engineer Jobs, daily.