Autoloader Engineer Jobs

1 job found

23d ago

Sr Databricks Data Engineer

Deloitte Sacramento, CA

Sacramento, CA

23d ago

$138k - $194k/yr Full time Autoloader Big Data Bigdata

Apply Now

Job Description

Senior Databricks Data Engineer based in Sacramento, CA, on site, focusing on designing, building, and optimizing cloud data solutions to modernize platforms and enable analytics and AI at enterprise scale.

Responsibilities

Establish and promote best practices for data architecture, integration, and modeling across projects.
Own the design, development, and maintenance of robust data pipelines and architectures supporting large-scale enterprise data needs.
Lead initiatives to improve data quality, operational efficiency, and scalable processes.
Evaluate, pilot, and integrate new big data and analytics technologies; lead and develop teams of data engineers and architects.
Consult on and implement governance, security, and compliance strategies for modern cloud data ecosystems.
Translate technical concepts and business value for executives, business leads, and technology teams.
Oversee CI/CD practices and automation with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.
Provide clear guidance to colleagues across project teams.

Requirements

Ability to work independently and collaboratively within a team.
Effective written and verbal communication skills.
Meticulous attention to detail and quality of work product.
Ability to build and sustain professional relationships.
Proven ability to lead projects or workstreams.
Strong capability to manage and prioritize multiple tasks in a fast-paced, dynamic environment.
Strong interpersonal skills and professional demeanor.
Ability to meet deadlines.

Technologies

Databricks
AWS
Azure
GCP
Delta Lake
Apache Spark
PySpark
Unity Catalog
Delta Live Tables
Autoloader
Structured Streaming
Databricks Workflows
Apache Airflow
Azure DevOps
AWS Code Pipeline
Jenkins
TFS
PowerShell
Databricks Lakeflow

Qualifications Required

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience focusing on Databricks on AWS, Azure, or GCP
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data engineering pipelines, code, and compute resources
Ability to travel 50% on average, depending on client engagements
Limited immigration sponsorship may be available

Preferred

Master's degree in Computer Science, Engineering, or a related field
Experience across one or more cloud ecosystems (AWS, Azure, GCP) and related big data services
Experience tuning and optimizing performance in Databricks and Apache Spark environments
Experience with Databricks Lakeflow
Experience with artificial intelligence and machine learning solutions

View full job page →

Get Autoloader Engineer Jobs alerts

New Autoloader Engineer Jobs, daily.