Data Engineer- Azure Databricks- Financial Domain Exp Must
Backend Developer
Application Security
Azure
Azure Data Engineer
Azure Databricks
Azure Functions
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Engineer
Cloud Infrastructure
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Cloud Technology
Data
Data Analysis
Data Analytics
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Databricks Pyspark
Databricks Spark
DevOps
DevSecOps
Distributed Systems
Engineering
ETL
GraphQL
Informatica
Information Technology (IT)
Infrastructure As Code
Microsoft Azure
Programming
Programming Language
Programming Languages
Security Automation
Security Devops
Snowflake
Software Architecture
Software Development
Software Engineering
Spark
SQL
Job Description
Data Engineering opportunity in Eden Prairie, MN focused on Azure Databricks and financial-domain delivery.
Responsibilities
- Develop services, controls, and reusable patterns that help the team deliver value safely, quickly, and sustainably in public cloud environments
- Build highly scalable Azure cloud applications using microservices and Azure Functions, including public cloud enabling patterns with security considerations
- Contribute to high-performance, collaborative technical work to support high-quality delivery
- Review new product capabilities and break implementation into functional changes suitable for verification
- Gather and analyze data to support informed decision-making and produce detailed, realistic estimates
- Work effectively with business stakeholders and third-party technical organizations
- Design and develop ETL/ELT solutions on Azure Databricks, LakeBase, and Spark
- Design, build, optimize, and manage large-scale data pipelines for analytics and machine learning feature enablement
- Write advanced and complex SQL, including performance tuning and optimization
- Build AI-based solutions to address business needs, process automation, and improved operational efficiency
- Improve data reliability, data integrity, system efficiency, and data quality
- Participate in the architectural evolution of data engineering patterns, frameworks, systems, and platforms, including best practices and standards for managing (cut off in source)
- Mentor other data engineers by providing technical direction and teaching how to leverage cloud data platforms
Requirements
- 7+ years experience in Python and Scala
- 5+ years experience in Apache Spark (including PySpark and Spark SQL)
- 3+ years experience with DevOps automation using Terraform
- 3+ years experience in API design and lifecycle management (including GraphQL and REST)
- 3+ years building and deploying cloud solutions using Azure Databricks with UC, Snowflake, Azure Functions, and Service Bus
- 5+ years SQL experience, including complex data schema design and query performance optimization
- Experience building LLM integrations for workflow automation and business needs
- Experience with CI/CD and tools such as GitHub Actions, GIT, Artifactory, and Sonar
Technology Stack
- Python, Scala
- Apache Spark, PySpark, Spark SQL
- Azure Databricks, UC
- Snowflake
- Azure Functions, Service Bus
- Terraform
- SQL
- ETL/ELT, LakeBase
- AI/ML
- GraphQL, REST
- LLM integrations
- CI/CD, GitHub Actions, GIT, Artifactory, Sonar
- Microservices
Location and Schedule
- Work location: In person
- City/State: Eden Prairie, MN
- Work type: Onsite
- Expected hours: 40.0 per week
Compensation
- Rate: USD 70 - 80 per hourly