Senior Data Engineer

Senior Data Platform Engineer role focused on building and modernizing enterprise cloud data capabilities on Azure.

Responsibilities

• Design, develop, and maintain scalable data pipelines, ingestion frameworks, transformation processes, and reusable data products using Azure Databricks, PySpark, SQL, and Delta Lake
• Implement Bronze, Silver, and Gold architecture patterns to support enterprise reporting, analytics, AI, and self-service data consumption
• Create reusable frameworks, utilities, and platform components to improve engineering productivity, quality, consistency, and deployment speed
• Deliver and support batch, near-real-time, and streaming integration solutions, including modernization of legacy warehouse and ETL workloads
• Act as technical lead for complex initiatives by producing solution designs, leading technical reviews, recommending tools and approaches, and guiding production implementation
• Partner with data architects and platform leaders to ensure solutions are scalable, secure, governed, cost-conscious, and operationally supportable
• Mentor engineers and promote standards for coding, testing, documentation, performance, and production readiness
• Implement data quality and validation capabilities, reconciliation, monitoring, metadata, and lineage; support RBAC and enterprise security controls
• Build and maintain CI/CD, automated testing, deployment, and release processes using Azure DevOps and Git-based practices
• Contribute to platform observability through alerting, operational dashboards, health metrics, performance tuning, and cost optimization
• Participate in production support, incident response, pager, and on-call rotations; troubleshoot issues, lead root cause analysis, and implement durable remediation
• Create and maintain operational runbooks, support procedures, technical documentation, and knowledge-sharing materials
• Collaborate with architecture, governance, security, analytics, application, and business teams to translate requirements into platform solutions
• Support technical discovery, estimation, roadmap planning, delivery execution, and evaluation of emerging cloud, data, and AI capabilities

Role Focus

• Approximately 70% development and 30% operations and support

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Data Science, or a related field
• Master’s degree preferred
• 10+ years of experience in data engineering, data warehousing, or data platform engineering
• 5+ years designing or implementing cloud-based data warehouse, data lake, or data platform solutions
• 3+ years hands-on experience with Azure Databricks in enterprise production environments
• Experience with ETL/ELT development, data modeling, large-scale integration, and cloud platform modernization
• Experience leading technical implementations, solution design, design reviews, and production releases
• Experience supporting production environments, incident response, root cause analysis, and operational support processes
• Experience in regulated environments such as financial services, banking, or audit-sensitive contexts preferred

Technologies

• Azure Databricks, PySpark, SQL, Delta Lake
• Bronze, Silver, Gold (Medallion Architecture patterns)
• ETL, ELT, Unity Catalog, Databricks Workflows, Delta Live Tables
• RBAC, Azure DevOps, Git-based practices
• Power BI, MicroStrategy, MDM

Skills & Competencies

• Advanced proficiency in SQL, Python, PySpark, and Spark performance optimization
• Strong experience with Azure Databricks, Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables
• Strong understanding of lakehouse concepts, Medallion Architecture, dimensional modeling, data warehousing, and analytics-oriented data structures
• Experience with Azure DevOps, Git, CI/CD, automated testing, and deployment practices; Databricks Asset Bundles preferred
• Experience implementing data quality, validation, reconciliation, monitoring, metadata, lineage, governance, and security controls
• Technical leadership, solution design, analytical thinking, troubleshooting, and problem-solving ability
• Ability to independently lead complex work across design through production support, balancing innovation with reliability, security, cost, and supportability
• Collaboration, mentoring, documentation, and communication across technical and business teams
• Preferred experience: Power BI, MicroStrategy, MDM, streaming architectures, infrastructure automation, or AI-assisted development
• Databricks Data Engineer certification (Associate or Professional) preferred

Location

• Austin, TX (hybrid remote)
• Work location: Austin, TX 78746

Compensation

• USD 75 - 85 per hour

Application Questions

• Are you local to Austin, TX?
• Are you comfortable to attend a face to face interview in Austin, TX?
• Do you have strong Python Development experience along with SQL & Databricks?
• Are you comfortable to do a 90 minutes Hackerrank coding test?
• Do you need sponsorship now or in future to work in the United States?