Senior Data Engineer
Senior
Azure Data Factory
Cloud
Cloud Data Engineering
Data Architecture
Data Build Tool
Data Engineer
Data Engineering
Data Factory
Data Factory Azure
Data Integration
Data Management
Data Modeling
Data Operations
Data Pipeline
Data Pipelines
Data Platform
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Dataform
Dataops
Microsoft Azure
SQL
Sqlmesh
Job Description
Senior Data Engineer will help bring a production data platform to steady operations while delivering scoped production transformations and platform improvements.
Responsibilities
- Own production operations for a platform supporting production reporting delivery and billing
- Manage environment setup and management, including refresh and orchestration operations
- Run monitoring and alerting and handle incident response
- Maintain security-role hierarchies and grants; support performance and ongoing maintenance
- Build and maintain pipeline automation and environment promotion so deployments are routine and reliable
- Create staging and gold dbt models with tests aligned to defined business semantics and existing conventions
- Identify and flag requests that would conflict with established definitions
- Review teammates’ work, including AI-generated code, for correctness and adherence to standards
- Analyze and absorb legacy and vendor source systems, including working with many undocumented schemas as an ongoing requirement
- Validate new platform outputs against legacy system reporting and billing figures prior to cutover
- Deliver well-scoped production data work (for example dbt modeling and transformation) within existing patterns to support parallel architecture and design work
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or related field
- 7+ years of experience designing, developing, and maintaining large-scale data pipelines and data warehouse solutions
- SQL at working depth, including joins, window functions, grain and aggregation tradeoffs, NULL semantics, and query reasoning
- Production Python experience, including writing and maintaining Python code that ran in production
- Production pipeline ownership, including scheduled pipelines and diagnosing and fixing failures
- Transformation framework experience: strong working depth in at least one modern framework (dbt preferred; SQLMesh, Dataform, Coalesce, Databricks declarative pipelines, or comparable in-house framework also qualify)
- Production operations experience, including deployments, environment management, on-call (or equivalent), and incident response
- Dimensional modeling knowledge: grain, facts and dimensions, conformance, and slowly-changing history
- Git-based workflow: branching, pull requests, code review, and CI as standard practice
- Clear communication in writing, in conversation, and in presentations, including explaining technical findings to non-technical stakeholders
- Independent investigation of unfamiliar problems prior to escalation
- AI tooling used as a regular part of engineering work for research, design, and verification; built tooling for their own workflow; verifies AI output before relying on it
Technologies
- SQL, Python, dbt
- SQLMesh, Dataform, Coalesce, Databricks declarative pipelines
- Git
- Azure (Data Factory), Azure (ADLS Gen2), Azure (Key Vault)
- AWS, GCP
- Snowflake
- Terraform
Preferred Qualifications
- Multi-tenant or customer-facing data experience (isolation, audit, lineage)
- Azure experience (Data Factory, ADLS Gen2, Key Vault); AWS or GCP experience transfers
- Snowflake
- Terraform or equivalent infrastructure-as-code
- Legacy and vendor-system reverse-engineering
- Logistics, supply-chain, or warehousing domain experience
- Familiarity with BI tools
Compensation and Location
- Location: City of Industry, CA (onsite)
- Salary: USD 140,000 - 180,000 per year
Benefits
- Medical, dental, and vision insurance
- Basic and voluntary life and voluntary ancillary coverages for accident, critical illness and hospital indemnity
- Paid sick leave
- Bereavement pay
- Holiday and vacation pay
- 401k plan eligibility on the first day of the third month following hire date
- 401(k)
- 401(k) matching
- Employee assistance program
- Flexible spending account
- Health insurance
- Health savings account
- Life insurance
- Paid time off
- Professional development assistance
- Referral program