Lead Data Engineer
Job Description
Lead research and development on advanced analytic data solutions, including GenAI-powered pipelines, in an in-office environment.
Responsibilities
- Plan and lead R&D for advanced analytic data solutions leveraging GenAI
- Partner with business and technology leaders to gather scope, requirements, business needs, and data across the company
- Design and deliver data pipelines needed for critical decision science work
- Collaborate with Decision Science Products, Decision Science, and client teams on priority projects
- Plan, estimate, design, develop, test, support production rollout, and sustain solutions
- Consult and collaborate with project team members during delivery
- Lead design reviews
- Contribute hands-on development work
- Communicate technical concepts and progress with colleagues and leaders
Requirements
- 7+ years in a data engineering development capacity using multiple environments (Dev, QA, Prod, etc.) and DevOps procedures for code deployment/promotion
- Experience translating project scope and high-level requirements into technical data engineering tasks
- Experience defining solutions for sophisticated data engineering problems supporting advanced analytics
- Experience working with a variety of GenAI models, tools, and concepts
- Understanding of Knowledge Graphs, Data Mesh, and other data sharing platforms
- 3+ years building and using relational databases (preferably Snowflake or PostgreSQL)
- 3+ years leading and deploying code using source control tools such as GitLab/GitHub
- 2+ years using job scheduling software such as Apache Airflow, Amazon MWAA, GitLab Runners, or UC4
- Experience collaborating across multiple project teams in a fast-paced environment
- Experience defining and estimating level-of-effort for data engineering work
- Experience with project and sprint planning
- Ability to communicate technical concepts and solutions to non-technical team members
- Experience designing and building data structures aligned to requirements
- Multiple years in ELT/ETL pipeline development and maintenance
- Multiple years of demonstrated expertise using SQL and Python
- Experience with containerization technologies such as Docker or Kubernetes
- Proven ability to work collaboratively in a multi-team environment
Technologies
- GenAI
- Knowledge Graphs
- Data Mesh
- Snowflake, Snowpark
- Data Exchange, Data Marketplace, Snowpipe
- GitLab, GitHub
- Apache Airflow, Amazon MWAA, GitLab Runners, UC4
- ELT/ETL
- SQL, Python
- Docker, Kubernetes
- AWS EMR, EC2, S3
Preferred Qualifications
- Experience leading development of GenAI-based systems, including model selection, pipeline orchestration, and deployment strategies
- Experience defining GenAI architectures
- Knowledgeable with Disney Parks attendance, reservations, and/or products
- Experience with cloud technologies, preferably AWS EMR, EC2, and S3
- Experience with advanced Snowflake offerings such as Snowpark, Data Exchange, Data Marketplace, and Snowpipe
Education
- Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable
Location and Compensation
- Location: Lake Buena Vista, FL (onsite)
- Salary: USD 135,200 - 181,200 per year
- Additional information: Base pay offered will consider internal equity and may vary by geographic region, job-related knowledge, skills, and experience
- Incentives: A bonus and/or long-term incentive units may be provided, plus full range of medical, financial, and/or other benefits depending on level and position
Note: This is not a remote role. The team works in office.