Principal Data Engineer

Lead identity and device data engineering across marketing measurement, activation, and analytics at petabyte scale.

• Define technical strategy for Identity & Device data products within Marketing and Consumer Data, covering pipeline architecture, platform patterns, and a modernization roadmap
• Drive end-to-end delivery for complex, high-impact initiatives across Identity360, device/consumer identifier datasets, and dependent Marketing data products
• Set technical approach and quality bar for other engineers, including sequencing and implementation standards
• Act as principal SME and escalation point for identity foundational data, including:
  • Data modeling
  • Batch and near-real-time processing
  • Performance at scale
  • Production reliability
• Support operational readiness through participation in an on-call rotation and incident leadership for critical pipelines
• Establish engineering standards for the Identity & Device team and Marketing Data Engineering, including:
  • Code quality
  • CI/CD
  • Observability
  • Data governance: GDPR, RTBF, CCPA
  • Reusable platform patterns across Databricks, AWS, and orchestration tooling
• Mentor and develop engineers via design reviews, pairing, and coaching, building depth in big-data engineering, identity domain knowledge, and operational excellence
• Collaborate cross-functionally with Product, QA, Operations, Privacy/Legal, and peer Marketing data squads (Data Collection, Activation, Acquisition) to align requirements and resolve cross-domain dependencies
• Represent technical decisions tied to Bangalore execution to relevant stakeholders

Required Qualifications

• 10+ years of related big data engineering experience modeling and building large data pipelines
• Python, Scala, or Java experience
• AWS fundamentals preferred (S3, Lambda, IAM)
• 5+ years hands-on performance tuning at scale using distributed systems such as Hadoop or Databricks with tools/libraries including Spark, Presto, and Hive for petabyte-level datasets
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, BigQuery, or Databricks)
• Strong knowledge of data modeling and data warehousing methodologies and practices
• Operational experience in a multi-system/services production environment as a senior data engineer and point-of-contact/SME
• Problem solving with strong attention to detail plus excellent analytical and communication skills
• Experience with data integration and orchestration toolsets (e.g., Airflow) and CI/CD
• Demonstrated ability with cloud infrastructure technologies, including Terraform
• Strong understanding of slowly changing dimensions (SCD tables) and how to build scalable, performant datasets
• Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field, and/or equivalent work experience

Technologies

• Python, Scala, Java
• AWS, S3, Lambda, IAM
• Hadoop, Databricks, Spark, Presto, Hive
• Snowflake, Redshift, BigQuery
• Airflow, CI/CD, Terraform
• GDPR, RTBF, CCPA

Location and Compensation

• Location: New York, NY (onsite)
• Base salary range: USD 187,900 - 252,000 per year
• Compensation note: A bonus and/or long-term incentive units may be provided as part of the compensation package

Benefits

• Full range of medical, financial, and/or other benefits

Pay Transparency Disclaimer

• The hiring range for this position in SFO, CA is $187,900 - $252,000 per year and New York is $179,700 - $241,000 per year
• PAY TRANSPARENCY
• This position is with Disney Entertainment & Sports LLC, part of Disney Entertainment and ESPN Product & Technology