Senior Data Engineer

Build and scale a modern data platform that powers analytics, data science, and data products across Versant brands in a hybrid role based in North Hollywood.

Responsibilities

• Design and implement lakehouse architecture with Delta Lake, including medallion pipeline patterns (Bronze/Silver/Gold), schema enforcement, and time travel
• Build and operate batch and real-time ingestion pipelines using Databricks Auto Loader, Structured Streaming, and Change Data Capture (CDC) patterns
• Implement data governance and security with Unity Catalog and RBAC, following compliance-driven practices for sensitive environments
• Optimize performance and manage costs using FinOps strategies, including cluster sizing, workload tagging, Spark tuning, and Photon acceleration
• Create and maintain CI/CD pipelines and orchestration workflows using Databricks Workflows, Delta Live Tables, and tools such as Airflow
• Partner with Data Science teams on ML workflows using MLflow, feature store integration, and model lifecycle management
• Ensure data quality, observability, and lineage across media-focused datasets such as streaming logs, ad impressions, and audience metrics
• Provide technical mentorship through code reviews, pairing, and ongoing knowledge sharing

Requirements

• Bachelor’s degree in Computer Science, Data Engineering, or equivalent practical experience
• 5+ years building production-grade data pipelines in cloud environments using Spark-based platforms (e.g., Databricks, EMR, Dataproc, open-source Spark)
• Expertise in PySpark and SQL, with experience operating scaled pipelines in production
• Hands-on experience building batch or streaming production ingestion pipelines using distributed processing frameworks (e.g., Spark, Flink) and query engines such as Presto
• Proficiency with orchestration tools such as Apache Airflow or Dagster, including hands-on CI/CD, monitoring, alerting, and production data quality
• Experience with modern data architectures, including event-driven and distributed systems
• Proficiency with Git and collaborative development workflows
• Hands-on experience building and operating batch and real-time pipelines using Spark-based batch and streaming patterns (e.g., Structured Streaming, CDC), including experience on Databricks or comparable platforms
• Solid understanding of infrastructure, networking, and data security fundamentals

Technologies

• Delta Lake, Delta Live Tables, Databricks Auto Loader, Structured Streaming, Change Data Capture (CDC)
• Unity Catalog, RBAC
• Databricks Workflows, Airflow, PySpark, SQL, Spark, Flink, Presto, Dagster
• FinOps, Photon acceleration
• MLflow, feature store integration, CI/CD, Git
• Lakehouse, medallion pipeline patterns (Bronze/Silver/Gold), schema enforcement, time travel
• Spark tuning, cluster sizing, workload tagging

Benefits

• Medical, dental and vision insurance
• 401(k)
• Paid leave
• Tuition reimbursement
• Company sponsored benefits and a variety of other benefits and perks

Preferred Qualifications

• Experience building Lakehouse platforms and medallion pipelines in Databricks
• Familiarity with Unity Catalog, data governance, and compliance frameworks (e.g., PCI)
• Hands-on experience with CI/CD pipelines, orchestration tools, and infrastructure-as-code
• Experience with Lakeflow Spark Declarative Pipelines (SDP), MLflow, feature stores, and MLOps practices
• Background in media and entertainment data (e.g., video metadata, ad tech, audience analytics)
• Experience building data platforms within the media industry, including audience analytics
• Experience working with large-scale analytical datasets such as event logs, clickstream data, and audience metrics
• Comfort using AI-assisted development tools (e.g., ChatGPT)
• Databricks or cloud certifications (e.g., Databricks Certified Data Engineer)

Additional Information

• Location: North Hollywood, CA (hybrid). Generally in the office a minimum of three days per week from the LA office.
• Salary: $140,000 - $160,000 per year
• Candidates may be required to attend an in-person interview with a VERSANT employee in New York as part of the selection process.
• Equal employment opportunity policy applies without regard to protected characteristics.
• Reasonable accommodation: email [email protected] for support during the application and recruitment process.
• Fair pay transparency statement applies; actual compensation may vary based on skills, qualifications, experience, and location.
• For LA County and City Residents Only: qualified applicants with criminal histories may be considered consistent with applicable legal requirements.
• VERSANT Media is not accepting unsolicited assistance from search firms; resumes submitted without a valid written Statement of Work may be treated as VERSANT’s sole property and no fee will be paid.