Lead Data Engineer, Consumer

What you’ll deliver

• Lead delivery of source-to-Silver data products.
• Design and oversee ingestion, transformation, standardization, orchestration, and publication of data from operational, SaaS, file, streaming, API, and partner sources.
• Ensure Silver-layer data products are validated, standardized, documented, reusable, performant, secure, and ready for Gold-layer analytics and AI.
• Establish repeatable processing patterns for batch, incremental, change-data-capture, and event-driven workflows.
• Implement engineering foundations using reusable frameworks, templates, libraries, and pipeline patterns to reduce repeated effort across products and domains.

How the team and delivery will work

• Lead and develop a distributed/offshore team of Data Engineers delivering end-to-end from source ingestion to curated Silver-layer data products.
• Translate product roadmaps and requirements into engineering plans, milestones, estimates, dependencies, and delivery commitments.
• Apply modern engineering practices including source control, peer review, automated testing, CI/CD, observability, release management, and incident remediation.
• Define and maintain standards for naming, partitioning, schema evolution, error handling, replay/recovery, performance, cost management, and documentation.
• Partner with the Data Platform team to use approved workspace, compute, storage, security, and deployment patterns.
• Identify technical debt and lead pragmatic improvements to increase delivery speed, reliability, and maintainability.

Collaboration across data modeling and product

• Work with Data Modelers to implement canonical entities, conformed dimensions, data contracts, source-to-target mappings, and enterprise modeling standards.
• Partner with Product Managers and domain leaders to clarify intended outcomes, source-system realities, priority use cases, and acceptance criteria.
• Coordinate dependencies with source-system owners, platform teams, analytics teams, and external partners.

Trusted, governed data outcomes

• Establish profiling, reconciliation, quality testing, freshness monitoring, lineage, and alerting for every delivered data product.
• Ensure source-to-Silver traceability including authoritative source identification, transformation logic, ownership, metadata, and data-quality expectations.
• Implement access controls and sensitive-data practices, including classification, masking, retention, and regional/data-residency requirements.
• Drive resolution of data defects, schema changes, pipeline failures, and quality issues with clear ownership and service-level expectations.

Requirements

• 8+ years of data engineering, software engineering, or data-platform experience, including 2+ years leading engineers, technical workstreams, or large-scale delivery.
• Demonstrated experience designing and delivering enterprise-scale ingestion and transformation pipelines.
• Strong hands-on expertise in SQL, Python, Spark/PySpark, and modern ELT/ETL patterns.
• Experience with Databricks, Delta Lake, or a comparable lakehouse platform, plus cloud storage, orchestration, and CI/CD.
• Strong understanding of Bronze/Silver/Gold (or equivalent layered data-platform patterns).
• Experience implementing data quality, observability, lineage, metadata, and production support practices.
• Proven ability to partner with architects, data modelers, product leaders, and domain stakeholders.
• Experience leading distributed teams and creating effective delivery practices across time zones.
• Strong communication skills, including explaining technical choices, risks, and tradeoffs to non-technical stakeholders.

Preferred qualifications

• Experience with Unity Catalog or comparable governance, catalog, and access-control capabilities.
• Experience with data contracts, schema evolution, change data capture, APIs, event streaming, and large-volume data processing.
• Experience supporting multiple data domains, global data products, or regional data-residency requirements.
• Experience with infrastructure-as-code, DataOps, and automated environment provisioning.
• Experience in a regulated, high-scale, or operationally sensitive environment.

Technologies

• SQL, Python, Spark, PySpark, ELT, ETL
• Databricks, Delta Lake, CI/CD
• Bronze/Silver/Gold, change-data-capture

Additional information

• Candidates may be required to attend an in-person interview with a Versant Media employee at one of our locations prior to a hiring decision.
• If you are a qualified individual with a disability or a disabled veteran and require support during the application and/or recruitment process, you may request a reasonable accommodation by emailing [email protected].
• Versant Media is committed to fair and equitable compensation practices.
• Versant Media is not accepting unsolicited assistance from search firms for this employment opportunity.