Data Engineer (Contract to Perm)
Big Data
Bigdata
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
ETL
Informatica
Integration
Reporting and Analytics
Software Development
Spark
SQL
Job Description
Locality is hiring a Senior Data Engineer (Contract to Perm) in New York, NY (onsite) to take primary internal ownership of the Audience Engine. This role combines hands-on Databricks pipeline leadership with work across identity resolution, partner integrations, and applied AI enablement. You will help reduce dependency on third-party vendors while maintaining reliability, data integrity, and clear governance across ingestion, activation, and reporting workflows.
What you’ll own
- Establish and own the technical management of the Audience Engine, including transitioning knowledge from external partners to a scalable, internally managed platform
- Architect, own, and continuously optimize Databricks Bronze / Silver / Gold pipelines for large-scale audience ingestion, enrichment, and activation workflows
- Maintain platform reliability by reducing reliance on third-party vendors while preserving uptime and data integrity across pipelines and integrations
- Monitor and troubleshoot ingestion failures, schema drift, and data quality issues across batch and near real time pipelines
- Ensure accurate and timely audience publication by maintaining robust downstream delivery processes and SLAs across activation platforms and partners
- Own and maintain integrations with external data and activation partners (including Experian, FreeWheel, and DSPs), ensuring seamless data exchange and operational reliability
Identity, AI enablement, and downstream impact
- Lead identity resolution across datasets including HHID, DSID, MAID, IP, and campaign logs, improving matching logic to strengthen the Household Identity Graph
- Support and operationalize applied AI/ML workflows in Databricks, including integration with Akkio, and translate business needs into production-ready data features and datasets
- Design scalable data models and query-ready datasets that power BI reporting, audience insights, and predictive modeling, aligned to downstream analytics requirements
Collaboration and governance
- Partner cross functionally with Product, Analytics, and Ad Operations to onboard new data sources and activation endpoints that enable new use cases and revenue opportunities
- Enforce data governance, lineage, and documentation standards while ensuring compliance with privacy, consent, and data usage regulations across all audience and identity data systems
Skills and experience
- 4+ years of experience as a Data Engineer or Data Platform Engineer, ideally in advertising
- Strong hands-on experience with Databricks, Spark, and modern cloud data platforms
- Proven experience designing and maintaining multi-layered architectures (Bronze / Silver / Gold)
- Deep expertise in SQL and proficiency in Python (or another scripting language)
- Experience working with identity graphs, audience datasets, and complex join logic across large datasets
- Hands-on experience building automated pipelines and third-party data integrations
- Familiarity with adtech or martech audience activation platforms, including exposure to identity providers such as Experian
- Practical experience supporting AI/ML workflows or AI-enabled analytics, including data quality, schema design, and performance optimization in cloud-scale environments
Tools you’ll use
- Databricks, Spark, SQL, Python, Akkio, BI
Contract to perm compensation and benefits
- Competitive base salary in the range of $60-$80/hr (dependent on years of experience, level of education, and applicable skill set)
- Excellent benefits package including health care benefits, retirement benefits, and paid days off (including but not limited to)
Similar Jobs
L