Lead Data Engineer
Job Description
Join Prove’s Data Science team to design and run high-throughput data pipelines that support real-time and batch inference for verified digital trust.
- Architect, build, and maintain resilient data pipelines for real-time and batch inference workflows
- Move data from raw intelligence into verifiable scientific statements that support identity verification
- Translate domain logic across multidisciplinary teams into efficient, maintainable, modular DAGs and execution workflows
- Implement data governance practices including metadata management and semantic data ontologies
- Partner cross-functionally with data science, informatics, and product engineering for high-integrity data extracts
- Continuously evaluate and benchmark data engineering tools, frameworks, and storage paradigms
- Optimize distributed processing pipelines for high availability, computational efficiency, and cost governance
- Drive engineering quality through code reviews, architecture discussions, automated testing, and CI/CD
- Model team values including humility, passion, inclusion, and leadership
Requirements
- 7+ years of professional experience in data engineering or building complex, data-driven software products
- Deep understanding of data systems, distributed compute engines, data modeling, and modern pipeline architecture
- Proven collaboration across technical and scientific functions, translating cross-disciplinary logic into working data systems
- Strong intellectual curiosity and proactive experimentation with new technologies
- Hands-on orchestration experience with modern tools such as Airflow, Dagster, or Prefect
- Experience with large-scale distributed frameworks
- Strong motivation to learn Prove’s products and markets through in-house and external training
Technologies
- Airflow
- Dagster
- Prefect
- Linux/Unix
- Shell scripting
- CI/CD
Preferred Qualifications
- Advanced degree (Master’s or Ph.D.) in Computer Science, Mathematics, Statistics, Physics, or a related quantitative field
- Experience architecting solutions for identity resolution, graph data structures, or fraud prevention systems
- Terminal-native fluency in Linux/Unix, command-line utilities, and shell scripting
- Experience in high-growth or pre-IPO technology companies
Location and Reporting
- Location: New York, NY (hybrid)
- Department: Data Science
- Reports to: Director of Informatics
- FLSA status: Exempt
Compensation
- Base salary range: USD 175,000 - 200,000 per year
- Metro 2 anticipated pay range: $175,000 - $200,000 plus company bonus
Prove defines: Metro 2 includes the NYC metro area, Seattle metro area, Los Angeles metro area, and the Miami metro area. Metro 3 covers all other domestic United States cities excluding the San Francisco Bay Area.
Benefits
- Competitive salaries plus Bonus Plan (for eligible roles) and Equity Plan
- Modern Health for financial, mental, and physical wellness
- 401(k) Retirement Plan & Match (US Offices) and Local Country Pension (International Offices)
- Unlimited Vacation and flexible hours
- Comprehensive medical benefits for you and your family
- Emotional & Physical Wellness: EAP & Prove Well-Being Reimbursement
- Bottomless snacks & beverages for certain office locations
- Daily GrubHub stipend for lunch if coming into the office (US Offices)
- A collaborative place to connect with other talented Provers
Similar Jobs
J