Capital One is hiring a Senior Lead Data Engineer in Plano, TX (onsite) to lead the design and development of an enterprise streaming data platform focused on low latency and fault-tolerant delivery. The role will also involve leading engineering efforts, shaping data pipeline best practices, and collaborating with agile teams including Cyber & Security.
Key Responsibilities
- Lead the design and build of an enterprise scalable streaming data platform that supports meaningful and timely insights with low latency and fault tolerance
- Develop next-generation distributed streaming data pipelines and analytics data stores using streaming frameworks such as Flink and Spark Streaming, with implementation in Java, Scala, or Python
- Lead a group of engineers delivering data pipelines using big data technologies including Spark, Flink, Kafka, Snowflake, AWS Big Data Services, and Redshift on medium to large scale datasets
- Influence best practices for data pipeline design and data architecture, including processing of structured and unstructured data
- Work in an agile environment with a focus on CI/CD and Application Resiliency Standards, partnering with Cyber & Security teams
- Collaborate across agile teams to design, develop, test, implement, and support technical solutions using full-stack development tools and technologies
- Lead developers with experience spanning machine learning, distributed microservices, and full stack systems
- Use programming languages such as Java, Scala, and Python alongside open source relational databases and NoSQL databases, and cloud-based data warehousing services including Redshift and Snowflake
- Stay current with technology trends through experimentation, learning, participation in internal and external technical communities, and mentoring engineering peers
- Partner with digital product managers to deliver robust cloud-based solutions that support financial empowerment for millions of Americans
- Perform unit tests and conduct code reviews to ensure solutions are rigorously designed, cleanly implemented, and tuned for performance
Required Qualifications
- Bachelor’s Degree
- At least 6 years of experience in application development (internship experience does not apply)
- At least 2 years of experience in big data technologies
- At least 1 year of experience with cloud computing (AWS, Microsoft Azure, or Google Cloud)
Additional/Preferred Qualifications
- Master’s Degree
- 9+ years of experience in application development including Python, SQL, Scala, or Java
- 4+ years of experience with a public cloud (AWS, Microsoft Azure, or Google Cloud)
- 5+ years experience with distributed data or computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
- 4+ years of experience working on real-time data and streaming applications
- 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
- 4+ years of data warehousing experience (Redshift or Snowflake)
- 4+ years of experience with UNIX/Linux, including basic commands and shell scripting
- 2+ years of experience with Agile engineering practices
- Experience leveraging interactive AI tooling to accelerate productivity, using capabilities beyond basic code completion
Technologies
- Flink, Spark Streaming, Java, Scala, Python
- Spark, Kafka
- Snowflake, AWS Big Data Services, Redshift
- Open Source RDBMS, NoSQL
- AWS, Microsoft Azure, Google Cloud
- CI/CD
Compensation and Location
Location: Plano, TX (onsite)
Salary: USD 209,000 - 238,500 per year for Sr. Lead Data Engineer
This role is also eligible to earn performance-based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI).
Benefits
Capital One offers a comprehensive, competitive, and inclusive set of health, financial, and other benefits that support total well-being.
Additional Employment and Recruiting Notes
- Capital One may sponsor a new qualified applicant for employment authorization for this position
- Expected application acceptance period: minimum of 5 business days
- No agencies please
- Capital One is an equal opportunity employer (EOE, including disability/vet) committed to non-discrimination in compliance with applicable federal, state, and local laws
- Capital One promotes a drug-free workplace
- Accommodation requests: contact Capital One Recruiting at 1-800-304-9102 or RecruitingAccommodation@capitalone.com
- Recruiting process technical support: Careers@capitalone.com
- Capital One does not provide, endorse, or guarantee third-party products, services, educational tools, or other information available through this site
EEO and Accessibility
Capital One is committed to non-discrimination in compliance with applicable laws, and provides an accommodation contact path for applicants who need assistance when applying.
