Data Engineer 4

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions across full-stack development tools and technologies
• Provide technical influence to a team of developers, data analysts, and data scientists with deep experience in machine learning, distributed microservices, lakehouse architecture, and full-stack systems
• Use Python and Spark with open-source relational and NoSQL databases, and cloud data warehousing platforms such as Databricks and Snowflake
• Design, build, and support robust cloud-first data solutions in partnership with product managers and software engineers
• Independently design, build, and deliver cloud data solutions and applications with little or no support from supervisors or managers
• Architect and enforce common data engineering design patterns to improve code quality, maintainability, and reusability across platforms and pipelines
• Build data pipelines and platforms with a focus on scalability, resilience, and operational efficiency
• Implement data security standards, including encryption at rest/transit and fine-grained access control, to support compliance with data privacy regulations
• Serve as a data engineering ambassador by communicating technical concepts and data outcomes clearly to internal and external stakeholders
• Stay current with data trends, experiment with new technologies, participate in internal and external communities, and mentor other members of the data community

Requirements

• Bachelor’s Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• 4+ years of experience in application development (internship experience does not apply)
• 2+ years of experience in distributed data
• 2+ years of experience with SQL
• 2+ years of experience with one programming language: Python, Java, or Scala
• 2+ years of experience in data pipeline design and development
• 1+ year of experience in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

Preferred Qualifications

• 7+ years of application development experience with demonstrated proficiency in Python, SQL, Scala, or Java
• 4+ years designing, deploying, and operating data workloads in at least one public cloud environment (AWS, Microsoft Azure, or Google Cloud)
• 4+ years building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks
• 4+ years designing, implementing, and operating real-time or streaming data pipelines
• 2+ years working on data observability (e.g., Monte Carlo, Splunk) or data orchestration tools (e.g., Airflow, Dagster)
• 4+ years working with unstructured or semistructured data using NoSQL databases (e.g., MongoDB, Cassandra, DynamoDB)
• 4+ years designing and supporting data warehousing solutions (e.g., Snowflake, Redshift)
• 2+ years working in an Agile development environment
• 2+ years developing user-centric reusable data products

Technologies

• Python, Spark
• Databricks, Snowflake
• SQL, NoSQL, relational databases
• Encryption at rest/transit, fine-grained access control
• Machine learning, distributed microservices, lakehouse architecture

Additional Information

Immigration Authorization Notice: Capital One will not sponsor a new applicant for employment authorization, or offer any immigration related support for this position (including H1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN, E-3, O-1, and other work authorization requiring employer immigration support).

Accommodation: If you require an accommodation, contact Capital One Recruiting at 1-800-304-9102 or email [email protected].

Recruiting process questions: For technical support or questions about Capital One’s recruiting process, email [email protected].