Sr Data Engineer II

Work on AI-ready data platforms that convert large-scale payment data into real-time intelligence and production-ready analytics and Generative AI capabilities.

Responsibilities

• Design, build, and evolve highly scalable, AI-ready data platforms supporting critical payment capabilities.
• Enable real-time intelligence, advanced analytics, machine learning, and Generative AI use cases using enterprise-grade data engineering.
• Operate across data engineering, distributed systems, and cloud-native technologies to transform large-scale payment data into intelligent, secure, and reliable products and experiences.
• Develop and implement data pipelines, data models, APIs, and event-driven architectures for downstream AI and analytics.
• Deliver production-grade engineering outcomes by solving complex technical problems across distributed and cloud environments.

Requirements

• Experience designing, implementing, and operating large-scale distributed data platforms and systems.
• Strong experience with NoSQL technologies such as Cassandra, Elasticsearch, Couchbase, or Redis.
• Experience with large-scale distributed data processing, including Apache Spark.
• Strong programming experience with Python, Java, Scala, or similar languages.
• Experience designing data pipelines, data models, APIs, and event-driven architectures.
• Solid understanding of distributed system concepts including scalability, reliability, resiliency, security, and performance.
• Demonstrated ability to solve complex engineering problems and deliver production-grade solutions.

Technologies

• Cassandra, Elasticsearch, Couchbase, Redis
• Apache Spark
• Python, Java, Scala
• Apache Kafka, Docker, Kubernetes, OpenShift
• Generative AI, LLMs, RAG, embeddings
• Vector databases/search, AI agent architectures
• MLOps/LLMOps
• Microservices

Preferred Qualifications

• Experience building AI/ML-ready data platforms for machine learning, Generative AI, or intelligent application use cases.
• Familiarity with Generative AI, LLMs, RAG, embeddings, vector databases/search, and AI agent architectures.
• Experience integrating AI/ML models and services into production environments.
• Familiarity with MLOps/LLMOps, including deployment, evaluation, monitoring, observability, governance, and lifecycle management.
• Experience with distributed messaging and streaming platforms such as Apache Kafka.
• Experience with real-time distributed processing using technologies such as Spark, Kafka, Cassandra, and Elasticsearch.
• Experience developing microservices and cloud-native applications.
• Experience with Docker, Kubernetes, OpenShift, or similar container platforms.
• Experience with CI/CD, DevOps, infrastructure automation, and production observability.
• Understanding of Responsible AI, data privacy, security, governance, and model risk considerations.
• Experience architecting large-scale systems with an emphasis on availability, scalability, performance, resiliency, and cost efficiency.
• Ability to stay current with emerging AI and data engineering technologies and identify opportunities to apply them to real-world business problems.
• Strong communication, collaboration, and technical leadership skills, including mentoring engineers and influencing technical direction.

Benefits

• Competitive base salaries
• Bonus incentives
• 6% Company Match on retirement savings plan
• Free financial coaching and financial well-being support
• Comprehensive medical, dental, vision, life insurance, and disability benefits
• Flexible working model with hybrid, onsite or virtual arrangements depending on role and business need
• 20+ weeks paid parental leave for all parents, regardless of gender, offered for pregnancy, adoption or surrogacy
• Free access to global on-site wellness centers staffed with nurses and doctors (depending on location)
• Free and confidential counseling support through the Healthy Minds program
• Career development and training opportunities

VISA SPONSORSHIP

• Depending on factors such as business unit requirements, the nature of the position, cost and applicable laws, American Express may provide visa sponsorship for certain positions.