Data Analytics Engineer

EXL Service is seeking an experienced Data Analytics Engineer to design, build, and optimize scalable data pipelines and analytics infrastructure that support financial products and decision-making. The role blends ETL/ELT engineering with cloud-native workflow orchestration and deep SQL and PySpark expertise within a regulated environment.

This position is based in San Francisco, CA (onsite) and offers a salary range of USD 140,000 - 155,000 per year. The ideal candidate will bring 5+ years of relevant experience and advanced data engineering skills across the analytics data stack.

Responsibilities

• Design, build, and maintain scalable ETL/ELT data pipelines across cloud and on-prem sources, ensuring data quality, lineage, and auditability.
• Develop and optimize Python/PySpark and SQL transformations for large-scale, high-volume financial datasets.
• Architect and operate data pipeline orchestration using tools such as Airflow, Databricks Workflows, and Step Functions to automate ingestion, transformation, and delivery.
• Build and maintain CI/CD pipelines with GitHub and GitHub Actions to enable automated testing, deployment, and version-controlled infrastructure changes.
• Develop AWS-based solutions using S3, Glue, EMR, Redshift, Lambda, and IAM to support analytics, reporting, and downstream ML use cases.
• Deploy and manage infrastructure and pipelines as code, applying best practices for environment promotion, rollback, and monitoring.
• Monitor and troubleshoot pipeline performance, query efficiency, and cost across the data stack.
• Collaborate with data scientists, analysts, product, and risk/compliance teams to translate requirements into robust data solutions.
• Apply data governance, security, and regulatory compliance standards for financial data, including PII, SOX, and PCI.
• Document pipeline architecture, data models, and processes; contribute to engineering standards and code review practices.

Requirements

• Advanced proficiency in Python for scripting, automation, and data engineering workflows.
• Hands-on experience with PySpark for distributed processing at scale.
• Expert-level SQL, including advanced SQL capabilities such as window functions, query optimization, complex joins, and performance tuning.
• Solid experience with AWS services and cloud application/data development, including S3, Glue, EMR, Redshift, Lambda, IAM, and CloudWatch.
• Proven expertise building and orchestrating data pipelines using Airflow, Databricks Workflows, Step Functions, or equivalent.
• Hands-on CI/CD experience with GitHub / GitHub Actions for automated build, test, and deployment.
• Deep understanding of ETL/ELT design patterns, data modeling, and data warehousing concepts.
• Experience deploying infrastructure and pipelines via code (for example, version-controlled deployments).
• Demonstrated ability to optimize pipeline performance, query execution, and cloud resource and cost efficiency.
• Bachelor’s degree in computer science, Engineering, Data Science, or a related field (or equivalent practical experience).
• 5+ years of experience in data engineering, analytics engineering, or a related technical role.
• Prior experience in banking, fintech, or financial services, with awareness of regulatory and data-security requirements.

Technologies

• Python, PySpark, SQL
• Amazon Web Services (AWS): S3, Glue, EMR, Redshift, Lambda, IAM, CloudWatch
• Airflow, Databricks Workflows, Step Functions
• GitHub, GitHub Actions
• ETL, ELT, Terraform, CloudFormation
• Kafka, Kinesis, Spark Structured Streaming
• Great Expectations, dbt
• Databricks, Delta Lake, Unity Catalog, notebooks

Preferred / Desired Skills (Nice to Have)

• Hands-on experience with Databricks, including Delta Lake, Unity Catalog, notebooks, and cluster optimization.
• Familiarity with Terraform or CloudFormation for infrastructure as code.
• Experience with streaming technologies such as Kafka, Kinesis, and Spark Structured Streaming.
• Exposure to data quality and testing frameworks including Great Expectations and dbt tests.
• Knowledge of dbt for transformation and analytics engineering workflows.
• Understanding of financial data domains such as payments, lending, risk, fraud, or accounting data.
• Relevant certifications including AWS Certified Data Analytics / Solutions Architect and Databricks Certified Data Engineer.

Soft Skills

• Strong analytical and problem-solving skills with attention to detail and data accuracy.
• Excellent communication skills, including translating technical concepts for non-technical stakeholders.
• Collaborative mindset with experience working cross-functionally with analysts, engineers, and business teams.
• Self-directed approach and comfort owning projects end-to-end in a fast-paced, regulated environment.
• Strong ownership mentality around data quality, reliability, and documentation.