Aws Cloudwatch Engineer Jobs

1 job found

Today

Software Engineer III - Senior Databricks/Spark/AWS Data Engineer

JPMorganChase Columbus, OH

Columbus, OH

Today

Full time Artificial Intelligence AWS Aws Cloudwatch

Apply Now

Job Description

Lead secure, scalable data engineering efforts on the Databricks-on-AWS lakehouse, delivering production pipelines that support workforce analytics and BI reporting.

Responsibilities

Design, build, and maintain new Databricks data pipelines using PySpark, focusing on secure, high-quality production code
Review and debug data engineering processes implemented by others to improve reliability and delivery
Optimize and tune PySpark jobs and Databricks clusters for performance, scalability, and cost efficiency (including partitioning, caching, and resource management)
Build scalable data frameworks for end-to-end Databricks pipelines using medallion lakehouse patterns (bronze/silver/gold) for workforce data analytics
Implement data quality checks and validation using Delta Lake and Delta Live Tables expectations to ensure accuracy and reliability
Set up robust monitoring and alerting to detect and address data ingestion issues early, using Databricks and AWS CloudWatch to improve performance and throughput
Identify recurring pipeline issues and opportunities to eliminate or automate remediation using Databricks Workflows and AWS-native automation
Use AI and agentic AI solutions to accelerate pipeline development and apply AI-assisted engineering tools (e.g., Claude, GitHub Copilot) to improve productivity and code quality
Provision and deliver curated, reliable datasets to BI partners using Sigma, Tableau, and Alteryx
Collaborate with business stakeholders to understand requirements, then create architecture and design artifacts for complex applications
Participate in software engineering communities of practice exploring new and emerging technologies, supporting a culture of diversity, opportunity, inclusion, and respect

Requirements

3+ years of applied experience in data engineering with formal training or certification in software engineering concepts, including design, application development, testing, and operational stability
Advanced hands-on expertise in Apache Spark (PySpark) for large-scale distributed processing
Strong proficiency building and operating production pipelines on Databricks using Delta Lake and lakehouse patterns
Strong expertise across AWS data ecosystem including S3, EMR, Glue, Lambda, and Athena, plus AWS storage and compute services
Experience with data formats including Parquet and Iceberg
Strong Python skills for data processing and application development (with Java or Scala as a plus)
Experience with automation and continuous delivery using CI/CD pipelines and tooling such as Git/Bitbucket, Jenkins, or Spinnaker
Hands-on experience across system design, application development, testing, and operational stability, with advanced understanding of agile methodologies, application resiliency, and security
In-depth knowledge of the financial services industry and its IT systems
SQL and data modeling skills for efficient data management and retrieval (experience with Oracle is a plus)
Practical experience scheduling and automating job execution using Airflow and Autosys

Preferred Qualifications / Capabilities & Skills

Databricks certifications such as Databricks Certified Data Engineer Associate/Professional
Familiarity with generative AI and agentic AI frameworks, including AI coding assistants like Claude and GitHub Copilot
Deeper expertise in the AWS cloud platform and its broader service catalog

Technologies

Databricks, Apache Spark, PySpark, AWS
Delta Lake, Delta Live Tables, Delta Live Tables expectations
Databricks Workflows, AWS CloudWatch
AI, Agentic AI, Claude, GitHub Copilot
Sigma, Tableau, Alteryx
Python, Java, Scala, Git, Bitbucket, Jenkins, Spinnaker
S3, EMR, Glue, Lambda, Athena
Parquet, Iceberg, SQL, Oracle
Airflow, Autosys

Benefits

Competitive total rewards package including base salary determined based on role, experience, skill set, and location
Eligible roles may receive commission-based pay and/or discretionary incentive compensation in cash and/or forfeitable equity
Comprehensive health care coverage
On-site health and wellness centers
A retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching
Additional details about total compensation and benefits provided during the hiring process
Equal opportunity employer; values diversity and inclusion

Location

Columbus, OH (onsite)

About the Team

Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing
Corporate teams help set businesses, clients, customers, and employees up for success

View full job page →

Get Aws Cloudwatch Engineer Jobs alerts

New Aws Cloudwatch Engineer Jobs, daily.