Data Engineer - AI (Spark, Databricks and GCP)

Cotiviti is seeking a Data Engineer with an AI focus to design and manage data pipelines, oversee data management and validation, and partner with engineering, database administration, and cloud teams to maintain data integrity across environments. This remote role offers an annual salary between USD 101,000 and 132,000 and requires at least two years of data engineering experience along with a bachelor’s degree or equivalent work experience. The position emphasizes Spark, Databricks, and Google Cloud Platform to support robust data operations.

Responsibilities

• Develop and maintain Spark scripts at intermediate to advanced levels to manage data, validate quality, and enable data integration.
• Design and maintain SQL scripts of basic to intermediate complexity for data handling and validation tasks.
• Tune queries to improve efficiency for ongoing daily tasks.
• Perform data analysis to identify issues and anomalies.
• Collaborate with Engineering, DBA, Cloud Operations and other groups to troubleshoot environmental or network issues that affect work; provide after-hours or weekend support as needed.
• Build and sustain data pipelines as required by projects.
• Verify task results to ensure all requirements are met.
• Follow industry and organizational compliance standards to preserve data integrity.
• Maintain productivity records and manage tasks through the department ticketing system within deadlines.
• Meet organizational and personal goals as outlined in performance reviews and goal-setting processes.
• Complete special projects and any other duties as assigned.
• Perform duties with or without reasonable accommodation.

Requirements

• Bachelor’s degree in Computer Science, Information Technology or equivalent work experience.
• At least 3 years of hands-on experience with big data technologies (Spark, S3, Kafka, Ray, Hadoop, etc.).
• 3+ years working with big data and cloud technologies (Databricks, AWS, Azure, Hadoop, Spark, Snowflake, etc.).
• 3+ years of cloud experience across AWS, Azure, GCP, OCI, and similar platforms.
• 3+ years of experience with relational databases (Oracle, MS SQL, Vertica, etc.) and SQL/PLSQL or other data integration/ETL tools.
• Databricks or AWS certifications are a strong plus.
• Familiarity with data pipeline orchestration tools such as Airflow or Databricks Workflows.
• 3+ years of data analysis experience, with preference for healthcare data such as enrollment, medical claims, or pharmacy claims.
• Proficient in Microsoft Office Suite (PowerPoint, Word, Excel, Outlook).
• Flexible work schedule.
• Experience using project management tools like JIRA.
• Databricks and/or Snowflake environment familiarity is advantageous.

Technologies

• Spark
• Amazon S3
• Kafka
• Ray
• Hadoop
• Databricks
• AWS
• Azure
• Snowflake
• Google Cloud Platform (GCP)
• Oracle
• MS SQL
• Vertica
• SQL
• PL/SQL
• Airflow
• Databricks Workflows
• JIRA
• Microsoft Office Suite

Benefits

• Medical insurance
• Dental insurance
• Vision insurance
• Disability insurance
• Life insurance
• 401(k) savings plan
• Paid family leave
• Holidays (9 per year)
• PTO (17-27 days per year)

Mental requirements

• Strong analytical abilities.
• Excellent verbal, listening and written communication skills.
• Ability to multitask and prioritize to meet deadlines and tight turnaround times.
• Ability to work well independently or within a team.

Working conditions and physical requirements

• Typically stationary work with extended periods of sitting or standing.
• Repetitive motions that involve the wrists, hands, or fingers.
• Must maintain a dedicated, secure home workspace.
• Reliable high-speed internet connectivity and a suitable office setup.