Senior Data Engineer - PGIM Technology

Prudential's PGIM Technology is seeking a Senior Data Engineer to design, build, and maintain scalable data and AI workflows on cloud platforms. The role emphasizes hands-on engineering, from data ingestion and transformation to deploying AI use cases such as semantic search and Retrieval-Augmented Generation, with collaboration across analysts, scientists, and business stakeholders. This hybrid position is based in Newark, New Jersey, offering a salary range of USD 115,000 to 155,000 per year.

Responsibilities

• Design, build, and maintain scalable data pipelines for ETL and ELT processes.
• Develop solutions using Microsoft Fabric, Azure Data Factory, or similar platforms.
• Implement data quality checks, transformations, and data modeling practices.
• Handle large-scale datasets with PySpark and SQL.
• Deploy and integrate machine learning models within data pipelines.
• Develop AI-enabled capabilities, including semantic search, embeddings, and Retrieval-Augmented Generation (RAG).
• Build and expose data and AI services via REST APIs.
• Contribute to CI/CD pipelines using GitHub Actions or Azure DevOps.
• Apply best practices for data security, governance, and access control.
• Monitor, troubleshoot, and optimize pipelines and AI systems.
• Collaborate with analysts, data scientists, and stakeholders to align solutions with business needs.
• Contribute to documentation and reusable components to support broader teams.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
• 3–6 years of experience in data or AI engineering.
• Strong Python and SQL skills.
• Experience with data pipeline tools such as Azure Data Factory, Microsoft Fabric, AWS Glue, or equivalent.
• Experience with Spark or PySpark.
• Familiarity with data lakes and data warehouses.
• Experience deploying machine learning models.
• Basic understanding of MLOps.
• Exposure to Generative AI concepts, including embeddings, RAG, and LLM APIs.
• Experience with REST APIs and microservices.
• Familiarity with CI/CD and Git.
• Understanding of data governance and security.
• Strong problem-solving and communication skills.

Technologies

• Python
• SQL
• PySpark / Spark
• Microsoft Fabric
• Azure Data Factory
• AWS Glue
• GitHub Actions
• Azure DevOps
• REST APIs

Benefits

• Market-competitive base salary with annual bonus opportunities at every level.
• Comprehensive medical, dental, vision, life, and disability insurance, plus PTO and leave options including parental and military leave.
• 401(k) plan with company match up to 4%.
• Company-funded pension plan.
• Wellness programs, including up to $1,600 per year for personal wellbeing purchases.
• Work/Life Resources covering parenting, housing, senior care, finances, pets, legal matters, education, mental health, and career development.
• Education benefit to support traditional college enrollment and accredited certificate programs.
• Employee Stock Purchase Plan allowing purchases at 85% of the lower price after one year of service.