Lead AI & Data Engineer

Medidata is building AI-enabled data capabilities within its Virtual Twin ecosystem, and this role will help design, build, and deploy production-grade AI and data products. In a hybrid setting based in New York, NY, you will lead scalable and secure AI-enabled data solutions that connect modern data architectures with LLMs, RAG, and agent orchestration.

What you will do

• Build scalable, automated data pipelines and AI-powered solutions using modern AI technologies, vector databases, and automation, applying CI/CD practices with Python, SQL, and Git.
• Design, develop, and deploy production-grade AI-powered solutions using LLMs, RAG, and agentic workflows.
• Implement and maintain integrations via API and MCP to connect applications and agents to enterprise data, tools, and skills under governance, security, and compliance controls.
• Apply agent orchestration and workflow patterns using frameworks such as LangGraph, LangChain, or equivalent technologies.
• Develop agent-driven metadata management solutions, including data catalogs, business glossaries, lineage documentation, and governance policy support.
• Use advanced skills in data architecture, data science engineering, data modeling, and data quality with cloud-native technologies.
• Document and communicate technical work clearly to stakeholders across technical and non-technical groups.
• Collaborate in Agile environments and with cross-functional teams.

What you bring

• 7+ years of experience in data engineering or data science, including at least 2 years of hands-on experience in AI and data engineering, grounded in enterprise data architecture.
• Bachelor’s degree in a technical or scientific field (such as Data Engineering, Data Science, Computer Science, or similar).
• Hands-on experience developing AI-native solutions with LLMs, RAG, AI agents, and agent orchestration patterns.
• Experience integrating AI/ML pipelines into production-grade data infrastructure, including API and MCP standards and integration topics such as model access controls, context management, auditability, security, compliance monitoring, and lifecycle management.
• Experience implementing AI governance and guardrails, including access controls, data privacy, auditability, and policy enforcement across AI services and agents.
• Proficiency in Python, SQL, and modern data engineering practices, including CI/CD and Git-based development.
• Hands-on experience with tools and concepts such as Airflow, CDC, batch processing, and job scheduling.
• Strong problem-solving, communication, and collaboration skills, with the ability to troubleshoot complex issues and translate business requirements into secure, scalable, production-ready solutions.
• Healthcare or clinical trial data experience is not required, but interest in learning how Life Sciences data improves is important.

Technologies

• Python, SQL, Git
• LLMs, RAG, vector databases
• CI/CD
• API, MCP
• LangGraph, LangChain
• Data catalogs, data modeling, cloud-native technologies
• Airflow, CDC, batch processing, job scheduling

Additional skills that are a plus

• Knowledge graphs, advanced vector search, and retrieval optimization techniques.
• Familiarity with clinical research data standards and formats such as CDISC SDTM and ADaM.
• Experience with AI evaluation frameworks, benchmarking, and continuous monitoring of AI solution performance.
• Experience optimizing cloud infrastructure, AI workloads, and data processing costs.
• Familiarity with semantic layers, metadata-driven architectures, and automated data cataloging.

Compensation and benefits

The salary range for this position physically based in the NYC/NJ Metro Area is $135,000-$180,000 per year. Base pay is one component of Medidata’s Total Rewards, and many non-sales positions may be eligible for annual bonuses based on applicable plan documents.

• Medical, dental, life and disability insurance
• 401(k) matching
• Family leave
• Flexible paid time off
• 10 paid holidays per year

Location and work model

• New York, NY (hybrid)
• Medidata follows a hybrid office policy for in-person positions, with expectations to work on site a certain number of days per week per Company policy.

Education: Bachelor’s degree in a technical or scientific field (e.g., Data Engineering, Data Science, Computer Science, or similar).

Minimum experience: 7 years.

Role focus: Lead scalable, secure, reliable AI-enabled data solutions using LLMs, RAG, agent orchestration, and modern data architectures within the Virtual Twin ecosystem.