Generative Ai Security Evaluation

Generative Ai Security Evaluation Engineer Jobs

5 jobs found

Today

Cloud Security Engineer - AI Trainer

DataAnnotation Los Angeles, CA 🇺🇸 Remote

Los Angeles, CA

Today

$50 - $100/hr Contract Ai Security Cloud Platforms Cloud Security

Apply Now

Job Description

DataAnnotation is seeking an experienced Cloud Security Engineer - AI Trainer to support the training of AI models focused on cybersecurity. This fully remote contract role involves reviewing AI-generated cybersecurity content for accuracy and validity, while also working on security-oriented technical problem solving that helps improve next-generation AI security outputs.

The position is open to qualified candidates based in the US, Canada, UK, Ireland, Australia, and New Zealand, with flexible scheduling and hourly compensation.

Responsibilities

Assess AI-generated cybersecurity content, including threat analysis, vulnerability assessments, and offensive security techniques, to determine real-world accuracy and validity.
Develop and solve security-focused technical problems used to train AI systems.
Produce clear technical explanations and security-relevant code.
Provide feedback that directly informs the next iteration of AI security models.

Requirements

At least 2+ years of hands-on experience in a cybersecurity role, such as penetration testing, red teaming, incident response, detection engineering, DFIR, malware analysis, threat intelligence, or adjacent areas, including government or military cyber operations.
Coding experience required, with the ability to navigate and patch a codebase.
Fluent English (native or bilingual level) and strong technical writing skills.
Bachelor’s degree preferred.
Cybersecurity certifications are a plus, including OSCP, CRTO, GREM, GCFA, CEH, and similar.

Compensation and Contract Details

Job type: contract
Location: Los Angeles, CA (remote)
Rate: projects paid hourly at $50 to $100 USD per hour

Benefits

Fully remote: work from anywhere in the US, Canada, UK, Ireland, Australia, and New Zealand.
Flexible schedule: choose which projects to take on and when to work.
Competitive pay: hourly projects at $50 to $100 USD per hour.
Higher bonus rates: available for certain projects.
Impact: help shape cybersecurity-focused AI technologies.

Additional Notes

Payment is made via PayPal.
DataAnnotation will never ask you for money.
This opportunity is only available to people located in the US, Canada, UK, Ireland, Australia, and New Zealand.

View full job page →

7d ago

Machine Learning Engineer

GitHub, Inc. Remote 📍 United States Remote

Remote

7d ago

$108k - $286k/yr Full time Agentic Ai Ai Enabled Security Ai Security

Apply Now

Job Description

GitHub, Inc. is seeking an experienced Machine Learning Engineer to design, build, and deploy agentic LLM-based solutions that help detect and prevent fraud, abuse, and security threats on GitHub at scale. This role also focuses on identifying vulnerabilities that enable abuse, performing ad-hoc analysis, and measuring the real-world impact of safety and integrity efforts.

Responsibilities

Design, build, and deploy agentic solutions that use large language models to detect and prevent fraud, abuse, and security threats at scale, including applications such as content classification and multi-step agentic investigation.
Build production-grade systems that run reliably against high-volume event streams, leveraging AI coding assistants to accelerate and improve engineering work.
Build and operate scalable ML systems on cloud platforms, including Azure AI Foundry, for training, deploying, and serving models and agentic solutions in production.
Evaluate and improve existing models and agentic solutions using offline evaluations (including tool-use loops and LLM-as-judge evaluation), performance metrics, and feedback from operational deployments.
Identify vulnerabilities in products that lead to abuse and consult with product teams when reviewing new features.
Collaborate with cross-functional partners such as data scientists, software engineers, product managers, and content moderators to integrate agentic solutions into production systems.
Document the systems you build and support the technical growth of peers.

Requirements

4+ years experience in machine learning, or a related field.
OR a Bachelor's Degree in Computer Science, Software Development, Electrical or Computer Engineering, Mathematical Sciences, or a related field, plus 2+ years experience in machine learning, or a related field.
OR a Master's Degree in Machine Learning, Computer Science, Software Development, Electrical or Computer Engineering, Mathematical Sciences, or a related field.
OR equivalent experience.
Strong understanding of large language models, including hands-on experience applying them at scale for classification, agentic workflows, or agents.
Strong software engineering skills, including experience building with AI coding assistants.
Experience designing or evaluating agentic systems such as tool-use loops, multi-step workflows, or LLM-as-judge evaluation.
Hands-on experience building and operating classification or detection systems at scale, including experience with imbalanced data and precision/recall tradeoffs.
Experience in Trust and Safety, National Security, or fighting spam, malware, fraud, and threat actor activity at scale.
Experience in responsible AI and Safety-by-Design.
Experience managing user data and privacy.
Solid understanding of machine learning algorithms (supervised and unsupervised learning, anomaly detection, etc.) and practical implementation.

Technologies

Large language models (LLMs)
Azure AI Foundry
Tool-use loops
LLM-as-judge evaluation
AI coding assistants
Machine learning
Precision/recall tradeoffs

What We Value

Collaboration: the best work is done together.
Empathy: putting people first.
Quality: setting the standard for excellence.
Positive Impact: making the world a better place through the work.
Shipping: creating things for the people using them.

Compensation

The base salary range for this job is USD $107,700.00 - USD $285,900.00 /Yr.

Locations

Remote (United States)

View full job page →

7d ago

Application Security Engineer - AI Trainer

DataAnnotation Wichita Falls, TX 🇺🇸 Remote

Wichita Falls, TX

7d ago

$50 - $100/hr Contract Ai Security Ai Security Training Application Security

Apply Now

Job Description

Fully remote, flexible, and impact-driven: This contract role helps train AI chatbots for cybersecurity. You will review AI-generated security content, build security-focused training tasks, and provide feedback that improves the next generation of AI security models. Work on your own computer with scheduling control, and earn $50 to $100 USD per hour on paid projects.

Why this role

Fully remote: work from anywhere in the US, Canada, UK, Ireland, Australia, or New Zealand.
Flexible schedule: choose which projects you take on and when you work.
Competitive hourly pay: $50 to $100 USD per hour depending on project.
Potential higher rates: certain projects may offer higher bonus rates.
Real-world influence: help shape AI technologies built for cybersecurity use cases.

Responsibilities

Assess AI-generated cybersecurity content, including threat analysis, vulnerability assessments, and offensive security techniques, with an emphasis on real-world accuracy and validity.
Develop and solve security-focused technical problems used to train AI systems.
Produce clear technical explanations and write security-relevant code that supports training and evaluation.
Deliver feedback that directly informs improvements to future AI security model outputs.

Requirements

2+ years of hands-on experience in cybersecurity, such as penetration testing, red teaming, incident response, detection engineering, DFIR, malware analysis, threat intelligence, or adjacent fields, including government or military cyber operations.
Coding experience required: comfort navigating and patching a codebase.
English fluency (native or bilingual level) with strong writing skills.
Bachelor’s degree preferred.
Cybersecurity certifications are a plus (examples include OSCP, CRTO, GREM, GCFA, CEH, and similar).

Additional details

Payment is made via PayPal.
They will never ask for money from you.
This job is available only to candidates in the US, Canada, UK, Ireland, Australia, and New Zealand.

View full job page →

11d ago

Lead Generative AI Engineer, VP

Citi Jacksonville, FL

Jacksonville, FL

11d ago

$126k - $189k/yr Full time Backend Developer Manager Agentic Ai

Apply Now

Job Description

Citi is seeking a Lead AI/GenAI Engineer to drive the design, development, and deployment of generative AI capabilities within the AI Automation Forward Deployed Engineering (FDE) team. This hybrid role in Jacksonville, FL focuses on hands-on model engineering, enterprise-grade safety and governance, and cloud-native AI pipeline delivery for intelligent operations and automation.

Responsibilities

Lead hands-on development and delivery of intelligent operations solutions, including advanced chatbots, agentic workflow automation, and AI-assisted decision platforms.
Fine-tune, prompt-engineer, and deploy open-source LLMs such as Llama, Mistral, and Gemma, along with proprietary models via Google Vertex AI.
Implement RAG (Retrieval-Augmented Generation) pipelines.
Architect and implement AI safety frameworks and guardrails, including examples such as NeMo Guardrails and Llama Guard, plus deterministic controls to prevent hallucinations, protect data privacy, and support regulatory compliance.
Design and deploy scalable AI pipelines using Python, containerization (Docker, Kubernetes), and cloud-native architectures.
Apply deep knowledge of financial services technology to align AI models with risk management, auditability, and financial data security standards.

Required Skills and Qualifications

Deep understanding of LLM architectures, vector databases (including Pinecone, Milvus, Chroma), embedding models, and AI orchestration frameworks (including Google ADK and LangChain).
Hands-on experience with Google Vertex AI (or equivalent enterprise cloud AI suites) and deployment of open-source models such as Llama, Mistral, and Gemma.
Proven experience implementing deterministic controls, semantic routing, guardrails, and bias and hallucination detection in production AI systems.
Expert-level Python skills, including AI/ML libraries such as PyTorch, Hugging Face, Pandas, and NumPy, plus API development experience with FastAPI and Flask.
Strong containerization experience with Openshift, Docker, and Kubernetes, along with CI/CD pipelines and cloud-native delivery practices.
Experience with financial services technology, including familiarity with financial products, compliance requirements, and the competitive AI landscape in fintech.
Demonstrated ability to lead AI and automation programs from concept through production, providing credible technical direction at a senior level.

Technologies

Google Vertex AI, Python
Docker, Kubernetes, Openshift
NeMo Guardrails, Llama Guard
Llama, Mistral, Gemma
RAG (Retrieval-Augmented Generation)
Pinecone, Milvus, Chroma
Google ADK, LangChain
PyTorch, Hugging Face, Pandas, NumPy
FastAPI, Flask
CI/CD pipelines, containerization

Benefits

Hybrid working model with 3 days in the office and 2 days working remotely.
Continuous learning and development opportunities.
Competitive financial wellbeing and benefits package.
Medical, dental & vision coverage.
401(k).
Life, accident, and disability insurance.
Wellness programs.
Paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays.
Discretionary and formulaic incentive and retention awards (for eligible employees).

Additional Qualifications

Experience deploying and tuning models on Google Vertex AI or equivalent enterprise-grade cloud AI platforms.
Familiarity with additional AI orchestration tools and emerging open-source LLM frameworks beyond core requirements.
Bachelor’s degree/University degree or equivalent experience; Master’s degree in Computer Science, Artificial Intelligence, or a related technical discipline is preferred.

Job Details

Location: Jacksonville, FL (hybrid)
Time Type: Full time
Job Family Group: Technology
Job Family: Applications Development
Primary Location: Irving Texas United States
Salary Range: USD 125,760 - 188,640 per year
Anticipated Posting Close Date: Aug 15, 2026
Other Relevant Skills: Artificial Intelligence (AI), Containerization, Generative AI, Large Language Models (LLMs), Python (Programming Language)

Automated Processing Notice

Citi uses automated processing, including artificial intelligence, for legitimate business interests (or reasonable and appropriate business purposes) to identify and align the candidate’s skills and abilities with a specific job opening. If you choose or consent, Citi can also match your skills and abilities to other suitable roles. Citi’s automated processing and AI do not involve relying on automatic or autonomous decision-making.

View full job page →

12d ago

Security Engineer, AI Platform Engineering

Saronic Technologies San Diego, CA

San Diego, CA

12d ago

Full time Agent Agentic Ai Ai Security

Apply Now

Job Description

Saronic Technologies is hiring a Security Engineer for AI Platform Engineering in San Diego, CA (onsite) to help make AI safe and self-service across the company. The role focuses on secure-by-default AI platform capabilities, governance and standards, and practical education for teams using AI.

Responsibilities

Build AI-powered applications, agents, and automations for teams across the company on cloud platforms, using real backends, infrastructure, and integrations to real data and systems to deliver properly hardened, compliant, well-hosted, secure-by-default solutions.
Teach teams to use AI safely and effectively and establish company-wide standards for responsible and safe AI usage.
Own AI governance, visibility, and inventory, including monitoring and logging of AI usage and prompt- and output-level data-loss-prevention to protect sensitive information, including customer data.
Implement guardrails for AI usage by treating AI agents as identities with least privilege, governing model and agent access, and making the sanctioned path the best path to reduce ungoverned “shadow AI” risk.
Build agents and workflows that work in practice, including design of tool use and MCP integrations, management of context and memory, and validation of quality using evaluative loops.
Represent the AI security team across the organization and partner with teams to understand their goals, enable effective and safe AI usage, and deliver solutions that require engineering, security guardrails, and proper hosting.
Enable broader business adoption by adding the right guardrails and visibility and building more complex, well-hosted, secure AI solutions so ideas mature into properly managed capabilities rather than creating avoidable security and liability exposure.

Requirements

Demonstrated AI building experience: shipped agents, tools, or automations with LLMs and can explain the hard parts.
Ability to build real backends and infrastructure beyond demos, including APIs, data pipelines, authentication and secrets, hosting and deployment (containers, Infrastructure-as-Code, CI/CD), and systems integration supported by observability, evals, and graceful failure handling.
Fluency with modern agent concepts such as agentic loops, agent harnesses, context engineering, tool use/function calling, MCP, and evals.
Working knowledge of AI model and platform strengths and weaknesses, including selecting appropriate models for specific tasks.
Enough software and security foundation to build and secure production systems, with sound judgment on data handling and safe AI usage.
Immense kindness and patience when supporting and teaching non-technical users.
Ability to obtain and maintain a U.S. security clearance.
No focus on degrees or certifications in Artificial Intelligence; emphasis is on outcomes and demonstrated capability, with openness to self-taught practitioners and career-changers.

Technologies

LLMs, cloud platforms, APIs, data pipelines, authentication, secrets
Containers, Infrastructure-as-Code, CI/CD, observability, evals
Agentic loops, agent harnesses, context engineering, tool use, function calling
MCP, prompt- and output-level data-loss-prevention

Preferred Qualifications

Experience with AI governance, DLP, monitoring/logging, or guardrails for AI usage.
Experience building and hosting AI solutions in the cloud.
Security background (application, cloud, or data protection).
Experience teaching, enabling, or supporting non-technical teams.

How You Work

Kind and patient by default, meeting people where they are and staying through problems rather than handing off issues.
Low-ego and intellectually humble, explaining technical ideas to non-technical colleagues without condescension, acknowledging unknowns, and owning mistakes.
A natural teacher and enabler who turns one-off help into reusable guidance and patterns.
Deeply empathetic and a good listener, starting from the person’s actual problem and acting as the voice of the user back to security and platform teams.
Values-aligned and high-agency, collaborating across engineering, security, legal, and business teams with integrity while navigating ambiguity.

View full job page →

Generative Ai Security Evaluation Engineer Jobs

Cloud Security Engineer - AI Trainer

Machine Learning Engineer

Application Security Engineer - AI Trainer

Lead Generative AI Engineer, VP

Security Engineer, AI Platform Engineering

Get Generative Ai Security Evaluation Engineer Jobs alerts