Talent.com
DeWinter Group
AI Safety & Evaluation EngineerDeWinter Group • Campbell, CA
AI Safety & Evaluation Engineer

AI Safety & Evaluation Engineer

DeWinter Group • Campbell, CA
30+ days ago
Salary
$50.00 hourly
Job type
  • Full-time
Job description

Title: AI Safety and Evaluations Engineer
Job Type: Contract
Contract Length: 12 Months
Pay Range: $50/hr – $/hr
Start Date: ASAP
Location: Remote

About the Opportunity:

Our client, a leader in AI testing and Generative AI solutions, is looking for a skilled AI Safety and Evaluations Engineer to join their team for a 12-month engagement. This project involves designing and building rigorous evaluation frameworks to measure model bias, hallucinations, and toxicity, ensuring models are safe and compliant before deployment. This is a high-impact role that requires a self-motivated professional who can hit the ground running and deliver results quickly.

Key Responsibilities & Deliverables:

This role is focused on the successful completion of specific tasks and deliverables. Your responsibilities will include:

  • Designing and building rigorous evaluation frameworks to measure model bias, hallucinations, and toxicity.
  • Creating automated "Eval" datasets to benchmark new models before they are promoted to production.
  • Developing metrics for "Grounding" and "Faithfulness" in RAG-based systems.
  • Building monitoring tools that flag harmful or non-compliant AI outputs in real-time.
  • Partnering with legal and ethics teams to translate policy into technical safety constraints.
Required Skills & Experience:

We are looking for someone with a proven track record of successful contract engagements. The ideal candidate will have:
  • 3+ years of experience in AI Research or Quality Engineering.
  • Deep expertise in model evaluation techniques and NLP metrics (ROUGE, BLEU, BERTScore). This isn't a learning role—you need to be a subject matter expert.
  • Demonstrated ability to work autonomously and manage your own time effectively to meet project goals.
  • Experience with Python, data analysis tools, and LLM-as-a-Judge frameworks.
  • Strong communication skills to provide clear and concise status updates to the project team.
#LI-LD1
#LI-JN1
Create a job alert for this search

AI Safety & Evaluation Engineer • Campbell, CA

Similar jobs

Applied AI Engineer

Augment ComputingPalo Alto, CA, United States
Full-time

AugmentCode is the only AI coding assistant built for professional software engineers working in large, production‑grade codebases.Our Context Engine understands your entire repo, enabling develope... Show more

 • Promoted

Staff Engineer - Applied AI

GEICOPalo Alto, CA, United States
Full-time

At GEICO, we offer a rewarding career where your ambitions are met with endless possibilities.Every day we honor our iconic brand by offering quality coverage to millions of customers and being the... Show more

 • Promoted

Lead AI Platform Engineer, Agentic GenAI

Capital One National AssociationSan Jose, CA, United States
Full-time

A leading financial services firm in San Jose seeks a Distinguished AI Engineer to design and implement robust AI platforms.The role requires extensive experience in developing scalable AI solution... Show more

 • Promoted

Senior Software Engineer, Simulator Evaluation & Metrics

WaymoMountain View, CA, United States
Full-time

A leading autonomous driving technology company is seeking a Senior Software Engineer to architect evaluation methodologies for their hybrid simulator.Responsibilities include developing high-throu... Show more

 • Promoted

Applied AI Engineer

Qualified HealthPalo Alto, CA, United States
Full-time

At Qualified Health, we’re redefining what’s possible with Generative AI in healthcare.Our infrastructure provides guardrails for safe AI governance, healthcare-specific agent creation, and real-ti... Show more

 • Promoted

AI Security Engineer: Secure AI Apps & ML Systems

Fortinet, Inc.Sunnyvale, CA, United States
Full-time

A leading cybersecurity firm is seeking an experienced AI Security Engineer in Sunnyvale, CA.This role involves safeguarding AI applications, collaborating with teams, and providing expertise in se... Show more

 • Promoted

AI Researcher - Efficiency Engineer, Hybrid

Cisco Systems, Inc.San Jose, CA, US
Full-time

The application window is expected to close on: 06/20/2026.Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.This position is based... Show more

 • Promoted

Applied AI Engineer

Catalyst LabsSanta Clara, CA, United States
Full-time

Catalyst Labs is a leading talent agency with a specialized vertical in Applied AI, Machine Learning, and Data Science.We stand out as an agency that’s deeply embedded in our clients’ recruitment o... Show more

 • Promoted

Staff AI Application Engineer Lead

FortinetSunnyvale, CA, US
Full-time

Fortinet is a fast-growing and dynamic cybersecurity company with a growing and hard-working IT business application team.We are looking for a Senior AI Engineering Lead to join our growing team.In... Show more

 • Promoted

Senior Software Engineer, Generative AI, Ads Safety

Google Inc.Mountain View, CA, United States
Full-time

Senior Software Engineer, Generative AI, Ads Safety.Google place Mountain View, CA, USA.Bachelor’s degree or equivalent practical experience.C++ and Python programming languages.LLMs, Multi-Modal, ... Show more

 • Promoted

Senior Principal Analyst, AI Safety Strategy & Policy

GoogleSunnyvale, CA, United States
Full-time

A leading tech company based in Sunnyvale, CA, is seeking a Senior Principal Analyst for Search, Trust, and Safety.This pivotal role involves developing strategies and policies to enhance user safe... Show more

 • Promoted

AI Agent Developer

FlashiiSan Jose, CA, United States
Full-time

Get AI-powered advice on this job and more exclusive features.Flashii is looking for an AI Agent Developer for a short‑term internship that could lead to a full‑time opportunity.We are based in San... Show more

 • Promoted

Staff Software Engineer, Quantitative Evaluation (Hybrid)

WaymoMountain View, CA, United States
Full-time

An autonomous driving technology company in San Francisco is seeking a data-minded software engineer to improve the evaluation of its onboard software.The ideal candidate will have 5+ years of expe... Show more

 • Promoted

Autonomous AI Data & Evaluation Engineer

Pony.aiFremont, CA, United States
Full-time

A leading autonomous technology company in Fremont is seeking a Software Engineer to design and implement data handling tools for autonomous vehicles.Candidates should have strong programming skill... Show more

 • Promoted

Machine Learning Engineer, Safety

xAIPalo Alto, CA, United States
Full-time

AI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge.Our team is small, highly motivated, and focused on engineering excelle... Show more

 • Promoted

AI Developer / AI Engineer

Peppr AI (YC W25)Cupertino, CA, United States
Full-time

This range is provided by Peppr AI (YC W25).Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.As a founding AI engineer at Peppr AI, you\'ll take ... Show more

 • Promoted

AI Applications and Data Science Engineer

CXAppSan Ramon, CA, United States
Full-time

CXAPP is a forward-thinking technology company that leverages AI and data science to drive innovation and deliver cutting-edge solutions.We are seeking talented AI Applications and Data Science Eng... Show more

 • Promoted

AI Engineer

RelantoFremont, CA, United States
Full-time

Direct message the job poster from Relanto.Engage with customers to understand problem statements and technical requirements.Design and architect AI/ML cloud solutions.Develop system architecture d... Show more

 • Promoted

Senior AI Systems Engineer: Knowledge Synthesis

eGain CorporationSunnyvale, CA, United States
Full-time

A leading technology company in Sunnyvale is seeking a Senior Software Development Engineer to design and develop AI systems that transform customer operations.You will leverage cutting-edge techno... Show more

 • Promoted

Senior AI Systems Engineer: Inference Kernels & Runtimes

NVIDIA GruppeSanta Clara, CA, United States
Full-time

NVIDIA Gruppe is seeking talented AI systems engineers to advance innovative technologies in AI inference systems software.This role involves developing cutting-edge libraries, code generators, and... Show more