AI Evaluation AnalystVirtualVocations • Ontario, California, United States
AI Evaluation Analyst
VirtualVocations • Ontario, California, United States
[job_card.1_day_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]
A company is looking for an AI Agent Evaluation Analyst (Freelance).
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking with the ability to reason about complex systems
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Exposure to LLMs, prompt engineering, or AI-generated content
[job_alerts.create_a_job]
Ai Analyst • Ontario, California, United States
[internal_linking.similar_jobs] A company is looking for a UX & Engagement Analyst (Journeyman) to support user experience optimization for Military Community and Family Policy digital platforms.
Key Responsibilities Evaluate us...[show_more] A company is looking for an Intrusion Analyst to provide technical expertise in confronting complex cyber challenges.Key Responsibilities Conduct advanced digital forensic investigations to uncov...[show_more] A company is looking for a Business Analyst with Accounting and AI Expertise.Key Responsibilities Integrate into client teams to collaborate with designers and engineers on accounting projects A...[show_more] A company is looking for an AI Knowledge Designer to optimize customer support through AI interactions and documentation.
Key Responsibilities Design and refine conversational flows for the AI cha...[show_more] A company is looking for a Credit Authorizers, Checkers, and Clerks - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs based on professional experience in credit-related tasks...[show_more] A company is looking for an Assessment Scientist to design and deliver AI-native assessments.Key Responsibilities Design and deliver assessments using AI-assisted workflows and evidence-centered ...[show_more] 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for a Clinical Evaluator to evaluate healthcare conversations conducted by AI agents.Key Responsibilities Review and evaluate healthcare conversations for professionalism, em...[show_more] 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for an Advanced Analyst.Key Responsibilities Partner with the business to prioritize high-impact initiatives Define, create, and visualize metrics for project success tracki...[show_more] A company is looking for an AI Agent Evaluation Analyst (Freelance).Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, miss...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for a Compensation, Benefits, and Job Analysis Specialist - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs related to your field and provide structured ...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] 
When it comes to finding the perfect locums assignment, sometimes it is all about who you know.CompHealth has been around for a long time and have a vast network of connections to facilities across...[show_more] A company is looking for an Associate Analyst Technology to ensure the smooth exchange of data between the organization and partner systems.
Key Responsibilities Provide technical expertise for ED...[show_more] A company is looking for a Detectives and Criminal Investigators - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs and provide structured feedback based on professional exper...[show_more]
UX Engagement Analyst
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Intrusion Analyst
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Business Analyst with AI Expertise
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Knowledge Designer
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Trainer for Credit Evaluation
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
AI Assessment Scientist
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Investment Analyst - AI Model Trainer
Data Annotation • Chino, CA, United States
[job_card.full_time] +1
[last_updated.last_updated_1_day] • [promoted]
Clinical Evaluator
VirtualVocations • Ontario, California, United States
[job_card.part_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Remote M&A Associate - AI Trainer ($50-$60 / hour)
Data Annotation • Rancho Santa Margarita, California
[filters.remote]
[job_card.full_time] +1
[last_updated.last_updated_30] • [promoted]
Remote Financial Advising Expert - AI Trainer ($50-$60 / hour)
Data Annotation • Chino Hills, California
[filters.remote]
[job_card.full_time] +1
[last_updated.last_updated_30] • [promoted]
Advanced Data Analyst
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
AI Agent Evaluation Analyst
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
Remote Investment Analyst – AI Trainer ($50-$60 / hour)
Data Annotation • Upland, California
[filters.remote]
[job_card.full_time] +1
[last_updated.last_updated_30] • [promoted]
AI Trainer Specialist
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
Remote Investment Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Tustin, CA, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Travel Speech Language Pathologist (SLP) - $1,749 to $3,557 per week in Phelan, CA
AlliedTravelCareers • Phelan, CA, US
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Associate Technology Analyst
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
AI Trainer for Detectives
VirtualVocations • Ontario, California, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]