AI Evaluation AnalystVirtualVocations • Billings, Montana, United States
AI Evaluation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.variable_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]
A company is looking for an AI Agent Evaluation Analyst (Freelance).
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking with the ability to reason about complex systems
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Exposure to LLMs, prompt engineering, or AI-generated content
[job_alerts.create_a_job]
Ai Analyst • Billings, Montana, United States
[internal_linking.similar_jobs] A company is looking for a Sr AI Solutions Analyst II to support the discovery, design, and adoption of AI solutions across the business.
Key Responsibilities Partner with business units to identi...[show_more] A company is looking for an Assessment Scientist to design and deliver AI-native assessments.Key Responsibilities Design and deliver assessments using AI-assisted workflows and evidence-centered ...[show_more] 
In this hourly, remote contractor role, you will review AI-generated legal analyses and / or generate expert legal content, evaluating reasoning quality and step-by-step issue-spotting while providin...[show_more] A company is looking for a Data Analyst to drive the collection, activation, and optimization of product data within experience platforms.
Key Responsibilities Lead the execution of product data i...[show_more] In this hourly, remote contractor role, you will review AI-generated finance analyses and / or generate expert finance content, evaluating reasoning quality and step-by-step problem-solving while pro...[show_more] In this hourly, remote contractor role, you will review AI-generated medical responses and / or generate expert healthcare content, evaluating reasoning quality and step-by-step clinical problem-solv...[show_more] As a remote, hourly paid R Engineer, you will review AI-generated responses and generate high-quality R and data-analysis-focused content, evaluating the reasoning quality and step-by-step problem-...[show_more] A company is looking for an Implementation Analyst (Remote).Key Responsibilities Design, develop, implement, and support EDI and application integration solutions Manage client timelines and del...[show_more] A company is looking for a Part-Time Evaluator in Data Analytics.Key Responsibilities Analyze and evaluate assessments to determine students' mastery of learning objectives and course competencie...[show_more] A company is looking for a Research Analyst - Consumer Insights Research.Key Responsibilities Collaborate with team members to ensure project deadlines and logistics are met Assist in operationa...[show_more] 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for a Credit Analyst - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs based on professional experience in the credit analysis field Develop prompts tha...[show_more] A company is looking for a Competitive Research Analyst to join their Competitive Insight team.Key Responsibilities Track competitors and market movements to maintain comprehensive market awarene...[show_more] A company is looking for an Applied Scientist for a short-term assignment.Key Responsibilities Lead the design, development, deployment, and scaling of advanced ML models using NLP and generative...[show_more] A company is looking for an AI Agent Evaluation Analyst (Freelance).Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, miss...[show_more] A company is looking for a Credit Authorizers, Checkers, and Clerks - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs based on professional experience in credit-related tasks...[show_more] A company is looking for a Statistical Assistant - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs related to your professional field and provide structured feedback Develop...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
Senior AI Solutions Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Assessment Scientist
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Legal Expert - AI Trainer
SuperAnnotate • Billings, Montana, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Data & Analytics Strategy Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Senior Finance Specialist - AI Trainer
SuperAnnotate • Billings, Montana, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Medical General Expert - AI Trainer
SuperAnnotate • Billings, Montana, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote R Engineer - AI Trainer
SuperAnnotate • Billings, Montana, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Implementation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Data Analytics Evaluator
VirtualVocations • Billings, Montana, United States
[job_card.part_time]
[last_updated.last_updated_1_day] • [promoted]
Research Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Investment Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Billings, MT, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Credit Analyst AI Trainer
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Competitive Research Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
Applied Scientist - Texas
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Agent Evaluation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Trainer for Credit Evaluation
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Statistical Assistant AI Trainer
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Investment Analyst - AI Model Trainer
Data Annotation • Billings, MT, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]