AI Evaluation AnalystVirtualVocations • Billings, Montana, United States
AI Evaluation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.variable_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]
A company is looking for an AI Agent Evaluation Analyst (Freelance).
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking with the ability to reason about complex systems
Familiarity with structured data formats, such as JSON / YAML
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Exposure to LLMs, prompt engineering, or AI-generated content
[job_alerts.create_a_job]
Ai Analyst • Billings, Montana, United States
[internal_linking.similar_jobs] A company is looking for a Sr AI Solutions Analyst II to support the discovery, design, and adoption of AI solutions across the business.
Key Responsibilities Partner with business units to identi...[show_more] A company is looking for an Assessment Scientist to design and deliver AI-native assessments.Key Responsibilities Design and deliver assessments using AI-assisted workflows and evidence-centered ...[show_more] A company is looking for a Data Analyst to drive the collection, activation, and optimization of product data within experience platforms.
Key Responsibilities Lead the execution of product data i...[show_more] A company is looking for an Implementation Analyst (Remote).Key Responsibilities Design, develop, implement, and support EDI and application integration solutions Manage client timelines and del...[show_more] A company is looking for a Part-Time Evaluator in Data Analytics.Key Responsibilities Analyze and evaluate assessments to determine students' mastery of learning objectives and course competencie...[show_more] A company is looking for a Research Analyst - Consumer Insights Research.Key Responsibilities Collaborate with team members to ensure project deadlines and logistics are met Assist in operationa...[show_more] 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for a Credit Analyst - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs based on professional experience in the credit analysis field Develop prompts tha...[show_more] A company is looking for a Competitive Intelligence Analyst (Remote).Key Responsibilities : Conduct ongoing competitive research and monitor industry trends to inform strategic decision-making De...[show_more] A company is looking for a Compensation, Benefits, and Job Analysis Specialist - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs related to your field and provide structured ...[show_more] A company is looking for a Competitive Research Analyst to join their Competitive Insight team.Key Responsibilities Track competitors and market movements to maintain comprehensive market awarene...[show_more] A company is looking for an Applied Scientist for a short-term assignment.Key Responsibilities Lead the design, development, deployment, and scaling of advanced ML models using NLP and generative...[show_more] A company is looking for an Associate Analyst Technology to ensure the smooth exchange of data between the organization and partner systems.
Key Responsibilities Provide technical expertise for ED...[show_more] A company is looking for an AI Agent Evaluation Analyst (Freelance).Key Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, miss...[show_more] A company is looking for a Credit Authorizers, Checkers, and Clerks - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs based on professional experience in credit-related tasks...[show_more] A company is looking for a Statistical Assistant - AI Trainer (Contract).Key Responsibilities Evaluate AI model outputs related to your professional field and provide structured feedback Develop...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] A company is looking for a Clinical Evaluator to evaluate healthcare conversations conducted by AI agents.Key Responsibilities Review and evaluate healthcare conversations for professionalism, em...[show_more]
Senior AI Solutions Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Assessment Scientist
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Data & Analytics Strategy Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Implementation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Data Analytics Evaluator
VirtualVocations • Billings, Montana, United States
[job_card.part_time]
[last_updated.last_updated_variable_days] • [promoted]
Research Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Investment Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Billings, MT, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Credit Analyst AI Trainer
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Competitive Intelligence Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Trainer Specialist
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Competitive Research Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
Applied Scientist - Texas
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Associate Technology Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Agent Evaluation Analyst
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
AI Trainer for Credit Evaluation
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Statistical Assistant AI Trainer
VirtualVocations • Billings, Montana, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote Investment Analyst - AI Model Trainer
Data Annotation • Billings, MT, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Clinical Evaluator
VirtualVocations • Billings, Montana, United States
[job_card.part_time]
[last_updated.last_updated_variable_days] • [promoted]