AI Agent Evaluation AnalystVirtualVocations • Macon, Georgia, United States
AI Agent Evaluation Analyst
VirtualVocations • Macon, Georgia, United States
[job_card.variable_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]
A company is looking for an AI Agent Evaluation Analyst (Freelance).
Key Responsibilities
Review evaluation tasks and scenarios for logic, completeness, and realism
Identify inconsistencies, missing assumptions, or unclear decision points
Help define clear expected behaviors (gold standards) for AI agents
Required Qualifications
Excellent analytical thinking regarding complex systems and logical implications
Familiarity with structured data formats (ability to read JSON / YAML)
Experience with policy evaluation, logic puzzles, or structured scenario design
Background in consulting, academia, or research
Exposure to LLMs, prompt engineering, or AI-generated content
[job_alerts.create_a_job]
Ai Analyst • Macon, Georgia, United States
[internal_linking.similar_jobs]
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
As an hourly paid, fully remote Go Engineer for AI Data Training, you will review AI-generated Go code and explanations or generate your own, evaluate the reasoning quality and step-by-step problem...[show_more]
Provides analyst support for claims research activities including reviewing and researching claims to ensure regulatory requirements are appropriately applied, identifying root-cause of processing ...[show_more] As a remote, hourly paid R Engineer, you will review AI-generated responses and generate high-quality R and data-analysis-focused content, evaluating the reasoning quality and step-by-step problem-...[show_more] As a Senior SQL Engineer, you will work remotely on an hourly paid basis to review AI-generated SQL queries, database designs, and data-processing logic, as well as generate high-quality reference ...[show_more] 
Are you looking to earn extra income from the comfort of your home? We're seeking motivated individuals to explore and test a variety of remote side hustle opportunities featured on FinanceBuzz.Thi...[show_more] As a remote, hourly paid Swift Engineer, you will review AI-generated responses and generate high-quality Swift and iOS-focused content, evaluating the reasoning quality and step-by-step problem-so...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] As a Senior C++ Engineer, you will work remotely on an hourly paid basis to review AI-generated C++ code, systems designs, and technical explanations, as well as generate high-quality reference imp...[show_more] We’re seeking biologists to join our AI training project.In this role, you will review AI-generated responses, evaluate reasoning quality and step-by-step problem-solving, and provide expert feedba...[show_more] As an hourly paid, fully remote Rust Engineer for AI Data Training, you will review AI-generated Rust code and explanations or generate your own, evaluate the reasoning quality and step-by-step pro...[show_more] As a Senior C Engineer, you will work remotely on an hourly paid basis to review AI-generated C code, low-level systems designs, and technical explanations, as well as generate high-quality referen...[show_more] As an hourly paid, fully remote JavaScript Engineer for AI Data Training, you will review complex AI-generated code and explanations or generate new ones, evaluate the reasoning quality and step-by...[show_more] We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] 
We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more] Are you ready to take on a pivotal role in ensuring the highest standards in AI training? We are looking for a Python Quality Assurance Lead to help ensure consistently high standards in our AI dat...[show_more] Are you ready to take on a pivotal role in ensuring the highest standards in AI training? We are looking for a Java Tech Quality Assurance Lead to help ensure consistently high standards in our AI ...[show_more] In this hourly, remote contractor role, you will review AI-generated English-language responses and / or generate expert language content, evaluating reasoning quality and step-by-step edits while pr...[show_more]
Remote Investment Analyst - AI Model Trainer
Data Annotation • Macon, GA, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Remote Go Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Analyst, Claims Research (Remote)
Molina Healthcare • Macon, GA, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Remote R Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Senior SQL Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Side Hustle Evaluator - Flexible Online Gig Work
Finance Buzz • Centerville, Georgia, US
[filters.remote]
[job_card.temporary]
[last_updated.last_updated_30] • [promoted]
Remote Swift Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Financial Advising Specialist - AI Model Trainer ($50-$60 / hour)
Data Annotation • Macon, GA, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Remote Senior C++ Engineer - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Biologist - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Rust Engineer - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Senior C Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote JavaScript Engineer - AI Trainer
SuperAnnotate • Macon, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Investment Analyst - AI Trainer ($50-$60 / hour)
Data Annotation • Macon, GA, United States
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Remote Financial Advising Expert - AI Trainer ($50-$60 / hour)
Data Annotation • Macon, GA, United States
[filters.remote]
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Remote Python AI Data Training QA Lead - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote Java AI Data Training Tech Quality Lead - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]
Remote English Language Expert - AI Trainer
SuperAnnotate • Warner Robins, Georgia, US
[filters.remote]
[job_card.full_time]
[last_updated.last_updated_variable_days]