Talent.com
Mercor
Remote Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) - AIMercor • Aurora, Colorado, US
Remote Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) - AI

Remote Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) - AI

Mercor • Aurora, Colorado, US
18 hours ago
Salary
$60.00 hourly
Job type
  • Full-time
  • Remote
Job description
  • Role Overview Mercor is running a short, structured evaluation pilot to measure how frontier AI models perform on realistic professional work — and how experienced practitioners judge that work. We are hiring 25 experts across four business domains. Each expert completes one task, then evaluates five model attempts at that same task. This is a one-time study, not ongoing production work. What You'll Do - Sign an NDA before receiving any materials. - Solve one realistic, domain-specific task using the source files provided. - Review five model-generated outputs for that task. Rank them 1–5 with no ties, score each 0–100, and write evidence-based rationales that cite specific parts of the output. - Provide structured feedback on the task itself and on the study design. Time Commitment Up to 13 hours total : 3–10 hours to solve the task, approximately 2 hours to rank and score the five outputs, and approximately 1 hour to provide feedback. Study Conditions - No rubric and no golden answer are provided. Your professional judgment is the measurement. - Use of LLMs or other AI assistants is prohibited at every stage — solving, ranking, scoring, and writing rationales. Domains we are focused on - Admin / Business Operations - Marketing - Human Resources - Accounting Who We're Looking For - 5+ years of hands-on professional experience in one of the four domains above. - Currently or recently practicing, so you can judge the work the way a working professional would. - Able to write a clear, specific, evidence-grounded rationale for a judgment. - Reliable within a short turnaround window. Eligibility Restriction You are not eligible for this pilot if you have worked on Project Alchemy in any capacity — task author, reviewer, world expert, or anyone who has had access to its source world data. Prior exposure to that material would invalidate the study.
Create a job alert for this search

Remote Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) - AI Trainer ($60-$80 per hour) • Aurora, Colorado, US

Similar jobs

Remote Mathematics Expert (PhD) - $100+ per hour

Turing The Pinery, Colorado
Remote
Full-time
Quick Apply

Remote contract for PhDs in Mathematics, Statistics, or related fields.Work on cutting-edge projects with top AI labs while earning $100+/hour, fully remote, with flexible weekly hours.Help fine-tu... Show more

 • Promoted

Work From Home

AmazonFranktown, CO
Remote
Full-time

Job Title: Work From Home Customer Service Representative.As a Work From Home Customer Service Representative at Amazon, you will be the first point of contact for customers seeking assistance.You ... Show more

 • Promoted

Manager, Harmony Equine Center - Animal Welfare Organization

Denver Dumb Friends LeagueFranktown, CO, United States
Full-time

Job TypeFull-timeDescriptionManager, Harmony Equine Center Humane Colorado Franktown, COHumane Colorado is seeking an experienced and mission driven leader to manage the Harmony Equine Center and o... Show more

 • Promoted

Multifamily Housing Asset Manager

Monarch Investment and Management GroupFranktown, CO, United States
Full-time

DescriptionMultifamily Asset ManagerMonarch Investment & Management Group MIMG Franktown, CO(Relocation Assistance Available)Drive performance.At MIMG we don't chase industry standards, we rede... Show more

 • Promoted

Remote Sales Professional - Training Provided

Blue Ridge StaffingKeenesburg, CO, United States
Remote
Full-time

Job DescriptionJob DescriptionTired of traditional jobs that cap your income? At Blue Ridge Staffing, your success is based on your effort.We connect motivated individuals with opportunities to ear... Show more

 • Promoted

Travel Consultant (flexable Hours, Remote)

Roamarae Global TravelBrighton, Colorado, United States
Remote
Full-time
Quick Apply

We are expanding our team and seeking driven individuals to work as Remote Travel Consultants.This role involves helping clients design and book personalized travel experiences while building valua... Show more

Sales & Marketing Representative - Financial Services - Remote Options

Five Rings Financial, LLCKeenesburg, CO, United States
Remote
Full-time +1

Join to apply for theSales & Marketing Representative - Financial Services - Remote Optionsrole atFive Rings Financial, LLCWe are seeking aSales & Marketing Representative - Financial Servi... Show more

 • Promoted

Remote Product Tester - Flexible Studies

BuzzTestersElbert, CO
Remote
Full-time

Earn up to $400/week, depending on the number and type of studies you complete.Share honest feedback on products and services from independent brands.Assignments include short online surveys (10–20... Show more

 • Promoted

Work From Home

Remote JobsElizabeth, CO
Remote
Full-time

Remote Jobs are now available in your area.This position is a work from home position.You will not need to come into the office.Benefits: Health Insurance, 401K, Vacation& PTO.Apply now! Work f... Show more

 • Promoted

DATA ENTRY CLERK - REMOTE WORK FROM HOME - CUSTOMER SERVICE REPRESENTATIVE - ENTRY LEVEL IN ILLINOIS

USA - REMOTE ILLINOISElizabeth, CO, United States
Remote
Part-time

Administrative Assistant Remote & Customer Service Care Rep CCR / Data Entry Clerk - Work From Home Remote - Weekly / Daily PayHello from our family to yours! Take a look at our job and if you'... Show more

 • Promoted

Staff Engagement Manager

Jewish Community Center of DenverElbert, CO, United States
Full-time

The JCC Ranch Camp is not just the Denver JCC's overnight summer camp, it's also one of the best places to work during the summer.As a staff member, Ranch Camp will become your home away from home ... Show more

 • Promoted

Remote Data Entry - Product Support - $45 per hour

GL Inc.The Pinery, Colorado
$45.00 hourly
Remote
Part-time +1

Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more

 • Promoted

Consumer Insights Analyst

Earn HausElizabeth, Colorado, United States
Full-time +1

We are urgently seeking people interested in taking market research studies for well known brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Ear... Show more

 • Promoted

Recruiting Specialist

Americold LogisticsBrighton, CO, United States
Full-time

Human Resources RecruiterSupport full life cycle recruiting efforts to fill high-volume, critical positions across a given region in the organization.Drive strategic recruiting activities to pipeli... Show more

 • Promoted

Market Research Associate - No Experience

Clubshop | USThe Pinery, Colorado
Full-time

Discover The System Surveoo That’s Helping Newbies Earn Daily.Turn your free time into daily earnings by answering simple surveys.No product or tech skills needed.Join For Free And Start Earn... Show more

Consumer Research Associate (Remote)

GL IncThe Pinery, Colorado
$15.00 hourly
Remote
Part-time +1

Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more

 • Promoted

Manager, Intelligence Analyst - Training Manager/Content Developer

RaytheonAurora, CO, United States
Full-time

Manager, Intelligence Analyst - Training Manager/Content CreatorAt Raytheon, the foundation of everything we do is rooted in our values and a higher calling to help our nation and allies defend fre... Show more

 • Promoted

County Meter Reader

Meter ReaderElbert, CO
Full-time

Responsibilities The primary responsibility of this position is to read meters and record consumption of the water used, cleaning of meter boxes, and removal of vegetation impeding access to meters... Show more