Content Manager - Remote
TradeJobsWorkForce95828 Florin, CA, USRemote
Full-time
Content Manager Job Duties: Manages all content creation, revision, and retention across multiple ... Show more
Remote Shopify Shop App Agent Annotator - AI Trainer ($40-$50 per hour)Mercor • Rocklin, California, US
No longer accepting applications
Remote Shopify Shop App Agent Annotator - AI Trainer ($40-$50 per hour)
Mercor • Rocklin, California, US
7 days ago
Salary
$40.00 hourly
Job type
- Full-time
- Remote
Job description
## About the role Shopify's **Shop App Agent** is the AI assistant buyers talk to inside the Shop app — product discovery and search, orders, shipping and tracking, account questions, and Shop Cash. When a shopper is unhappy with one of its answers, they tap thumbs-down — and that's all they leave. No comment, no reason, no survey. Your job is to work out *why*, from the transcript alone. ## What you'll do - Read a full shopper ↔ agent conversation, including the tool calls the agent made and what came back - Find the specific turn that received the thumbs-down - Classify it with one primary tag — the *domain* of the complaint: a disliked recommendation, a failed order lookup, an action the agent couldn't take, a rejection of AI itself, or a genuinely broken response — and one specific secondary tag within it - Write a short comment: what the shopper wanted, what the agent did or failed to do, and why those tags fit - **Commitment:** ~5 hours/week ## What makes someone good at this The hardest part is not guessing. Plenty of thumbs-downs have no visible cause — the agent did nothing wrong, or the shopper simply didn't like the answer. There is an explicit `unknown` label for exactly that, and using it honestly matters more than producing a confident-sounding reason. We would rather record "we don't know why" than invent a cause that sends the wrong signal to the model. ## Qualifications **Required:** - **Rule discipline** — you can apply a fixed taxonomy the same way across hundreds of conversations, and you notice when a case sits between two labels rather than forcing it. - **Comfort reading structured traces** — tool calls, their inputs, and their raw outputs, so you can tell what the agent actually did from what it merely claimed. - **Restraint under ambiguity** — you are willing to label something `unknown` rather than reach for a plausible-sounding cause. - **Clear, concise written English** — enough to explain in two or three sentences why a conversation got the labels you gave it. **Strongly preferred:** - **Consumer e-commerce fluency** — orders, tracking, returns, refunds, and a working sense of where a merchant's responsibility ends and the platform's begins. - **Prior annotation, labelling, or model-evaluation work** against a defined rubric or taxonomy. **Preferred (nice to have — we'll ramp you on the specifics):** - Hands-on experience with the Shop app or other AI shopping assistants as a shopper. - Familiarity with customer-support operations and escalation paths. ## What this is not This is not a customer-support role and not an engineering role. You are not fixing the agent and not replying to shoppers — you are diagnosing why a real shopper was unhappy, precisely and repeatably, so the team can measure where the agent falls short.
Create a job alert for this search
Remote Shopify Shop App Agent Annotator - AI Trainer ($40-$50 per hour) • Rocklin, California, US
Similar jobs
Work-From-Home Product Advisor - $45 per hour
Great LionGalt, California$45.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Product Tester – Remote
OCPALincoln, California, US$15.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home in the UK to fulfill upcoming contracts with local and international companies.We guarantee 15-25 hours per week with an hourly pay of between £18/hr.Th... Show more
Test Products from Home – $25-$45/hr + Freebies
OCPALincoln Village, California, us$15.00 hourly
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Remote - Test Products - $25-$45 per hour
OCPALincoln, California, US$25.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home in the UK to fulfill upcoming contracts with local and international companies.We guarantee 15-25 hours per week with an hourly pay of between £18/hr.Th... Show more
Flexible Remote Work – Get Paid to Share Your Opinions on Top Brands
OCPAElk Grove, California, us$15.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Travel Agent
TradeJobsWorkForce95829 Vineyard, CA, USFull-time
Plan details for travel including transportation, accommodation, and airlines.Work with clients to determine their traveling needs.Guide clients on the appropriate styles of transportation, travel ... Show more
Remote Data Entry - Product Support - $45 per hour
GL Inc.Lincoln, California$45.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Get Paid to Test Products at Home – Flexible Hours, Weekly Pay
OCPALincoln Village, California, us$15.00 hourly
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Product Tester at Home - £18/hr-£32/hr - Part-time
OCPALincoln, California, US$15.00 hourly
Part-time +1
Product Testers are wanted to work from home in the UK to fulfill upcoming contracts with local and international companies.We guarantee 15-25 hours per week with an hourly pay of between £18/hr.Th... Show more
Cloud Engineer - AI Trainer
DataAnnotationSacramento, California, United States$104,000.00 yearly
Full-time
Salary: $104,000 - 208,000 per year.We require fluent English communication skills at a native or bilingual level.We require proficiency in at least one programming language or framework, such as J... Show more
Market Research Associate - No Experience
Clubshop | USLincoln, CaliforniaFull-time
Discover The System Surveoo That’s Helping Newbies Earn Daily.Turn your free time into daily earnings by answering simple surveys.No product or tech skills needed.Join For Free And Start Earn... Show more
Remote Product Tester – $45/hr + Free Products – Start Now!
OCPAWheatland, California, us$15.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Travel Agent
TradeJobsWorkforce95823 Sacramento, CA, USFull-time
Travel Agent Job Duties: Meets with clients to determine travel needs, budgets and prefer... Show more
Remote Product Tester
OCPALincoln, California, US$15.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home in the UK to fulfill upcoming contracts with local and international companies.We guarantee 15-25 hours per week with an hourly pay of between £18/hr.Th... Show more
Online Product Testing - $25-$45 per hour
OCPALincoln, California, US$25.00 hourly
Part-time +1
Product Testers are wanted to work from home in the UK to fulfill upcoming contracts with local and international companies.We guarantee 15-25 hours per week with an hourly pay of between £18/hr.Th... Show more
Work From Home Professionals - $45 per hour
GL1Lincoln, California$45.00 hourly
Remote
Part-time +1
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
Remote Product Tester - Flexible Studies
BuzzTestersGalt, CARemote
Full-time
Earn up to $400/week, depending on the number and type of studies you complete.Share honest feedback on products and services from independent brands.Assignments include short online surveys (10–20... Show more
Animator - Remote
TradeJobsWorkForce95832 Sacramento, CA, USRemote
Full-time
Animator Job Duties: Animates realistic, story-driven character actions that are exc... Show more
Conversion Optimization Specialist | Remote | Performance Marketing Agency
Atomic HRWest Sacramento, California, United StatesRemote
Full-time
Quick Apply
Conversion Optimization Specialist.In this role, you’ll own landing page and funnel testing across multiple Shopify-based stores—helping brands scale faster by turning insights into real results.Yo... Show more