Talent.com
AI Model Evaluator
AI Model EvaluatorScale AI • Austin, Texas, United States
[error_messages.no_longer_accepting]
AI Model Evaluator

AI Model Evaluator

Scale AI • Austin, Texas, United States
[job_card.variable_hours_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Join a global community of talented professionals to shape the future of AI. Earn up to $15 USD / hr and additional rewards based on quality of submission.

Outlier is committed to improving the intelligence & safety of AI models. Owned and operated by Scale AI , we’ve recently been featured in Forbes for partnering experts with top AI labs to provide the high quality data for LLMs. We believe AI can only perform as well as the data it’s trained on. That’s why we work with contributors from all over the world , who help improve AI models by providing expert human feedback . This data has led to AI advancements for the world's leading AI labs and large language model builders.

We’ve built a best-in-class remote work platform for our freelance contributors to provide valuable, specialized skills, and we in turn strive to provide them with a positive experience based on our core pillars of reliability, transparency, and flexibility.

What you will be doing

We are looking for someone who speaks fluent English to contribute their expertise toward training and refining cutting-edge AI systems.

  • Adopt a “user mindset” to produce natural data to meet the realistic needs you have or would use AI for.
  • Use the tool of rubrics to address user needs in a structured way.
  • Evaluate AI outputs by reviewing and ranking reasoning and problem-solving responses from large language models.
  • Contribute across projects depending on your specific skillset and experience.

What we’re looking for

  • Education : Bachelor’s degree or higher (or currently enrolled).
  • Analytical and Problem-Solving Skills : Ability to develop complex, professional-level prompts and evaluate nuanced AI reasoning.
  • Strong Writing : Clear, concise, and engaging writing to explain decisions or critique responses.
  • Attention to Detail : Commitment to accuracy and ability to assess technical aspects of model outputs.
  • Nice to Haves :

  • Experience in fields like literature, creative writing, history, philosophy, theology, etc.
  • Prior writing or editorial experience (content strategist, technical writer, editor, etc.).
  • Interest or background in AI, machine learning, or creative tech tools.
  • Compensation and benefits

    Earn up to $15 USD / hr, paid out weekly

    Rates vary based on quality, accuracy, and time spent. Paid via PayPal & AirTM

    Free access to

    Model Playground

    Interact, experiment and engage with leading large language models free of cost

    Flexible schedule and

    time commitment

    No contracts, no 9-to-5. You control your schedule. (Most experts spend 5-10 hours / week, up to 40 hours working from home

    Join a global community of

    Coding experts

    Join a global network of experts contributing to advanced AI tools

    Disclaimer : For non-core work, such as during initial project onboarding or project overtime phases, lower rates may apply. Certain projects offer incentive payments. Please review the payment terms for each project.

    Equal Opportunity Employer : Outlier is committed to fostering a diverse and inclusive work environment. We welcome applicants from all backgrounds and celebrate diversity in our workforce.

    [job_alerts.create_a_job]

    Model • Austin, Texas, United States

    [internal_linking.similar_jobs]
    AI Trainer -Remote Writing Evaluator

    AI Trainer -Remote Writing Evaluator

    Outlier • Austin, TX, United States
    [filters.remote]
    [job_card.full_time]
    Earn up to $15 / hour + performance bonuses.Outlier, a platform owned and operated by Scale AI, is looking for.If you're passionate about improving models and excited by the future of AI, this is you...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    AI Trainer -Remote Text Quality Evaluator

    AI Trainer -Remote Text Quality Evaluator

    Outlier • Austin, TX, United States
    [filters.remote]
    [job_card.full_time]
    Earn up to $16 USD / hourly and work remotely and flexibly.Outlier, a platform owned and operated by Scale AI, is looking for. If you're passionate about improving models and excited by the future of ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    AI Agent Evaluation Analyst (Freelance)

    AI Agent Evaluation Analyst (Freelance)

    Mindrift • Austin, TX, US
    [filters.remote]
    [job_card.part_time] +1
    [filters_job_card.quick_apply]
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...[show_more]
    [last_updated.last_updated_variable_days]
    Remote Machine Learning Engineer - AI Trainer ($80-$120 per hour)

    Remote Machine Learning Engineer - AI Trainer ($80-$120 per hour)

    Mercor • Austin, Texas, US
    [filters.remote]
    [job_card.part_time]
    At Mercor, we’re building the talent engine that helps leading labs and research orgs move AI forward.Our latest initiative focuses on benchmarking and improving model performance and training spee...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Principal Machine Learning & AI Engineer

    Principal Machine Learning & AI Engineer

    Quantum Search Partners • Austin, TX, US
    [job_card.full_time]
    Principal Machine Learning & AI Engineer.Fraud Prevention & AML Platform (Series-C).Stock Options (Potential Flexibility). Conduct in-depth research to assess the technical and product feasi...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote Medical Expert - AI Evaluation - AI Trainer ($80-$100 per hour)

    Remote Medical Expert - AI Evaluation - AI Trainer ($80-$100 per hour)

    Mercor • Pflugerville, Texas, US
    [filters.remote]
    [job_card.part_time]
    Mercor is seeking highly qualified • •Medical Experts • • with strong clinical knowledge and excellent analytical skills to support a high-impact AI research initiative in partnership with a leading A...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Remote Cinematic Video Evaluator - AI Trainer ($45-$45 per hour)

    Remote Cinematic Video Evaluator - AI Trainer ($45-$45 per hour)

    Mercor • Austin, Texas, US
    [filters.remote]
    [job_card.full_time]
    Overview : • • Mercor is seeking highly discerning • •video evaluators • •.Specifically : artistic professionals such as • •video editors, motion graphics designers, producers, animators, cinematographer a...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote PhD-Level Computational Biologist — Design Data-Centric Benchmarks for AI Models - AI Trainer ($65-$85 per hour)

    Remote PhD-Level Computational Biologist — Design Data-Centric Benchmarks for AI Models - AI Trainer ($65-$85 per hour)

    Mercor • Pflugerville, Texas, US
    [filters.remote]
    [job_card.full_time]
    Role Overview • • Mercor is seeking computational biology experts to contribute to a unique project with a top-tier AI research organization. This short-term initiative challenges AI models with hidde...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote M&A Associate - AI Trainer ($50-$60 / hour)

    Remote M&A Associate - AI Trainer ($50-$60 / hour)

    Data Annotation • Pflugerville, Texas
    [filters.remote]
    [job_card.full_time] +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote Machine Learning Researchers(PhD) - AI Trainer ($120-$120 per hour)

    Remote Machine Learning Researchers(PhD) - AI Trainer ($120-$120 per hour)

    Mercor • Austin, Texas, US
    [filters.remote]
    [job_card.full_time]
    Role Overview • • Mercor is driving a leading AI research initiative focused on benchmarking and enhancing model performance across a range of machine learning tasks. We are seeking Machine Learning R...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote Senior Machine Learning Engineer - LLM Evaluation / Task Creations (India Based) - AI Trainer ($21-$21 per hour)

    Remote Senior Machine Learning Engineer - LLM Evaluation / Task Creations (India Based) - AI Trainer ($21-$21 per hour)

    Mercor • Austin, Texas, US
    [filters.remote]
    [job_card.full_time]
    Role Description • • Mercor is hiring on behalf of a leading AI research lab to bring on highly skilled • •Machine Learning Engineers • • with a proven record of building, training, and evaluating high-...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote Investment Analyst – AI Trainer ($50-$60 / hour)

    Remote Investment Analyst – AI Trainer ($50-$60 / hour)

    Data Annotation • Austin, Texas
    [filters.remote]
    [job_card.full_time] +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Remote Audio Generalist Evaluator Expert - AI Trainer ($35-$40 per hour)

    Remote Audio Generalist Evaluator Expert - AI Trainer ($35-$40 per hour)

    Mercor • Austin, Texas, US
    [filters.remote]
    [job_card.full_time]
    Mercor is seeking detail-oriented writing experts to contribute to a high-impact audio AI research project with a leading lab. Freelancers will author prompt–golden answer pairs that train and evalu...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)

    Remote FinTech Product Analyst - AI Trainer ($50-$60 / hour)

    Data Annotation • Austin, Texas
    [filters.remote]
    [job_card.full_time] +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Multimodal Content Evaluator (Remote)

    Multimodal Content Evaluator (Remote)

    Scale AI • Austin, Texas, United States
    [filters.remote]
    [job_card.full_time]
    Join a global community of talented professionals to shape the future of AI.Earn up to $15 USD / hr and additional rewards based on quality of submission. Outlier is committed to improving the intelli...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Generative AI Engineer - Austin, TX

    Generative AI Engineer - Austin, TX

    QTech • Austin, TX, United States
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Job Title : Generative AI Engineer Location : Austin, TX Domain : Technology Duration : Long Term Contract Looking for W2 Candidates.No C2C <...[show_more]
    [last_updated.last_updated_variable_days]
    Gen AI Architect (Austin)

    Gen AI Architect (Austin)

    Flexon Technologies Talent360.ai • Austin, TX, US
    [job_card.part_time]
    Location : Sunnyvale, CA or Austin, TX.Machine Learning Implementations.Experience in Structured Data Modelling, NLP, Time Series Modelling. Good Understanding of LLM concepts (RAG, Prompting, Few Sh...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Associate Analyst

    Associate Analyst

    Cynet Systems • Austin, TX, United States
    [job_card.full_time]
    Collaborate with AI / ML teams to guide model enhancements.Ensure accuracy and timeliness of data feeds and quarterly results. Design prompts that challenge the AI model in complex, nuanced ways.Docum...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]