Talent.com
Senior Research Scientist, Model Evaluation
Senior Research Scientist, Model EvaluationCohere • San Francisco, CA, United States
Senior Research Scientist, Model Evaluation

Senior Research Scientist, Model Evaluation

Cohere • San Francisco, CA, United States
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Senior Research Scientist, Model Evaluation

Join to apply for the Senior Research Scientist, Model Evaluation role at Cohere .

Why this role?

Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating next?generation evaluation methods and infrastructure to measure LLM progress.

As a Senior Research Scientist, Model Evaluation, You Will

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
  • Work on highly cross?functional teams to translate model feedback into trustworthy, repeatable evaluations.
  • Conduct research to advance the state?of?the?art in LLM evaluation methods, including training LLM judges; refining LLM?based data synthesis pipelines; and improving evaluation efficiency.
  • Build scalable and reusable tools for digging into model performance.

You May Be a Good Fit If

  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
  • You have strong software engineering skills.
  • We value and celebrate diversity

    We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.

    Full?Time Employees At Cohere Enjoy These Perks

  • ???? An open and inclusive culture and work environment
  • ???????? Work closely with a team on the cutting edge of AI research
  • ???? Weekly lunch stipend, in?office lunches & snacks
  • ???? Full health and dental benefits, including a separate budget to take care of your mental health
  • ???? 100% Parental Leave top?up for up to 6 months
  • ???? Personal enrichment benefits towards arts and culture, fitness and well?being, quality time, and workspace improvement
  • ???? Remote?flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co?working stipend
  • ?? 6 weeks of vacation (30 working days!)
  • Seniority level

    Mid?Senior level

    Employment type

    Full?time

    Job function

    Other

    Industries

    Software Development

    #J-18808-Ljbffr

    [job_alerts.create_a_job]

    Senior Research Scientist Model Evaluation • San Francisco, CA, United States

    [internal_linking.similar_jobs]
    Research Scientist

    Research Scientist

    University of California, San Francisco • San Francisco, CA, United States
    [job_card.full_time]
    Be among the first 25 applicants.THE DEPARTMENT OF MEDICINE AT THE UNIVERSITY OF CALIFORNIA SAN FRANCISCO (UCSF) is recruiting for the position of Research Scientist in the Division of Geriatrics.T...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Together AI • San Francisco, CA, United States
    [job_card.full_time]
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis.Together AI is building the fastest, most capable open‑source‑aligned LLMs and inference stack in the world.As part of the Turbo org...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Research Scientist (LLMs)

    Senior Research Scientist (LLMs)

    Aldea Inc • San Francisco, CA, United States
    [job_card.full_time]
    Aldea is a multi-modal foundational AI company reimagining the scaling laws of intelligence.We believe today's architectures create unnecessary bottlenecks for the evolution of software.Our mission...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Applied Research Scientist, Perception

    Senior Applied Research Scientist, Perception

    Waymo • San Francisco, CA, United States
    [job_card.full_time]
    Waymo is an autonomous driving technology company with the mission to be the world’s most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior GenAI Research Scientist - Scaling

    Senior GenAI Research Scientist - Scaling

    Databricks • San Francisco, CA, United States
    [job_card.full_time]
    Senior GenAI Research Scientist – Scaling.At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development.W...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Reward Models Scientist - RLHF

    Senior Reward Models Scientist - RLHF

    Anthropic • San Francisco, CA, United States
    [job_card.full_time]
    A leading AI research organization in San Francisco is seeking a Senior Research Scientist focused on reward modeling for large language models. The successful candidate will lead innovative researc...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Aging Research Scientist & Mentor in Geriatrics

    Aging Research Scientist & Mentor in Geriatrics

    University of California - San Francisco • San Francisco, CA, United States
    [job_card.full_time]
    A major academic institution in San Francisco is seeking a Research Scientist in the Division of Geriatrics.Candidates must hold an MD and / or PhD, possess ABIM certification or eligibility in Geria...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Scientist — Multimodal Bio Foundation Models

    Senior ML Scientist — Multimodal Bio Foundation Models

    ESRhealthcare • San Francisco, CA, United States
    [job_card.full_time]
    A biotechnology firm in South San Francisco is looking for a Senior Machine Learning Scientist to lead the design of advanced foundation models integrating biological data.The ideal candidate will ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    (Senior) Applied Machine Learning Scientist

    (Senior) Applied Machine Learning Scientist

    insitro • San Francisco, CA, United States
    [job_card.full_time]
    Machine learning lies at the core of insitro’s approach to rethinking drug development.We’ve built a ChemML Platform that uses AI to accelerate small molecule discovery from hit-finding through lea...[show_more]
    [last_updated.last_updated_30] • [promoted]
    ML Research Scientist

    ML Research Scientist

    Toposbio • San Francisco, CA, United States
    [job_card.full_time]
    Topos Bio is developing computational methods to drug intrinsically disordered proteins.We're seeking a research scientist to develop and optimize ML methods that bridge simulation, protein modelin...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Research Scientist, Reward Models

    Senior Research Scientist, Reward Models

    Anthropic • San Francisco, CA, United States
    [job_card.full_time]
    Remote-Friendly (Travel Required) | San Francisco, CA.Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for s...[show_more]
    [last_updated.last_updated_30] • [promoted]
    GenAI Evaluation Scientist Enterprise LLM Systems

    GenAI Evaluation Scientist Enterprise LLM Systems

    Scale AI • San Francisco, CA, United States
    [job_card.full_time]
    A leading AI technology company is seeking an AI Research Engineer to join their Enterprise Evaluations team.In this critical role, you will enhance evaluation systems for LLM-powered workflows.Can...[show_more]
    [last_updated.last_updated_30] • [promoted]
    ML Research Scientist, Foundation Models (Senior / Staff / Principal)

    ML Research Scientist, Foundation Models (Senior / Staff / Principal)

    Genesis Therapeutics • Burlingame, CA, United States
    [job_card.full_time]
    ML Research Scientist, Foundation Models (Senior / Staff / Principal).ML Research Scientist, Foundation Models (Senior / Staff / Principal). Get AI-powered advice on this job and more exclusive feat...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer - Model Evaluations, Public Sector

    Senior Machine Learning Engineer - Model Evaluations, Public Sector

    Scale AI • San Francisco, CA, United States
    [job_card.full_time]
    Machine Learning Engineer - Model Evaluations, Public Sector.Louis, MO; New York, NY; Washington, DC.The Public Sector ML team at Scale deploys advanced AI systems—including LLMs, agentic models, a...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior / Staff Research Scientist

    Senior / Staff Research Scientist

    Snorkel AI • San Francisco, CA, United States
    [job_card.full_time]
    Employer Industry : Artificial Intelligence Solutions.Why consider this job opportunity : .Opportunity for career advancement and growth within a rapidly scaling organization.Flexibility to work remot...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Research Scientist (diffusion)

    Research Scientist (diffusion)

    Genmo • San Francisco, CA, United States
    [job_card.full_time]
    We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the bo...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Research Scientist

    Research Scientist

    University of California - San Francisco • San Francisco, CA, United States
    [job_card.full_time]
    Wednesday, Dec 31, 2025 at 11 : 59pm (Pacific Time).Apply by this date to ensure full consideration by the committee.Saturday, Oct 10, 2026 at 11 : 59pm (Pacific Time). Applications will continue to be ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Research Scientist

    Research Scientist

    Martian • San Francisco, CA, United States
    [job_card.full_time]
    As a research scientist with Martian, you will develop new techniques to understand how AI models work.This work will focus on exploring and improving a technique we call “model mapping” : convertin...[show_more]
    [last_updated.last_updated_30] • [promoted]