Talent.com
Senior Research Scientist, Model Evaluation
Senior Research Scientist, Model EvaluationCohere • San Francisco, CA, United States
Senior Research Scientist, Model Evaluation

Senior Research Scientist, Model Evaluation

Cohere • San Francisco, CA, United States
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Senior Research Scientist, Model Evaluation

Join to apply for the Senior Research Scientist, Model Evaluation role at Cohere .

Why this role?

Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating next?generation evaluation methods and infrastructure to measure LLM progress.

As a Senior Research Scientist, Model Evaluation, You Will

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
  • Work on highly cross?functional teams to translate model feedback into trustworthy, repeatable evaluations.
  • Conduct research to advance the state?of?the?art in LLM evaluation methods, including training LLM judges; refining LLM?based data synthesis pipelines; and improving evaluation efficiency.
  • Build scalable and reusable tools for digging into model performance.

You May Be a Good Fit If

  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
  • You have strong software engineering skills.
  • We value and celebrate diversity

    We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.

    Full?Time Employees At Cohere Enjoy These Perks

  • ???? An open and inclusive culture and work environment
  • ???????? Work closely with a team on the cutting edge of AI research
  • ???? Weekly lunch stipend, in?office lunches & snacks
  • ???? Full health and dental benefits, including a separate budget to take care of your mental health
  • ???? 100% Parental Leave top?up for up to 6 months
  • ???? Personal enrichment benefits towards arts and culture, fitness and well?being, quality time, and workspace improvement
  • ???? Remote?flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co?working stipend
  • ?? 6 weeks of vacation (30 working days!)
  • Seniority level

    Mid?Senior level

    Employment type

    Full?time

    Job function

    Other

    Industries

    Software Development

    #J-18808-Ljbffr

    [job_alerts.create_a_job]

    Senior Research Scientist Model Evaluation • San Francisco, CA, United States

    [internal_linking.similar_jobs]
    Senior Reward Modeling Scientist - RLHF & LLM Evaluation

    Senior Reward Modeling Scientist - RLHF & LLM Evaluation

    Anthropic • San Francisco, CA, United States
    [job_card.full_time]
    A leading AI research organization in San Francisco seeks a Senior Research Scientist to enhance reward modeling methods. This position focuses on innovating architectures and methodologies in RLHF ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Applied Research Scientist - Foundation Models

    Senior Applied Research Scientist - Foundation Models

    Ambient AI • Redwood City, CA, United States
    [job_card.full_time]
    Build a safer world with us, one incident at a time.AI-powered physical security platform helping the world's leading enterprises reduce risk, improve operational efficiency, and gain critical insi...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Research Scientist (Generative Modeling)

    Research Scientist (Generative Modeling)

    WORLD LABS • San Francisco, CA, United States
    [job_card.full_time]
    We build foundational world models that can perceive, generate, reason, and interact with the 3D world - unlocking AI's full potential through spatial intelligence by transforming seeing into doing...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    ML Research Scientist (Health & Sensing)

    ML Research Scientist (Health & Sensing)

    Eight Sleep • San Francisco, CA, United States
    [job_card.full_time]
    Join the Sleep Fitness Movement.At Eight Sleep, we're on a mission to fuel human potential through optimal sleep.As the world's first sleep fitness company, we're redefining what it means to be wel...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Applied Research Scientist - Foundation Models

    Senior Applied Research Scientist - Foundation Models

    Ambient • San Francisco, CA, United States
    [job_card.full_time]
    Senior Applied Research Scientist - Foundation Models.Senior Applied Research Scientist - Foundation Models.Get AI-powered advice on this job and more exclusive features. AI-powered physical securit...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior Research Scientist

    Senior Research Scientist

    Sciforium • San Francisco, CA, United States
    [job_card.full_time]
    Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency serving platform. Backed by multi-million-dollar funding and direct spons...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Research Scientist

    Senior Research Scientist

    Strativ Group • San Francisco, CA, United States
    [job_card.full_time]
    Were partnered with an early-stage AI startup founded by.FAIR / frontier research leaders.This is not a bio company hiring AI talent. AI is applied to drug discovery.This role is for researchers who...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Senior Research Engineer, LLM Evaluation and Behavioral Analysis

    Together AI • San Francisco, CA, United States
    [job_card.full_time]
    Senior Research Engineer, LLM Evaluation and Behavioral Analysis.Together AI is building the fastest, most capable open‑source‑aligned LLMs and inference stack in the world.As part of the Turbo org...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Research Scientist, Foundation Model (LLM / VLM)

    Senior Research Scientist, Foundation Model (LLM / VLM)

    Waymo • San Francisco, CA, United States
    [job_card.full_time]
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Research Scientist

    Senior Research Scientist

    Adobe • San Francisco, CA, United States
    [job_card.full_time]
    Changing the world through digital experiences is what Adobe's all about.We give everyone-from emerging artists to global brands-everything they need to design and deliver exceptional digital exper...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Research Scientist, Model Architectures

    Research Scientist, Model Architectures

    Zyphra Technologies Inc • San Francisco, CA, United States
    [job_card.full_time]
    Research Scientist, Model Architectures.Zyphra's AI Architecture Research Team.This will involve designing and rigorously testing novel model architectures and training methodologies, with a focus ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Research Scientist

    Senior Research Scientist

    HeartFlow • San Francisco, CA, United States
    [job_card.full_time]
    Heartflow is a medical technology company advancing the diagnosis and management of coronary artery disease, the #1 cause of death worldwide, using cutting-edge technology.The flagship product-an A...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Research Scientist - Embeddings & IR Innovation

    Senior AI Research Scientist - Embeddings & IR Innovation

    MongoDB • Berkeley, CA, United States
    [job_card.full_time]
    A technology company located in Berkeley, California, is seeking a Senior Research Scientist to develop next-generation AI models. This role involves leading innovative research towards AI model dep...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Research Scientist : AI Evaluation & Alignment

    Research Scientist : AI Evaluation & Alignment

    Patronus AI • San Francisco, CA, United States
    [job_card.full_time]
    A leading AI company is seeking a Research Scientist to develop cutting-edge AI evaluation systems and conduct transformative research in language models. Candidates should have a background in empi...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Research Scientist, AI (Modeling Focus)

    Research Scientist, AI (Modeling Focus)

    Evolutionary Scale • Redwood City, CA, United States
    [job_card.full_time]
    New York, NY / / Redwood City, CA, United States.EvolutionaryScale, now part of CZI Biohub, is pioneering the next era of AI-powered biology to understand and program life at scale.Together, were co...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Research Scientist

    Senior AI Research Scientist

    Workato • San Francisco, CA, United States
    [job_card.full_time]
    Workato transforms technology complexity into business opportunity.As the leader in enterprise orchestration, Workato helps businesses globally streamline operations by connecting data, processes, ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Research Scientist

    Research Scientist

    techire ai • San Francisco, CA, United States
    [job_card.full_time]
    Interested in advancing how agents and LLMs learn from feedback in realistic environments?.You'll be joining a research-driven AI company building reinforcement learning simulation environments for...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    ML Research Scientist - Quantum Accelerated Generative Models

    ML Research Scientist - Quantum Accelerated Generative Models

    Sygaldry • San Francisco, CA, United States
    [job_card.full_time]
    Sygaldry Technologies is building quantum-accelerated AI servers to exponentially speed up training and inference for AI. By integrating quantum and AI, we're accelerating the path to superintellige...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]