Talent.com
Member of Technical Staff, Model Evaluation
Member of Technical Staff, Model EvaluationxAI • Palo Alto, CA, US
Member of Technical Staff, Model Evaluation

Member of Technical Staff, Model Evaluation

xAI • Palo Alto, CA, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Job Description

Job Description

About xAI

xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company's mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All engineers are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates.

Tech Stack

  • Python
  • JAX and XLA
  • Rust / C++
  • Spark

Focus

  • Provide complete assessment of models.
  • Deep dive into model training and data to identify the weakness point revealed in evaluation.
  • Communicate with modeling and data team to come up with plans to improve model quality.
  • Ideal Experience

  • Model assessment and evaluation task development (including public and in-house benchmarking).
  • Collect data and synthesize data for new evals.
  • Build infrastructure and framework for easy-to-use model evaluation, familiarity with inference frameworks like SGlang and vLLM.
  • Location

    The role is based in the Bay Area [San Francisco and Palo Alto]. Candidates are expected to be located near the Bay Area or open to relocation.

    Annual Salary Range

    $180,000 - $440,000 USD

    Benefits

    Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

    xAI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.

    [job_alerts.create_a_job]

    Member Of Technical Staff • Palo Alto, CA, US

    [internal_linking.similar_jobs]
    Tech Engagement Lead - Model Builder

    Tech Engagement Lead - Model Builder

    NVIDIA Corporation • Santa Clara, CA, US
    [job_card.full_time]
    Tech Engagement Lead - Model Builder page is loaded## Tech Engagement Lead - Model Builderlocations : US, CA, Santa Clara : US, CA, Remotetime type : Full timeposted on : Posted Todayjob requis...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Analog / Mixed-Signal Behavioral Modeling Engineer

    Analog / Mixed-Signal Behavioral Modeling Engineer

    NR Consulting • Fremont, CA, United States
    [job_card.full_time]
    Title : Analog / Mixed-Signal Behavioral Modeling Engineer.Write behavioral models for custom analog and mixed-signal circuits in SystemVerilog HDL. Collaborate with circuit design teams to understand ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Machine Learning Engineer

    Staff Machine Learning Engineer

    Cisco Systems, Inc. • San Jose, California, United States
    [job_card.full_time]
    Join the engineering team building theintelligent backbone of Splunk Observability Cloud.We are committed toleveragingthe latest advancements in data science and machine learning to unlock unpreced...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Member of Technical Staff, CUDA / GPU Kernel

    Member of Technical Staff, CUDA / GPU Kernel

    Xai • Palo Alto, CA, United States
    [job_card.full_time]
    AI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excelle...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Member of Technical Staff, Safety

    Member of Technical Staff, Safety

    x.ai • Palo Alto, CA, United States
    [job_card.full_time]
    AI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excelle...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Technical Staff Engineer - Design for Test

    Senior Technical Staff Engineer - Design for Test

    microchip • San Jose, CA, United States
    [job_card.full_time]
    Are you looking for a unique opportunity to be a part of something great? Want to join a 20,000-member team that works on the technology that powers the world around us? Looking for an atmosphere o...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Engineer, Wafer Metrology

    Staff Engineer, Wafer Metrology

    San Jose Staffing • San Jose, CA, US
    [job_card.full_time]
    At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible. At our core, Western Digital is a company o...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Software Engineer, Perception Evaluation

    Staff Software Engineer, Perception Evaluation

    Waymo • Mountain View, CA, United States
    [job_card.full_time]
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied t...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff ML Engineer — Lead Global ML Initiatives

    Staff ML Engineer — Lead Global ML Initiatives

    Minimal • Palo Alto, CA, United States
    [job_card.full_time]
    A leading technology firm in California is looking for an experienced Staff Machine Learning Engineer.In this role, you will drive the technical direction of machine learning technology, design and...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Engineer

    Staff Engineer

    Miyamoto International • Pleasanton, CA, United States
    [job_card.full_time]
    Miyamoto International is in business to make the world a better, safer place.We provide expert high-performance earthquake + structural engineering and disaster mitigation services, as well as res...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Evaluation Consultant

    Evaluation Consultant

    Paradise Architectural Panels and Steel • San Jose, CA, United States
    [job_card.full_time]
    About the job Evaluation Consultant.Paradise Architectural PANELS & STEEL is a leading manufacturer of high-quality architectural panels and steel products. We are committed to providing our clients...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Staff Systems Engineer

    Staff Systems Engineer

    Bio-Rad Laboratories • Pleasanton, CA, United States
    [job_card.full_time]
    Working within Bio-Rad's Life Science R&D Group as a Systems Engineer, you will take engineering concepts, requirements and transform them into functional prototypes and finished products that impr...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Integration Engineer

    Staff Integration Engineer

    PsiQuantum • Palo Alto, CA, United States
    [job_card.full_time]
    PsiQuantum'smission is to build the first useful quantum computers-machines capable of delivering the breakthroughs the field has long promised. Since our founding in 2016, our singular focus has be...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Engineer, Senior Staff

    Engineer, Senior Staff

    ACL Digital • Santa Clara, CA, United States
    [job_card.full_time]
    Expert experience in C++ Programming Language.Hands on experience with System Verilog.Knowledge in AI ML (Artificial Intelligence Machine Learning). Experience with software productization.Documenta...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Systems Engineer

    Staff Systems Engineer

    Intuitive • Sunnyvale, CA, United States
    [job_card.full_time]
    At Intuitive, we are united behind our mission : we believe that minimally invasive care is life-enhancing care.Through ingenuity and intelligent technology, we expand the potential of physicians to...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Engineer

    Staff Engineer

    VirtualVocations • San Jose, California, United States
    [job_card.full_time]
    A company is looking for a Staff Engineer to transform and integrate complex brokerage and financial data across its digital ecosystem. Key Responsibilities Design and implement software solutions...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Member of Technical Staff Fullstack Engineer

    Member of Technical Staff Fullstack Engineer

    Inflection Ai • Palo Alto, California, United States
    [job_card.full_time]
    At Inflection AI, our public benefit mission is to harness the power of AI to improve human well-being and productivity.The next era of AI will be defined by agents we trust to act on our behalf.We...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff MLOps Engineer

    Staff MLOps Engineer

    Grindr • Palo Alto, CA, United States
    [job_card.full_time]
    This is a hybrid role based in our Bay Area (SF or Palo Alto) or our Chicago offices and will require you to be in office Tuesdays and Thursdays. What’s so interesting about this role?.We at Grindr ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]