Talent.com
Member of Technical Staff, Model Evaluation
Member of Technical Staff, Model EvaluationxAI • Palo Alto, CA, US
Member of Technical Staff, Model Evaluation

Member of Technical Staff, Model Evaluation

xAI • Palo Alto, CA, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Job Description

Job Description

About xAI

xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company's mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All engineers are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates.

Tech Stack

  • Python
  • JAX and XLA
  • Rust / C++
  • Spark

Focus

  • Provide complete assessment of models.
  • Deep dive into model training and data to identify the weakness point revealed in evaluation.
  • Communicate with modeling and data team to come up with plans to improve model quality.
  • Ideal Experience

  • Model assessment and evaluation task development (including public and in-house benchmarking).
  • Collect data and synthesize data for new evals.
  • Build infrastructure and framework for easy-to-use model evaluation, familiarity with inference frameworks like SGlang and vLLM.
  • Location

    The role is based in the Bay Area [San Francisco and Palo Alto]. Candidates are expected to be located near the Bay Area or open to relocation.

    Annual Salary Range

    $180,000 - $440,000 USD

    Benefits

    Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

    xAI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.

    [job_alerts.create_a_job]

    Member Of Technical Staff • Palo Alto, CA, US

    [internal_linking.similar_jobs]
    Member of Technical Staff - Machine Learning

    Member of Technical Staff - Machine Learning

    Quantix Search • Santa Clara, CA, US
    [job_card.full_time]
    Member of Technical Staff – Machine Learning.San Francisco | Hybrid, 3 days / week | $200K – $280K + equity.I’m partnering with a rapidly scaling healthtech startup that has just ra...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Prompt and Benchmark Engineer, Evaluation of World Models

    Senior Prompt and Benchmark Engineer, Evaluation of World Models

    NVIDIA • Santa Clara, CA, United States
    [job_card.full_time]
    At NVIDIA, we're not just building the future, we're generating it! Our Cosmos generative AI engineering team is pushing the boundaries of what’s possible across multimodal learning, video generati...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Analog / Mixed-Signal Behavioral Modeling Engineer

    Analog / Mixed-Signal Behavioral Modeling Engineer

    NR Consulting • Fremont, CA, United States
    [job_card.full_time]
    Title : Analog / Mixed-Signal Behavioral Modeling Engineer.Write behavioral models for custom analog and mixed-signal circuits in SystemVerilog HDL. Collaborate with circuit design teams to understand ...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Member of Technical Staff, Safety

    Member of Technical Staff, Safety

    x.ai • Palo Alto, CA, United States
    [job_card.full_time]
    AI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excelle...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff ML Engineer, Cross‑Team Recommendations

    Staff ML Engineer, Cross‑Team Recommendations

    Pinterest • Palo Alto, CA, United States
    [job_card.full_time]
    A leading visual discovery platform is seeking a highly motivated Staff ML Engineer to work as a cross-team technical leader. This role involves innovating on large-scale machine learning recommenda...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Member of Technical Staff Epi Process Development Engineer

    Member of Technical Staff Epi Process Development Engineer

    KYOCERA SLD Laser, Inc • Fremont, CA, United States
    [job_card.full_time]
    SLD Laser, a world leader in commercialization of gallium nitride (GaN) based laser light sources, was acquired by KYOCERA Corporation and has commenced operations as a wholly owned subsidiary and ...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior Technical Staff Engineer - Design for Test

    Senior Technical Staff Engineer - Design for Test

    microchip • San Jose, CA, United States
    [job_card.full_time]
    Are you looking for a unique opportunity to be a part of something great? Want to join a 20,000-member team that works on the technology that powers the world around us? Looking for an atmosphere o...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Staff Software Engineer, Perception Evaluation

    Staff Software Engineer, Perception Evaluation

    Waymo • Mountain View, CA, United States
    [job_card.full_time]
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied t...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Engineer

    Staff Engineer

    Miyamoto International • Pleasanton, CA, United States
    [job_card.full_time]
    Miyamoto International is in business to make the world a better, safer place.We provide expert high-performance earthquake + structural engineering and disaster mitigation services, as well as res...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Staff Reliability Engineer

    Staff Reliability Engineer

    Bloom Energy • San Jose, CA, United States
    [job_card.full_time]
    At Bloom Energy, our vision for a world powered by clean, reliable, and affordable energy is more than just a dream-we're making it reality. For over two decades, we've been at the forefront of the ...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Field Evaluation Engineer

    Field Evaluation Engineer

    TUV SUD AMERICA, INC. • Fremont, CA, United States
    [job_card.full_time]
    At TUV SUD we are passionate about technology.Innovations impact our daily lives in countless ways, and we are dedicated to being a part of that progress. We test, we audit, we inspect, we advise.We...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Staff Systems Engineer

    Staff Systems Engineer

    Bio-Rad Laboratories • Pleasanton, CA, United States
    [job_card.full_time]
    Working within Bio-Rad's Life Science R&D Group as a Systems Engineer, you will take engineering concepts, requirements and transform them into functional prototypes and finished products that impr...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Staff Machine Learning Engineer

    Staff Machine Learning Engineer

    Uber • Sunnyvale, CA, United States
    [job_card.full_time]
    Applied AI is a horizontal AI team at Uber partnering with product and platform teams across the company to deliver cutting‑edge machine learning solutions for core business problems.The Computer V...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff Integration Engineer

    Staff Integration Engineer

    PsiQuantum • Palo Alto, CA, United States
    [job_card.full_time]
    PsiQuantum'smission is to build the first useful quantum computers-machines capable of delivering the breakthroughs the field has long promised. Since our founding in 2016, our singular focus has be...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    MEMS Development Engineer - Staff

    MEMS Development Engineer - Staff

    InvenSense • San Jose, CA, United States
    [job_card.full_time]
    The MEMS Development Engineer will be part of MEMS Team and responsible for development and debug of new designs through development and production phases. This position will require working with mu...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Member of Technical Staff Fullstack Engineer

    Member of Technical Staff Fullstack Engineer

    Inflection Ai • Palo Alto, California, United States
    [job_card.full_time]
    At Inflection AI, our public benefit mission is to harness the power of AI to improve human well-being and productivity.The next era of AI will be defined by agents we trust to act on our behalf.We...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Staff MLOps Engineer

    Staff MLOps Engineer

    Grindr • Palo Alto, CA, United States
    [job_card.full_time]
    This is a hybrid role based in our Bay Area (SF or Palo Alto) or our Chicago offices and will require you to be in office Tuesdays and Thursdays. What’s so interesting about this role?.We at Grindr ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Evaluation & Insights Engineer

    Evaluation & Insights Engineer

    Apple • Cupertino, CA, United States
    [job_card.full_time]
    Imagine what you could do here.At Apple, great new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly. Bring passion and dedication to your job and ...[show_more]
    [last_updated.last_updated_1_day] • [promoted]