Talent.com
LLM Evaluation Engineer
LLM Evaluation EngineerThe Fountain Group • Mountain View, CA
LLM Evaluation Engineer

LLM Evaluation Engineer

The Fountain Group • Mountain View, CA
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]
About the Role:
We are seeking a LLM Evaluation Engineer to join a forward-thinking team responsible for developing a sophisticated voice assistant platform. This isn’t your typical QA role – it’s a unique blend of technical engineering, machine learning evaluation, and data analysis. You’ll work closely with cutting-edge conversational AI technology, designing evaluation frameworks, building custom scripts, and creating data visualizations to assess platform performance.

Key Responsibilities:
  • Design and implement evaluation strategies for voice and language models, including automated testing approaches.
  • Analyze unstructured data from log store systems to identify performance gaps and optimize user experiences.
  • Build and maintain custom Python scripts to streamline data processing and generate actionable insights.
  • Develop visual reports to communicate findings and drive continuous improvement.
  • Collaborate with cross-functional teams globally to identify and address pain points in conversational AI performance.
  • Use prompt engineering techniques to refine LLM outputs and articulate system health.

Ideal Candidate:
  • 3+ years of experience in machine learning evaluation, data analysis, or related technical roles.
  • Intermediate to advanced Python scripting, including log parsing and API testing.
  • Familiarity with GenAI and LLMs, including automated workflows and API integrations.
  • Strong analytical mindset, capable of working independently and identifying innovative solutions.
  • Excellent communication skills, able to present complex findings clearly to both technical and non-technical stakeholders.

[job_alerts.create_a_job]

LLM Evaluation Engineer • Mountain View, CA

[internal_linking.similar_jobs]

ML Engineer: LLMs, VLMs & Reasoning AI | Equity

TensorSan Jose, CA, United States
[job_card.full_time]

An innovative AI company in San Jose is seeking a skilled Machine Learning Engineer with expertise in developing LLMs and VLMs.The ideal candidate will have a strong education background and proven...[internal_linking.show_more]

 • [job_card.promoted]

MT/QE Evaluations Project Manager

OSI EngineeringCupertino, California, US
[job_card.full_time]

A globally leading consumer device company headquartered in Cupertino, CA is looking for the.MT/QE Evaluations Project Manager.Is this the role you are looking for If so read on for more details, a...[internal_linking.show_more]

 • [job_card.promoted]

Foundational ML Researcher / Engineer -- Remote, Equity

Pathway Genomics CorporationPalo Alto, CA, United States
[filters.remote]
[job_card.full_time]

A cutting-edge AI startup is seeking R&D Engineers for groundbreaking work in attention-based machine learning models.Candidates should have a strong research background and experience in model tra...[internal_linking.show_more]

 • [job_card.promoted]

Trade and Industry - Entry Level Training Programs

DreamboundSanta Cruz, California, United States
[job_card.full_time]

Note: This is an educational program, not a job.Successful completion of the program does not guarantee employment but will equip you with valuable skills for the trades and industry job market.Are...[internal_linking.show_more]

 • [job_card.promoted]

Applied AI / ML Engineer

Catalyst LabsMountain View, CA, United States
[job_card.full_time]

Catalyst Labs is a leading talent agency with a specialized vertical in Applied AI, Machine Learning, and Data Science.We stand out as an agency that’s deeply embedded in our clients’ recruitment o...[internal_linking.show_more]

 • [job_card.promoted]

Reliability Engineer _ Ch

American IT SystemsCupertino, CA, United States
[job_card.full_time]
[filters_job_card.quick_apply]

Job Title: Reliability Engineer _ Ch Location: Cupertino, CA - Onsite Key Qualifications: Experience performing reliability testing including Mechanical Stress Tests, Shock/Drop/Vibra...[internal_linking.show_more]

Technology Programs - Entry Level Training Programs

DreamboundSanta Cruz, California, United States
[job_card.full_time]

Note: This is an educational program, not a job.Successful completion of the program does not guarantee employment but will equip you with valuable skills for the technology job market.Are you pass...[internal_linking.show_more]

 • [job_card.promoted]

Senior ML Inference Engineer - Distributed Systems & Equity

ByteDanceSan Jose, CA, United States
[job_card.full_time]

A leading technology company is seeking a Machine Learning Engineer - Inference to design and implement distributed inference infrastructure.Candidates should have at least 7 years of experience in...[internal_linking.show_more]

 • [job_card.promoted]

AI Agents ML Engineer - Production-Ready & Innovative

SylogicSan Jose, CA, United States
[job_card.full_time]

A stealth AI startup in Silicon Valley seeks skilled individuals to develop AI agents for diverse tasks.Ideal candidates will have strong foundations in machine learning and experience in engineeri...[internal_linking.show_more]

 • [job_card.promoted]

Remote ML Systems Researcher for LLMs & Hardware

MatXMountain View, CA, United States
[filters.remote]
[job_card.full_time]

A leading AI technology company in Mountain View is seeking an experienced professional to optimize large language models (LLMs) for their hardware.Responsibilities include building distributed tra...[internal_linking.show_more]

 • [job_card.promoted]

Software Engineer, Scenario Evaluation, Optimus Simulation

TeslaPalo Alto, CA, United States
[job_card.full_time]

Get AI-powered advice on this job and more exclusive features.As a Software Engineer for the Optimus Simulation team, you will be responsible for creating and maintaining the Optimus Bot validation...[internal_linking.show_more]

 • [job_card.promoted]

Senior Software Engineer - ML/LLM Serving

AlldusSan Jose, CA, United States
[job_card.full_time]

Senior Software Engineer - ML/LLM Serving.This range is provided by Alldus.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Direct message the jo...[internal_linking.show_more]

 • [job_card.promoted]

Senior Planner Evaluation Engineer — Hybrid

WaymoMountain View, CA, United States
[job_card.full_time]

A leading autonomous driving technology firm is seeking experienced data-minded software engineers to join their Planner Evaluation team.You will develop signals to measure performance and driving ...[internal_linking.show_more]

 • [job_card.promoted]

ML Eng TL, Behavior Planning - Lead Autonomous Driving

Australian Competition and Consumer CommissionMountain View, CA, United States
[job_card.full_time]

A leading technology company in Mountain View, California, is seeking a Machine Learning Engineering Team Lead.This role involves defining architecture for planning models, mentoring junior enginee...[internal_linking.show_more]

 • [job_card.promoted]

Endocrinologist

CommonSpirit HealthSanta Cruz, CA, US
[job_card.full_time]

Job Summary and Responsibilities.Dignity Health Medical Group - Dominican.Endocrinologist to join our dynamic team in Santa Cruz, CA.Work within a collaborative, physician-led and patient-centered ...[internal_linking.show_more]

 • [job_card.promoted]

ML Engineer: LLM Research & Production

Apple Inc.Cupertino, CA, United States
[job_card.full_time]

A leading technology company in Cupertino seeks a Research Scientist in AI to conduct groundbreaking research in deep learning and large language models.The ideal candidate will develop and fine-tu...[internal_linking.show_more]

 • [job_card.promoted]

Senior ML Serving Engineer for LLMs & Inference

AlldusSan Jose, CA, United States
[job_card.full_time]

A tech company in AI/ML is seeking a Senior Software Engineer specializing in ML Serving to build robust infrastructure for ML models.The ideal candidate has 5+ years of experience in software engi...[internal_linking.show_more]

 • [job_card.promoted]

Senior ML Engineer — Scalable Personalization Pipelines

Adobe Inc.San Jose, CA, United States
[job_card.full_time]

An innovative firm is seeking a Senior Machine Learning Engineer to lead the development of cutting-edge machine learning models that enhance personalized customer experiences.This role offers the ...[internal_linking.show_more]