Talent.com
LLM Evaluation Engineer
LLM Evaluation EngineerThe Fountain Group • Mountain View, CA
LLM Evaluation Engineer

LLM Evaluation Engineer

The Fountain Group • Mountain View, CA
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]
About the Role:
We are seeking a LLM Evaluation Engineer to join a forward-thinking team responsible for developing a sophisticated voice assistant platform. This isn’t your typical QA role – it’s a unique blend of technical engineering, machine learning evaluation, and data analysis. You’ll work closely with cutting-edge conversational AI technology, designing evaluation frameworks, building custom scripts, and creating data visualizations to assess platform performance.

Key Responsibilities:
  • Design and implement evaluation strategies for voice and language models, including automated testing approaches.
  • Analyze unstructured data from log store systems to identify performance gaps and optimize user experiences.
  • Build and maintain custom Python scripts to streamline data processing and generate actionable insights.
  • Develop visual reports to communicate findings and drive continuous improvement.
  • Collaborate with cross-functional teams globally to identify and address pain points in conversational AI performance.
  • Use prompt engineering techniques to refine LLM outputs and articulate system health.

Ideal Candidate:
  • 3+ years of experience in machine learning evaluation, data analysis, or related technical roles.
  • Intermediate to advanced Python scripting, including log parsing and API testing.
  • Familiarity with GenAI and LLMs, including automated workflows and API integrations.
  • Strong analytical mindset, capable of working independently and identifying innovative solutions.
  • Excellent communication skills, able to present complex findings clearly to both technical and non-technical stakeholders.

[job_alerts.create_a_job]

LLM Evaluation Engineer • Mountain View, CA

[internal_linking.similar_jobs]

ML Engineer: LLMs, VLMs & Reasoning AI | Equity

TensorSan Jose, CA, United States
[job_card.full_time]

An innovative AI company in San Jose is seeking a skilled Machine Learning Engineer with expertise in developing LLMs and VLMs.The ideal candidate will have a strong education background and proven...[internal_linking.show_more]

 • [job_card.promoted]

ML Systems Engineer

My3Tech IncSunnyvale, CA, United States
[job_card.full_time]

We're seeking an experienced engineer to build our ML data infrastructure platform.You'll create the systems and tools that enable efficient data preparation, feature engineering, and dataset manag...[internal_linking.show_more]

 • [job_card.promoted]

MT/QE Evaluations Project Manager

OSI EngineeringCupertino, California, US
[job_card.full_time]

A globally leading consumer device company headquartered in Cupertino, CA is looking for the.MT/QE Evaluations Project Manager.Is this the role you are looking for If so read on for more details, a...[internal_linking.show_more]

 • [job_card.promoted]

AIML - Sr. Software Development Engineer, Evaluation

AppleCupertino, CA, United States
[job_card.full_time]

At Apple, we create world-class innovative products that seamlessly combine cutting-edge hardware with intelligent software experiences, powered by advanced machine learning technologies.The Evalua...[internal_linking.show_more]

 • [job_card.promoted]

Senior Deep Learning Engineer - Model Evaluation & AI Systems

NVIDIASanta Clara, CA, United States
[job_card.full_time]

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years.It’s a unique legacy of innovation that’s fueled by great technology—and amazing people.T...[internal_linking.show_more]

 • [job_card.promoted]

Remote Equity Research Analyst - AI Trainer ($50-$60 per hour)

Data AnnotationSoquel, California
[filters.remote]
[job_card.full_time] +1

DataAnnotation is committed to creating high-quality AI.Join our team to help train the next generation of AI while enjoying the flexibility of remote work and the freedom to set your own schedule....[internal_linking.show_more]

 • [job_card.promoted]

Endocrinologist

CommonSpirit HealthSanta Cruz, US
[job_card.full_time]

Job Summary and Responsibilities.Dignity Health Medical Group – Dominican.Endocrinologist to join our dynamic team in Santa Cruz, CA.Work within a collaborative, physician-led and patient-centered ...[internal_linking.show_more]

 • [job_card.promoted]

Technology Programs - Entry Level Training Programs

DreamboundSanta Cruz, California, United States
[job_card.full_time]

Note: This is an educational program, not a job.Successful completion of the program does not guarantee employment but will equip you with valuable skills for the technology job market.Are you pass...[internal_linking.show_more]

 • [job_card.promoted]

Reliability Engineer

ClifyXSan Jose, CA, United States
[job_card.full_time]

Establish and maintain controls and document procedures related to NPI product quality and reliability.Aid Development and Production teams in introducing new products.Define and execute initiative...[internal_linking.show_more]

 • [job_card.promoted]

Reliability Engineer, Electrical Systems, NA

Vantage Data CentersSanta Clara, California, United States
[job_card.full_time]

Vantage Data Centers powers, cools, protects and connects the technology of the world's well-known hyperscalers, cloud providers and large enterprises.Developing and operating across North America,...[internal_linking.show_more]

 • [job_card.promoted]

ML Engineer, Foundation Model Evaluation

WaymoMountain View, CA, United States
[job_card.full_time]

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...[internal_linking.show_more]

 • [job_card.promoted]

Software Engineer, LLM Infrastructure

OpenReqCupertino, CA, United States
[job_card.full_time]

Etched is building AI chips that are hard-coded for individual model architectures.Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower laten...[internal_linking.show_more]

 • [job_card.promoted]

Principal Applied ML Engineer

Cadence Design SystemsSan Jose, CA, United States
[job_card.full_time]

At Cadence, we hire and develop leaders and innovators who want to make an impact on the world of technology.Chips are at the center of today's tech-driven world.But how we design and verify them h...[internal_linking.show_more]

 • [job_card.promoted]

ML Engineer

Syntricate TechnologiesCupertino, CA, United States
[job_card.full_time]

Location: Cupertino, CA (Onsite).Client Engineering with experince in NLP.Expereince in deploying Client models.Strong understanding of machine learning principles, especially in the context of LLM...[internal_linking.show_more]

 • [job_card.promoted]

ML Engineer

Catalyst Labs, LLCSunnyvale, CA, United States
[job_card.full_time]

Is a rapidly growing Tier 1 VC backed startup based in New York with $60 million in funding revolutionizing how outside sales and service teams work.Their AI technology captures and analyzes real-w...[internal_linking.show_more]

 • [job_card.promoted]

Senior ML Serving Engineer for LLMs & Inference

AlldusSan Jose, CA, United States
[job_card.full_time]

A tech company in AI/ML is seeking a Senior Software Engineer specializing in ML Serving to build robust infrastructure for ML models.The ideal candidate has 5+ years of experience in software engi...[internal_linking.show_more]

 • [job_card.promoted]

Machine Learning Engineer - GenAI, LLM, Agentic AI

Eightfold LLCSanta Clara, CA, United States
[job_card.full_time]

Eightfold is a global leader in AI-native enterprise talent platform, trusted by the world’s largest & most respected fortune 500 organizations.Our platform is built from the ground up operating at...[internal_linking.show_more]

 • [job_card.promoted]

Principal LLM Application Engineer

AllyNd PartnersPalo Alto, CA, United States
[job_card.full_time]

About the job Principal LLM Application Engineer.AllyNd's client is driving SOC transformation with its unique application of AI computing, initially focusing on generative AI-powered proactive thr...[internal_linking.show_more]