LLM Evaluation EngineerThe Fountain Group • Mountain View, CA
LLM Evaluation Engineer
The Fountain Group • Mountain View, CA
[job_card.30_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]3+ years of experience in machine learning evaluation, data analysis, or related technical roles. Intermediate to advanced Python scripting, including log parsing and API testing. Familiarity with GenAI and LLMs, including automated workflows and API integrations. Strong analytical mindset, capable of working independently and identifying innovative solutions. Excellent communication skills, able to present complex findings clearly to both technical and non-technical stakeholders.
About the Role :
We are seeking a LLM Evaluation Engineer to join a forward-thinking team responsible for developing a sophisticated voice assistant platform. This isn’t your typical QA role – it’s a unique blend of technical engineering, machine learning evaluation, and data analysis. You’ll work closely with cutting-edge conversational AI technology, designing evaluation frameworks, building custom scripts, and creating data visualizations to assess platform performance.
Key Responsibilities :
- Design and implement evaluation strategies for voice and language models, including automated testing approaches.
- Analyze unstructured data from log store systems to identify performance gaps and optimize user experiences.
- Build and maintain custom Python scripts to streamline data processing and generate actionable insights.
- Develop visual reports to communicate findings and drive continuous improvement.
- Collaborate with cross-functional teams globally to identify and address pain points in conversational AI performance.
- Use prompt engineering techniques to refine LLM outputs and articulate system health.
Ideal Candidate :
[job_alerts.create_a_job]
Engineer • Mountain View, CA
[internal_linking.similar_jobs]
A company is looking for a Staff MLOps Engineer - LLMOps.Key Responsibilities Build reusable CI / CD workflows for model training, evaluation, and deployment Automate model versioning, approval wo...[show_more]
A leading AI technology company based in Palo Alto, CA is seeking a Machine Learning Data Engineer.You will build and optimize the data infrastructure for our machine learning systems while collabo...[show_more]
Reliability Engineers at Lunar Energy will be responsible for ensuring product reliability throughout the entire lifecycle of our revolutionary home energy products.
This includes providing input du...[show_more]
A leading tech company in Cupertino is seeking a Machine Learning Research Engineer to enhance Siri's capabilities through innovative AI solutions.
This role involves developing systems for syntheti...[show_more]
Position Title : Product Quality Evaluator.Work Location : Sunnyvale, CA or Omaha, NE (2-3 days a week).This role will support the Product Quality Evaluator (PQE) team.
You'll work on client's core pr...[show_more]
PsiQuantum'smission is to build the first useful quantum computers-machines capable of delivering the breakthroughs the field has long promised.
Since our founding in 2016, our singular focus has be...[show_more]
An innovative AI company in San Jose is seeking a skilled Machine Learning Engineer with expertise in developing LLMs and VLMs.
The ideal candidate will have a strong education background and proven...[show_more]
Board Level debugging and bring-up skills.Familiar with Linux environment and basic python and shell script development.Hardware functional and Diagnostic Test.
Failure Analysis - Hardware Schematic...[show_more]
A leading smart technology company in Santa Clara is seeking a Machine Learning Engineer to focus on LLMs for autonomous vehicles and robotics.
D in Computer Science or related fields, solid backgro...[show_more]
This is a fully remote position open to candidates located in the United States with a strong preference for candidates based on the West Coast, with the ability to work in the Pacific Time Zone.Ap...[show_more]
Supermicro is a Top Tier provider of advanced server, storage, and networking solutions for Data Center, Cloud Computing, Enterprise IT, Hadoop / Big Data, Hyperscale, HPC and IoT / Embedded customers...[show_more]
You will be part of a team that drives systematic and data-driven improvements to the Autonomous Vehicle software by designing, implementing, and maintaining robust processes for evaluation and val...[show_more]
At TUV SUD we are passionate about technology.Innovations impact our daily lives in countless ways, and we are dedicated to being a part of that progress.
We test, we audit, we inspect, we advise.We...[show_more]
A pioneering aerospace company based in Palo Alto is looking for a Reliability Engineer to support the development of innovative electric Vertical Takeoff and Landing (eVTOL) aircraft.You will lead...[show_more]
A leading technology company in Palo Alto is seeking a Machine Learning Engineer to build and deploy models for core products.
Responsibilities include applying modern ML techniques and collaboratin...[show_more]
Staff Machine Learning Engineer, LLM Fine‑Tuning (Verilog / RTL Applications) We are looking for a.LLM‑based solutions for code / RTL workflows.
LLM fine‑tuning, evaluation, and production deployment.Pr...[show_more]
A tech company specializing in AI is seeking a Machine Learning Engineer to enhance its platform by developing and deploying generative AI models.
You will collaborate with cross-functional teams to...[show_more]
TikTok Ads Core ML Team aims at creating automatic delivery products for the next generation and developing advertising as a global business, instead of just a monetization tool to consolidate the ...[show_more]
Staff MLOps Engineer
VirtualVocations • Fremont, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
ML Data Engineer : Systems & Retrieval for LLMs
Zyphra Technologies Inc. • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Reliability Engineer
Lunar Energy • Mountain View, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
LLM Research Engineer : RAG & Synthetic Data
Apple Inc. • Cupertino, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Product Quality Evaluator
Spectraforce Technologies • Sunnyvale, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Site Reliability Engineer
PsiQuantum • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
ML Engineer : LLMs, VLMs & Reasoning AI | Equity
Tensor • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Board Level Reliability Engineer
Futran Tech Solutions Pvt. Ltd. • Santa Clara, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
LLM Engineer for AI, Robotics & Autonomous Driving
XPENG • Santa Clara, CA, United States
[job_card.full_time]
[last_updated.last_updated_less] • [promoted] • [new]
Customer Reliability Engineer
Cisco Systems, Inc. • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Sr. Reliability Engineer (26861)
Supermicro • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Evaluation Systems Engineer, Autonomous Vehicles
General Motors • Mountain View, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Field Evaluation Engineer
TUV SUD AMERICA, INC. • Fremont, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Reliability Engineer – eVTOL Systems RAM (Onsite)
Medium • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Senior ML Engineer (Level 5) — Deploy & Scale Models
Minimal • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Staff Machine Learning Engineer, LLM Fine‑Tuning (Verilog / RTL Applications)
Highbrow Technology Inc • San Jose, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Generative AI & LLM Engineer
Corvic • Mountain View, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Research Engineer (LLM / ML / RL) - TikTok Ads Core ML, Ranking
Tik Tok • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]