LLM Evaluation EngineerThe Fountain Group • Mountain View, CA
LLM Evaluation Engineer
The Fountain Group • Mountain View, CA
[job_card.30_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]3+ years of experience in machine learning evaluation, data analysis, or related technical roles. Intermediate to advanced Python scripting, including log parsing and API testing. Familiarity with GenAI and LLMs, including automated workflows and API integrations. Strong analytical mindset, capable of working independently and identifying innovative solutions. Excellent communication skills, able to present complex findings clearly to both technical and non-technical stakeholders.
About the Role :
We are seeking a LLM Evaluation Engineer to join a forward-thinking team responsible for developing a sophisticated voice assistant platform. This isn’t your typical QA role – it’s a unique blend of technical engineering, machine learning evaluation, and data analysis. You’ll work closely with cutting-edge conversational AI technology, designing evaluation frameworks, building custom scripts, and creating data visualizations to assess platform performance.
Key Responsibilities :
- Design and implement evaluation strategies for voice and language models, including automated testing approaches.
- Analyze unstructured data from log store systems to identify performance gaps and optimize user experiences.
- Build and maintain custom Python scripts to streamline data processing and generate actionable insights.
- Develop visual reports to communicate findings and drive continuous improvement.
- Collaborate with cross-functional teams globally to identify and address pain points in conversational AI performance.
- Use prompt engineering techniques to refine LLM outputs and articulate system health.
Ideal Candidate :
[job_alerts.create_a_job]
Engineer • Mountain View, CA
[internal_linking.similar_jobs]
A leading tech company in Cupertino is seeking a Machine Learning Research Engineer to enhance Siri's capabilities through innovative AI solutions.
This role involves developing systems for syntheti...[show_more]
Reliability Engineers at Lunar Energy will be responsible for ensuring product reliability throughout the entire lifecycle of our revolutionary home energy products.
This includes providing input du...[show_more]
A leading AI technology company based in Palo Alto, CA is seeking a Machine Learning Data Engineer.You will build and optimize the data infrastructure for our machine learning systems while collabo...[show_more]
About the Team : Site Reliability Engineering (SRE) of the AML (Applied Machine Learning) team combines system engineering and the art of machine learning to develop and run a massively distributed ...[show_more]
The Machine Learning (ML) System sub-team combines system engineering and the art of machine learning to develop and maintain massively distributed ML training and Inference system / services around ...[show_more]
PsiQuantum'smission is to build the first useful quantum computers-machines capable of delivering the breakthroughs the field has long promised.
Since our founding in 2016, our singular focus has be...[show_more]
An innovative AI company in San Jose is seeking a skilled Machine Learning Engineer with expertise in developing LLMs and VLMs.
The ideal candidate will have a strong education background and proven...[show_more]
Board Level debugging and bring-up skills.Familiar with Linux environment and basic python and shell script development.Hardware functional and Diagnostic Test.
Failure Analysis - Hardware Schematic...[show_more]
Establish and maintain controls and document procedures related to NPI product quality and reliability.Aid Development and Production teams in introducing new products.
Define and execute initiative...[show_more]
We are a well established semiconductor company focused in manufacturing SSD chips for globally recognized semiconductor companies.
With offices in the US, Europe, China and Singapore we are looking...[show_more]
Supermicro is a Top Tier provider of advanced server, storage, and networking solutions for Data Center, Cloud Computing, Enterprise IT, Hadoop / Big Data, Hyperscale, HPC and IoT / Embedded customers...[show_more]
At TUV SUD we are passionate about technology.Innovations impact our daily lives in countless ways, and we are dedicated to being a part of that progress.
We test, we audit, we inspect, we advise.We...[show_more]
A tech company specializing in AI is seeking a Machine Learning Engineer to enhance its platform by developing and deploying generative AI models.
You will collaborate with cross-functional teams to...[show_more]
A pioneering aerospace company based in Palo Alto is looking for a Reliability Engineer to support the development of innovative electric Vertical Takeoff and Landing (eVTOL) aircraft.You will lead...[show_more]
Staff Machine Learning Engineer, LLM Fine‑Tuning (Verilog / RTL Applications) We are looking for a.LLM‑based solutions for code / RTL workflows.
LLM fine‑tuning, evaluation, and production deployment.Pr...[show_more]
A leading digital experience company in California seeks a qualified AI / ML Developer to design and implement innovative predictive models.
Candidates should have an MS or PhD in a relevant field, ex...[show_more]
Job Title : Principal MLOps Engineer.Job Type : - Contract - 12+ Months.Department : Data Science / Machine Learning.We are seeking an experienced Principal MLOps Engineer to lead and scale our machin...[show_more]
Imagine what you could do here.At Apple, great new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly.
Bring passion and dedication to your job and ...[show_more]
LLM Research Engineer : RAG & Synthetic Data
Apple Inc. • Cupertino, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Reliability Engineer
Lunar Energy • Mountain View, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
ML Data Engineer : Systems & Retrieval for LLMs
Zyphra Technologies Inc. • Palo Alto, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Site Reliability Engineer - AML Global Recommendation - USDS
Tik Tok • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Senior Site Reliability Engineer, ML System
ByteDance • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Site Reliability Engineer
PsiQuantum • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
ML Engineer : LLMs, VLMs & Reasoning AI | Equity
Tensor • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Board Level Reliability Engineer
Futran Tech Solutions Pvt. Ltd. • Santa Clara, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Reliability Engineer
ClifyX • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
RTL Design Engineer
CyberCoders • San Jose, CA, US
[job_card.full_time]
[last_updated.last_updated_1_day] • [promoted]
Sr. Reliability Engineer (26861)
Supermicro • San Jose, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Field Evaluation Engineer
TUV SUD AMERICA, INC. • Fremont, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Generative AI & LLM Engineer
Corvic • Mountain View, CA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Reliability Engineer – eVTOL Systems RAM (Onsite)
Medium • Palo Alto, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Staff Machine Learning Engineer, LLM Fine‑Tuning (Verilog / RTL Applications)
Highbrow Technology Inc • San Jose, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
ML Engineer — Personalization & Recommendations (Equity Eligible)
Adobe • San Jose, California, United States
[job_card.full_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Prinicipal MlOps Engineer
Redolent • Sunnyvale, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Evaluation & Insights Engineer
Apple • Cupertino, CA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]