Job Summary
The Agent Evaluation team is responsible for testing whether AI agents return the correct and expected responses. We build the framework, metrics, and test cases that validate agent behavior, accuracy, and reliability before release. Our goal is to ensure agents perform consistently and meet product and user expectations.Job Description
Job Description:
Preferred skills:
Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.
Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law.
Skills:
CI/CD; Machine Learning (ML); Large Language Models (LLMs); Curious Mindset; AI Agents; Benchmarking; Evaluation Metrics
Salary:
Primary Location Pay Range: $142,651.46 - $213,977.19
Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later.
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits to eligible employees. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality - to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details.
Education
Bachelor's Degree
While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience.
Relevant Work Experience
5-7 Years
Agentic AI Evaluation Engineer • Washington D.C., District of Columbia, United States
Analytica is seeking an AI Engineer to support an enterprise-wide modernization of AI capabilities for a high-profile financial regulatory client.This role centers on sustaining and enhancing AI sy... Show more
Innovative Defense Technologies.IDT), provider of cutting-edge cloud-based integration, automated testing and data analysis for complex, mission-critical systems in the US Department of Defense (DO... Show more
District of Columbia, United States.About the job Founding AI Systems Engineer TS/SCI.Build the AI Backbone of US Intelligence: Founding Infrastructure Engineer.We're a stealth-mode, high-growth AI... Show more
We are looking for an AI Workflow Engineer to design, build, and operationalize AI-enabled workflows that integrate LLMs, retrieval systems, and biomedical data to support scientific and operationa... Show more
A consulting firm in the AI sector is looking for AI Writing Evaluators (Domain Experts) for a retainer-based project.This 2-week role requires strong analytical skills to evaluate AI-generated tex... Show more
We have an exciting opportunity for a Legal Engineer – AI and Automation at Eversheds Sutherland (US) LLP.We are searching for someone who wants to be a valued contributor and member of a talented ... Show more
A leading research organization in Washington, DC is seeking a Research Lead for AI Cyber Testing & Evaluation.You will manage research budgets, lead multidisciplinary teams, and conduct cutting-ed... Show more
Are you passionate about applying hands-on expertise to build and deliver artificial intelligence and machine learning solutions?.Are you looking to tackle a wide range of complex problems in a fie... Show more
We build AI agents that actually work in enterprise environments — not prototypes, not demos.We need engineer's who can own the entire agent stack: a production frontend, a robust backend, a proper... Show more
Recorded Future in Washington, DC is looking for a Senior Software Engineer to drive the development of scalable software systems focused on AI and Large Language Models.The ideal candidate will ha... Show more
Get AI-powered advice on this job and more exclusive features.At Accenture Federal Services, nothing matters more than helping the US federal government make the nation stronger and safer and life ... Show more
Mission Technologies, a division of HII, is seeking a Software Engineer focused on AI integration in Columbia, Maryland.The role involves designing and implementing AI frameworks, developing Java s... Show more
The AI Engineer is responsible for developing and implementing artificial intelligence solutions to solve complex problems.Develop and implement artificial intelligence solutions.Collaborate with t... Show more
Bytoa is seeking a Senior Applied AI Engineer who doesn't just implement AI solutions, but pioneers them.This role is for someone who sees the current state of AI in cyber as primitive and is drive... Show more
Hybrid Onsite: 4 Days onsite per week from Day1.Client is looking for GenAI Developer who has.Need expertise in chatbot technologies and OpenAI GPT models, specifically prompt engineering, Retrieva... Show more
Innovative Defense Technologies.IDT), provider of cutting-edge cloud-based integration, automated testing, and data analysis for complex, mission-critical systems in the U.Department of Defense (Do... Show more
A leading AI technology firm in Washington DC is seeking a Machine Learning Engineer to design automated evaluation pipelines.The role involves working with advanced AI systems in government settin... Show more
This role is contingent upon award***    Corner Alliance is a dynamic, growing consulting firm that devotes itself to providing an enriching employee experience while working in meaningfu... Show more
Full Time | On Site | Contingent Upon Award.Join Synertex LLC and bring your expertise to a mission that matters.We're looking for an Artificial Intelligence Technical Advisor to support a fas... Show more
Get AI-powered advice on this job and more exclusive features.This range is provided by Iceberg.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.... Show more
){kind=logo}
