Talent.com
Client Service Risk Manager - AI Evaluation and Testing
Client Service Risk Manager - AI Evaluation and TestingMcKinsey & Company • Phoenix
Client Service Risk Manager - AI Evaluation and Testing

Client Service Risk Manager - AI Evaluation and Testing

McKinsey & Company • Phoenix
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

You are someone who thrives in a high-performance environment, bringing a growth mindset and entrepreneurial spirit to tackle meaningful challenges that have a real impact.

In return for your drive, determination, and curiosity, we’ll provide the resources, mentorship, and opportunities to help you quickly broaden your expertise, grow into a well-rounded professional, and contribute to work that truly makes a difference.

When you join us, you will have :

  • Continuous learning :  Our learning and apprenticeship culture, backed by structured programs, is all about helping you grow while creating an environment where feedback is clear, actionable, and focused on your development. The real magic happens when you take the input from others to heart and embrace the fast-paced learning experience, owning your journey.
  • A voice that matters :  From day one, we value your ideas and contributions. You’ll make a tangible impact by offering innovative ideas and practical solutions, all while upholding our unwavering commitment to ethics and integrity. We not only encourage diverse perspectives, but they are critical in driving us toward the best possible outcomes.
  • Global community :  With colleagues across 65+ countries and over 100 different nationalities, our firm’s diversity fuels creativity and helps us come up with the best solutions. Plus, you’ll have the opportunity to learn from exceptional colleagues with diverse backgrounds and experiences.
  • Exceptional benefits :  On top of a competitive salary (based on your location, experience, and skills), we provide a comprehensive benefits package to enable holistic well-being for you and your family.

You will work within the firm's Technology Risk function as the Client Service Risk Manager, of AI Evaluation and Testing, ensuring our firm’s strong ability to evaluate and validate AI systems including those used and built internally and in client contexts by consultants for reliability, safety, security, and regulatory compliance.

You, as the lead in AI evaluation and testing, will define the vision and strategic roadmap for AI control objectives and testing. You will establish frameworks, requirements, and guidance to assess behavior and ensure reliability (including accuracy, bias, and other factors) across various modalities such as text, code, multimodal systems, and agents. Additionally, you will define and prioritize the requirements for tooling solutions that support AI governance, observability, and testing for all relevant AI systems and agents.

You will also develop or validate frameworks and methodologies for conducting red-team attacks on AI systems (jailbreaks, prompt injections, system prompt extraction). You will partner with first line of defense to implement evaluation gates and build a control environment with appropriate technical guardrails to ensure compliance with applicable regulations (e.g., the EU AI Act).

You will report directly to the Associate Director of Client Service Risk, Technology Knowledge & Frameworks, within the firm’s global Risk function. You will partner closely with Legal, Risk teams, Technology and Product teams to develop consistent and scalable RAI control objectives and testing methodologies.

You will support updates to existing robust policies and standards in line with development in AI capabilities (e.g., agentic).

Your collaboration will extend to various roles and functions across the firm, including senior leadership, to whom you will present and gain support for your strategy and program, report progress, and demonstrate impact.

  • U.S. Bachelor’s degree or equivalent work experience required
  • 8+ years in Trust & Safety (AI / ML), risk, or security domains with Data Science background; the scope of the models includes traditional ML models, GenAI and agentic deployments
  • 5+ years managing complex cross-functional projects
  • Familiarity with AI safety, testing and validation, and governance; experience partnering with engineers to build control and integrity systems
  • Knowledge of AI evaluation methods and ability to connect evaluation results to guardrails and controls; Strong knowledge of LLM behavior and jailbreak techniques
  • Proven ability to translate complex ethical challenges into actionable policies and frameworks; strong policy development, risk management, and analytical skills
  • Experience working with engineering and product teams to create tools, solutions, or automation
  • Ability to work in-person (team schedules and office locations may vary)
  • Ability to operate office equipment
  • Excellent leadership and communication skills, with a proven ability to navigate ambiguity and influence product direction in a fast-changing environment
  • Excellent written and oral communication skills, as well as interpersonal skills, including the ability to articulate technical concepts to both technical and non-technical audiences
  • [job_alerts.create_a_job]

    Client Service Risk Manager AI Evaluation and Testing • Phoenix

    [internal_linking.similar_jobs]
    Consulting Actuary - ACA Risk Adjustment

    Consulting Actuary - ACA Risk Adjustment

    Blue Cross and Blue Shield of North Carolina • Phoenix, Arizona, United States
    [job_card.full_time]
    As a Consulting Actuary -ACA Risk Adjustment, you will play a pivotal role in delivering a wide array of actuarial and analytical services for our organization. Responsibilities include leading pric...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Client Services Tech 1

    Client Services Tech 1

    Pace Analytical Services, LLC • Phoenix, Arizona, United States
    [job_card.full_time]
    Monday through Friday, 8 : 30 AM - 5 : 30 PM.Are you ready to work making the world a safer, healthier place? Join our mission to continuously move science forward. to innovate and advance all aspects ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Manager of Insights and Analytics

    Manager of Insights and Analytics

    Beyond Feedback • Scottsdale, Arizona, US
    [job_card.full_time]
    Job Description Job Description At Beyond Feedback we bring humanity back to business by partnering with business leaders to provide employee and customer lifecycle feedback programs.The tailor-m...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Counsel (Privacy and Cybersecurity)

    Counsel (Privacy and Cybersecurity)

    Carvana • Tempe, AZ
    [job_card.full_time]
    [filters_job_card.quick_apply]
    COUNSEL (PRIVACY AND CYBERSECURITY) About Carvana At Carvana, we’re changing the way people buy and sell cars.With an ambitio...[show_more]
    [last_updated.last_updated_variable_days]
    Compliance Manager

    Compliance Manager

    Avesis • Phoenix, Arizona, United States
    [job_card.full_time]
    Join us for an exciting career with the leading provider of supplemental benefits!.Through skill-building, leadership development and philanthropic opportunities, we provide opportunities to build ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    AI Delivery Engineer

    AI Delivery Engineer

    Expedient • Phoenix, AZ
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Join Expedient's AI CTRL product team as an AI Delivery Engineer, where you'll guide enterprise clients from contract signing to successful go-live of our AI platform. You'll manage implementation p...[show_more]
    [last_updated.last_updated_variable_days]
    Director AI

    Director AI

    Honeywell International Inc. • Phoenix, AZ, United States
    [job_card.permanent]
    As a Director of AI here at Honeywell, you will set the strategic direction and ensure the successful delivery of our technology projects. Collaborating closely with senior executives and cross-func...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Equity Compensation Client Service Manager

    Equity Compensation Client Service Manager

    Equiniti • Phoenix, Arizona, United States
    [job_card.full_time]
    This a remote position but candidates must reside in Arizona, Colorado, Nevada or Utah to ensure proximity to our client base and occasional in-person meetings or events • •.Equiniti is a leading int...[show_more]
    [last_updated.last_updated_30] • [promoted]
    AVP, Strategic Credit Initiatives

    AVP, Strategic Credit Initiatives

    Synchrony • Phoenix, AZ, United States
    [job_card.full_time]
    The AVP, Strategic Credit Initiatives role is responsible for managing and driving client initiatives across the Credit and Fraud lifecycle. This includes Acquisitions, Account Management and Transa...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Director AI

    Director AI

    Honeywell INC. • Phoenix, AZ, United States
    [job_card.permanent]
    As a Director of AI here at Honeywell, you will set the strategic direction and ensure the successful delivery of our technology projects. Collaborating closely with senior executives and cross‑func...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Remote Financial Advising Expert - AI Trainer ($50-$60 / hour)

    Remote Financial Advising Expert - AI Trainer ($50-$60 / hour)

    Data Annotation • Tempe, AZ, United States
    [filters.remote]
    [job_card.full_time] +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Manager, Data Privacy and AI Governance

    Senior Manager, Data Privacy and AI Governance

    Carvana • Tempe, AZ
    [job_card.full_time]
    [filters_job_card.quick_apply]
    About Carvana At Carvana, we’re changing the way people buy and sell cars.With an ambitious vision and a fundamentally different approach designed to be fun, fast,...[show_more]
    [last_updated.last_updated_variable_days]
    Manager, Client Service - WAMS

    Manager, Client Service - WAMS

    Charles Schwab • Paradise Valley, AZ, United States
    [job_card.full_time]
    At Schwab, you’re empowered to make an impact on your career.Here, innovative thought meets creative problem solving, helping us “challenge the status quo” and transform the finance industry togeth...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    AI Strategy Director - Enterprise Innovation (Hybrid)

    AI Strategy Director - Enterprise Innovation (Hybrid)

    Honeywell INC. • Phoenix, AZ, United States
    [job_card.full_time]
    A global technology firm in Phoenix is seeking a Director of AI to guide strategic AI initiatives and ensure successful IT project delivery. The role involves collaboration with senior executives, m...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Cybersecurity Customer Success Manager

    Cybersecurity Customer Success Manager

    Lumifi Cyber • Scottsdale, Arizona, US
    [job_card.full_time]
    Job Description Job Description Customer Success Manager Job Type : Full-Time, As the Customer Success Manager, you are responsible for ongoing customer relationship building with the goal of driv...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    AVP, Strategic Credit Initiatives

    AVP, Strategic Credit Initiatives

    Synchrony Financial • Phoenix, Arizona, US
    [job_card.full_time]
    Job ID : 2600298 Job Description : Role Summary / Purpose The AVP, Strategic Credit Initiatives role is responsible for managing and driving client initiatives across the Credit and Fraud lifecycle.Th...[show_more]
    [last_updated.last_updated_1_hour] • [promoted] • [new]
    Remote R Engineer - AI Trainer

    Remote R Engineer - AI Trainer

    SuperAnnotate • Paradise Valley, Arizona, US
    [filters.remote]
    [job_card.full_time]
    As a remote, hourly paid R Engineer, you will review AI-generated responses and generate high-quality R and data-analysis-focused content, evaluating the reasoning quality and step-by-step problem-...[show_more]
    [last_updated.last_updated_variable_days]
    Sr Analyst, Hybrid IA Compliance

    Sr Analyst, Hybrid IA Compliance

    Osaic • Scottsdale, Arizona, United States
    [job_card.full_time]
    Current Employees and Contractors Apply Here.Senior Analyst for Regulatory Services Opportunity in Financial Services.Senior Analyst, Hybrid IA Compliance. Atlanta : 2300 Windy Ridge Pkwy SE, Suite75...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]