Talent.com
Client Service Risk Manager - AI Evaluation and Testing
Client Service Risk Manager - AI Evaluation and TestingMcKinsey & Company • Phoenix
Client Service Risk Manager - AI Evaluation and Testing

Client Service Risk Manager - AI Evaluation and Testing

McKinsey & Company • Phoenix
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

You are someone who thrives in a high-performance environment, bringing a growth mindset and entrepreneurial spirit to tackle meaningful challenges that have a real impact.

In return for your drive, determination, and curiosity, we’ll provide the resources, mentorship, and opportunities to help you quickly broaden your expertise, grow into a well-rounded professional, and contribute to work that truly makes a difference.

When you join us, you will have :

  • Continuous learning :  Our learning and apprenticeship culture, backed by structured programs, is all about helping you grow while creating an environment where feedback is clear, actionable, and focused on your development. The real magic happens when you take the input from others to heart and embrace the fast-paced learning experience, owning your journey.
  • A voice that matters :  From day one, we value your ideas and contributions. You’ll make a tangible impact by offering innovative ideas and practical solutions, all while upholding our unwavering commitment to ethics and integrity. We not only encourage diverse perspectives, but they are critical in driving us toward the best possible outcomes.
  • Global community :  With colleagues across 65+ countries and over 100 different nationalities, our firm’s diversity fuels creativity and helps us come up with the best solutions. Plus, you’ll have the opportunity to learn from exceptional colleagues with diverse backgrounds and experiences.
  • Exceptional benefits :  On top of a competitive salary (based on your location, experience, and skills), we provide a comprehensive benefits package to enable holistic well-being for you and your family.

You will work within the firm's Technology Risk function as the Client Service Risk Manager, of AI Evaluation and Testing, ensuring our firm’s strong ability to evaluate and validate AI systems including those used and built internally and in client contexts by consultants for reliability, safety, security, and regulatory compliance.

You, as the lead in AI evaluation and testing, will define the vision and strategic roadmap for AI control objectives and testing. You will establish frameworks, requirements, and guidance to assess behavior and ensure reliability (including accuracy, bias, and other factors) across various modalities such as text, code, multimodal systems, and agents. Additionally, you will define and prioritize the requirements for tooling solutions that support AI governance, observability, and testing for all relevant AI systems and agents.

You will also develop or validate frameworks and methodologies for conducting red-team attacks on AI systems (jailbreaks, prompt injections, system prompt extraction). You will partner with first line of defense to implement evaluation gates and build a control environment with appropriate technical guardrails to ensure compliance with applicable regulations (e.g., the EU AI Act).

You will report directly to the Associate Director of Client Service Risk, Technology Knowledge & Frameworks, within the firm’s global Risk function. You will partner closely with Legal, Risk teams, Technology and Product teams to develop consistent and scalable RAI control objectives and testing methodologies.

You will support updates to existing robust policies and standards in line with development in AI capabilities (e.g., agentic).

Your collaboration will extend to various roles and functions across the firm, including senior leadership, to whom you will present and gain support for your strategy and program, report progress, and demonstrate impact.

  • U.S. Bachelor’s degree or equivalent work experience required
  • 8+ years in Trust & Safety (AI / ML), risk, or security domains with Data Science background; the scope of the models includes traditional ML models, GenAI and agentic deployments
  • 5+ years managing complex cross-functional projects
  • Familiarity with AI safety, testing and validation, and governance; experience partnering with engineers to build control and integrity systems
  • Knowledge of AI evaluation methods and ability to connect evaluation results to guardrails and controls; Strong knowledge of LLM behavior and jailbreak techniques
  • Proven ability to translate complex ethical challenges into actionable policies and frameworks; strong policy development, risk management, and analytical skills
  • Experience working with engineering and product teams to create tools, solutions, or automation
  • Ability to work in-person (team schedules and office locations may vary)
  • Ability to operate office equipment
  • Excellent leadership and communication skills, with a proven ability to navigate ambiguity and influence product direction in a fast-changing environment
  • Excellent written and oral communication skills, as well as interpersonal skills, including the ability to articulate technical concepts to both technical and non-technical audiences
  • [job_alerts.create_a_job]

    Client Service Risk Manager AI Evaluation and Testing • Phoenix

    [internal_linking.similar_jobs]
    Analyst I, Merchant Risk

    Analyst I, Merchant Risk

    Affirm • Phoenix, AZ, United States
    [job_card.full_time]
    Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest.The Merchant Risk Analyti...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Risk Analyst

    Risk Analyst

    Staff Financial Group • Phoenix, AZ, United States
    [job_card.full_time]
    A growing auto finance company building out its credit risk team.Analyze and forecast repossessions, origination risks, servicing exposure, and overall credit performance.Newly created position due...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote Senior Risk Adjustment Specialist (MA / ACA)

    Remote Senior Risk Adjustment Specialist (MA / ACA)

    Oscar Health • Phoenix, AZ, United States
    [job_card.full_time]
    A health insurance company seeks a Risk Adjustment Coder Specialist for a remote position focusing on Medicare Advantage and ACA lines of business. The role includes conducting medical record review...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Remote FP&A Manager – AI Trainer ($50-$60 / hour)

    Remote FP&A Manager – AI Trainer ($50-$60 / hour)

    Data Annotation • Phoenix, Arizona
    [filters.remote]
    [job_card.full_time] +1
    We are looking for a finance professional to join our team to train AI models.You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Contracts & Risk Analyst (Hybrid)

    Senior Contracts & Risk Analyst (Hybrid)

    SWCA Environmental Consultants • Phoenix, AZ, United States
    [job_card.full_time] +1
    A leading environmental consulting firm is seeking a Senior Contracts Analyst in Phoenix, AZ.This full-time, permanent position involves reviewing and drafting contracts, ensuring compliance with p...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Payments Risk Analyst II

    Payments Risk Analyst II

    Phoenix Staffing Services • Phoenix, AZ, United States
    [job_card.full_time]
    Ready to be pushed beyond what you think you're capable of? At Coinbase, our mission is to increase economic freedom in the world. It's a massive, ambitious opportunity that demands the best of us, ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Consulting Actuary - Medicare Risk Management

    Senior Consulting Actuary - Medicare Risk Management

    Blue Cross and Blue Shield of North Carolina • Phoenix, AZ, United States
    [job_card.full_time]
    Join our dynamic healthcare organization as a Senior Consulting Actuary, where you will play a crucial role in revolutionizing care delivery through strategic actuarial insights and data-driven ana...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Associate Actuary, Risk and Compliance Management

    Associate Actuary, Risk and Compliance Management

    Humana • Phoenix, AZ, United States
    [job_card.full_time]
    Join our dedicated team and make a significant impact on health!.As an Associate Actuary in Risk and Compliance Management, you will play a vital role in leading a talented group focused on identif...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Senior Divisional Risk Advisor, Digital & Analytics

    Senior Divisional Risk Advisor, Digital & Analytics

    Vanguard • Scottsdale, AZ, United States
    [job_card.full_time]
    A leading investment management company seeks a Divisional Risk Advisor to provide risk expertise across their services.The successful candidate will partner with cross-functional teams to deliver ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Autonomous Vehicle Claims & Risk Analytics Analyst

    Autonomous Vehicle Claims & Risk Analytics Analyst

    Waymo • Phoenix, AZ, United States
    [job_card.full_time]
    A leading autonomous driving technology company is seeking an experienced insurance claim analyst to support their claim processes. The role involves creating strategies for handling unique claims r...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Sr. AI Deployment Manager

    Sr. AI Deployment Manager

    Dialpad • Tempe, Arizona, US
    [job_card.full_time]
    Job Description Job Description About Dialpad Dialpad is the leading AI-powered customer communications platform, transforming how businesses communicate with their customers.More than 50,000 com...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Cybersecurity Customer Success Manager

    Cybersecurity Customer Success Manager

    Lumifi Cyber • Scottsdale, AZ, US
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Customer Success Manager Job Type : .Full-Time, As the Customer Success Manager, you are responsible for ongoing customer relationship building with the goal of driving contract renewals and expansio...[show_more]
    [last_updated.last_updated_variable_days]
    Client Service Manager

    Client Service Manager

    Heffernan Insurance Brokers • Scottsdale, AZ, United States
    [job_card.full_time]
    The Client Service Manager is responsible for delivering a high level of service to Heffernan Financial clients and to the advisors they support. The Client Service Manager is expected to enhance cl...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Risk Analyst - Cyber Risk

    Senior Risk Analyst - Cyber Risk

    Genesis10 • Phoenix, AZ, United States
    [job_card.temporary]
    Senior Risk Analyst - Cyber Risk.Genesis10 is currently seeking a Senior Risk Analyst - Cyber Risk with our client in the financial industry. This is a hybrid 6-month contract to hire role located i...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Risk Control Consultant

    Senior Risk Control Consultant

    Liberty Mutual Insurance • Phoenix, AZ, United States
    [job_card.full_time]
    A leading insurance company in Phoenix is seeking a Risk Engineering Technical Consultant to provide expert services in risk assessment and control. The ideal candidate will have over 8 years of con...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    AI Business Risk and Controls Advisor Senior

    AI Business Risk and Controls Advisor Senior

    Charlotte Staffing • Phoenix, AZ, United States
    [job_card.full_time]
    AI Business Risk And Controls Advisor Senior.At USAA, our mission is to empower our members to achieve financial security through highly competitive products, exceptional service and trusted advice...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Risk Manager

    Risk Manager

    New Mexico Staffing • Phoenix, AZ, United States
    [job_card.full_time]
    AECOM is looking for a Risk Manager to join our Tucson, AZ, or San Diego, CA or Phoenix, AZ, or El Paso, TX, or Carlsbad, NM office to support Federal projects and programs.Support risk assessments...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Client Project Manager

    Client Project Manager

    Beyond Feedback • Phoenix, Arizona, US
    [job_card.full_time]
    Job Description Job Description At Beyond Feedback we bring humanity back to business by partnering with business leaders to provide employee and customer lifecycle feedback programs.The tailor-m...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]