Talent.com
Sr. Engineering Manager, AI Evaluation Platform
Sr. Engineering Manager, AI Evaluation PlatformSeattle Staffing • Seattle, WA, US
[error_messages.no_longer_accepting]
Sr. Engineering Manager, AI Evaluation Platform

Sr. Engineering Manager, AI Evaluation Platform

Seattle Staffing • Seattle, WA, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Engineering Manager

Join Apple Services Engineering to build the next generation of AI evaluation systems. We are seeking a hands-on Engineering Manager to architect high-availability services and internal tools that enable self-service evaluation at scale. You will partner with researchers to operationalize their innovations, transforming complex workflows into intuitive, developer-first platforms. We are looking for a leader who thrives in the ambiguity of new initiatives and is passionate about building scalable infrastructure.

You will build and lead the engineering team responsible for democratizing AI evaluation across the organization. Your focus will be on architecting the developer experiencedesigning the APIs, SDKs, and platform services that turn complex evaluation metrics into simple, self-service calls. You will work hand-in-hand with researchers to operationalize sophisticated measurement techniques, ensuring they scale reliably within our high-availability infrastructure. In this role, you will define the engineering standards for a new organization, establishing the code quality, automation, and testing rigor required to support the rapid evolution of Generative AI and Agentic systems.

Minimum Qualifications :

  • 5+ years of direct engineering management experience, with a proven track record of hiring, mentoring, and retaining high-performing engineers. You have successfully managed teams that ship production-grade software.
  • 7+ years of hands-on software engineering experience with deep proficiency in the Python ecosystem (e.g., FastAPI, Pydantic, Pandas). You are capable of contributing to code reviews and architectural discussions on day one.
  • Customer Obsession & Product Thinking : Experience acting as a technical partner to internal customers. You can translate vague requirements from other teams into concrete engineering specifications and are comfortable prioritizing the roadmap in the absence of a dedicated Product Manager.
  • Demonstrated experience partnering with Applied Scientists or Researchers : You have the ability to navigate the ambiguity of research workflows and operationalize scientific code.
  • Functional literacy in AI / ML concepts : You understand the fundamental lifecycle of machine learning (datasets, training vs. inference, evaluation metrics) and can discuss the engineering challenges involved in serving models.
  • Strong expertise in API Design & Internal Tools : You have architected APIs that other developers rely on, with a focus on versioning, backward compatibility, and developer experience.
  • Operational excellence background : You have practical experience establishing CI / CD pipelines, containerization (Docker / Kubernetes), and monitoring (Datadog / Prometheus).

Preferred Qualifications :

  • Experience building MLOps & Platform Infrastructure : You have architected or managed teams that built the foundational infrastructure for AI, such as model registries, inference services, or feature stores (using tools like Kubernetes, Ray, or Kubeflow).
  • Deep familiarity with AI Evaluation Frameworks : You have used or contributed to modern evaluation tools like DeepEval, Ragas, TruLens, or LangSmith. You understand how to implement and scale model-based evaluation workflows.
  • Deep understanding of Generative AI & Agents : You understand the engineering challenges of relying on LLMs and Agents as software componentsspecifically managing token economics, handling rate limits, and evaluating non-deterministic, multi-step reasoning capabilities.
  • Builder Experience : You have thrived in startup-like environments or incubated new teams within larger orgs, navigating high ambiguity to define roadmaps where none existed.
  • Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant.

    [job_alerts.create_a_job]

    Sr Engineering Manager AI Evaluation Platform • Seattle, WA, US

    [internal_linking.similar_jobs]
    Senior Product Manager – AI Platform (Early-Stage)

    Senior Product Manager – AI Platform (Early-Stage)

    BluePill AI • Seattle, WA, United States
    [job_card.full_time]
    A startup in AI technology located in Seattle is seeking an early-stage Product Manager to define a new product category. You will play diverse roles including product strategist and customer resear...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Sr. Director - AI Engineering

    Sr. Director - AI Engineering

    salesforce.com, inc. • Seattle, WA, United States
    [job_card.full_time]
    Salesforce is the #1 AI CRM, where humans with agents drive customer success together.And innovation isn't a buzzword - it's a way of life. The world of work as we know it is changing and we're look...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Sr. Product Manager, AI Search

    Sr. Product Manager, AI Search

    ChatGPT Jobs • Seattle, WA, United States
    [job_card.full_time]
    Senior Product Manager, AI Search.ZipRecruiter – Seattle, WA – $180,000 to $220,000 Yearly – Full-Time – Hybrid work environment. Our mission is to actively connect people to their next great opport...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Sr. Director - AI Engineering

    Sr. Director - AI Engineering

    Salesforce, Inc. • Seattle, WA, United States
    [job_card.full_time]
    To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts. Job CategorySoftware EngineeringJob Details • • • •Abo...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Director of AI Engineering

    Director of AI Engineering

    VirtualVocations • Everett, Washington, United States
    [job_card.full_time]
    Director of AI Engineering to lead the development and deployment of AI-powered cybersecurity solutions.Key Responsibilities Define and execute the technical roadmap for AI engineering to enhance...[show_more]
    [last_updated.last_updated_1_hour] • [promoted] • [new]
    AI Platforms Manager

    AI Platforms Manager

    VirtualVocations • Everett, Washington, United States
    [job_card.full_time]
    A company is looking for an AI Platforms Manager (Remote).Key Responsibilities Own the strategic roadmap for AI platforms, focusing on zero and low-code agent builders Manage relationships with ...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior AI Platform Product Leader for Enterprise Workflows

    Senior AI Platform Product Leader for Enterprise Workflows

    Outreach • Seattle, WA, United States
    [job_card.full_time]
    A leading Sales Execution Platform in Seattle is seeking an experienced product manager to drive the strategy and vision for their AI platform. This role requires over 5 years of enterprise product ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Product Manager, AI

    Senior Product Manager, AI

    Outreach • Seattle, WA, United States
    [job_card.full_time]
    In This Role You Can Expect To.Own the vision and strategy for Outreach AI Platform that powers intelligent workflows across the product suite. Define and prioritize the platform roadmap, aligning w...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Sr. Director - AI Engineering

    Sr. Director - AI Engineering

    Salesforce • Seattle, WA, United States
    [job_card.full_time]
    To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts. Job Category : Software Engineering.About Salesforc...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior Product Manager AI Platform

    Senior Product Manager AI Platform

    VirtualVocations • Everett, Washington, United States
    [job_card.full_time]
    A company is looking for a Senior / Lead Product Manager for their Agentic AI Platform.Key Responsibilities Set strategy, determine roadmap, and manage prioritization for platform capabilities Fac...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior Product Manager, Cloud Platform & AI

    Senior Product Manager, Cloud Platform & AI

    Google • Seattle, WA, United States
    [job_card.full_time]
    A leading technology company in Seattle is seeking a Senior Product Manager for Google Cloud.This role involves guiding product strategy and working cross-functionally to launch impactful cloud sol...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Applied AI Engineering Manager - Generative AI & Agentic

    Applied AI Engineering Manager - Generative AI & Agentic

    Highspot • Seattle, WA, United States
    [job_card.full_time]
    A leading software company in Seattle is seeking a Principal Engineering Manager for their Applied AI team.The role involves leading a skilled team to apply Generative AI to business challenges and...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Engineering Manager - Applied AI

    Principal Engineering Manager - Applied AI

    Highspot • Seattle, WA, United States
    [job_card.full_time]
    Highspot is pioneering the category that is fundamentally changing the way companies increase sales productivity.On a mission to transform the way millions of people work with sales enablement, Hig...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Sr. Product Manager - Kernels, AI / ML, Annapurna Labs

    Sr. Product Manager - Kernels, AI / ML, Annapurna Labs

    Amazon Web Services (AWS) • Seattle, WA, United States
    [job_card.full_time]
    Product Manager – Kernels, AI / ML, Annapurna Labs.AWS Neuron is looking for an experienced Technical Product Manager to define and drive product strategy for the Neuron Kernel Interface (NKI), a com...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Director, Edge Infrastructure & AI Growth

    Senior Director, Edge Infrastructure & AI Growth

    Slope • Seattle, WA, United States
    [job_card.full_time]
    A defense technology company is seeking a Senior Director of Business Development to advance edge infrastructure initiatives. The role involves developing strategies for distributed computing system...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Sr Applied Scientist, AI / ML Evaluation Quality, Apple Services Engineering

    Sr Applied Scientist, AI / ML Evaluation Quality, Apple Services Engineering

    Apple • Seattle, WA, United States
    [job_card.full_time]
    At Apple, we believe that technology should enrich people's lives in meaningful ways.Apple Services—including the App Store, Apple Music, Apple TV+, and Podcasts—connect millions of users worldwide...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Product Manager, AI Platform & Intelligent Workflows

    Senior Product Manager, AI Platform & Intelligent Workflows

    Outreach • Seattle, WA, United States
    [job_card.full_time]
    A leading AI solutions provider in Seattle is seeking a Mid-Senior Product Manager to guide the strategy for their AI Platform. This role involves defining the roadmap, collaborating with engineerin...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Search Product Manager — Transform Job Discovery

    Senior AI Search Product Manager — Transform Job Discovery

    ChatGPT Jobs • Seattle, WA, United States
    [job_card.full_time]
    A leading online employment marketplace is seeking a Senior Product Manager to enhance AI-driven search experiences for users. This role involves shaping search solutions that connect job seekers ef...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]