Talent.com
Senior AI Engineer APM Experiences
Senior AI Engineer APM ExperiencesDatadog • New York City, New York, USA
Senior AI Engineer APM Experiences

Senior AI Engineer APM Experiences

Datadog • New York City, New York, USA
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

The opportunity

Datadogs APM Experiences team owns the core product experience for Application Performance Monitoring including distributed tracing service representation and more. Were building a new wave of AI-powered capabilities that help customers detect resolve and prevent performance issues this role you will lead endtoend development of LLM- and Agentbased features that can :

  • Debug and investigate application performance issues down to the root cause as both a developer assistant and a fully autonomous agent
  • Proactively recommend performance and reliability-based optimizations to prevent the next incident
  • Automatically create intelligent monitors and SLOs for the most important business flows and critical paths

This is a highly productminded engineering role : youll work from problem discovery and UX all the way to reliable scalable production systems.

What youll do

  • Shape AI experiences for APM. Design and ship LLM / agentic workflows that analyze traces metrics logs and other telemetry to generate diagnoses explanations and guided fixes.
  • Own the full loop. Prototype quickly define success metrics and evals run experiments iterate and ultimately productionize for scale and reliability.
  • Build robust agent systems. Develop tools retrieval and planning strategies and guardrails; manage prompts / evals; design fallbacks and humanintheloop paths.
  • Integrate with Datadogs platform. Leverage surfaces like Trace Explorer Service Catalog monitors and workflows to deliver endtoend value in the APM UI.
  • Partner deeply. Collaborate with PM Design and partner teams to build cohesive experiences.
  • Raise the bar on engineering. Write performant maintainable backend code own services in production and improve reliability for highthroughput lowlatency data systems.
  • Who you are

    Productminded engineer who ships AI to production

  • 4 years building backend or real-time ML systems; you value simplicity correctness and performance
  • Proven experience delivering LLM / agent features to production (prompting tooling evals safety / guardrails)
  • Comfortable owning user journeys iterating from prototype alpha GA and measuring impact with clear product metrics
  • Strong ML / applied science fundamentals

  • Solid grasp of the ML lifecycle (task definition dataset collection modeling evaluation deployment iteration) and statistics (experiment design confidence intervals)
  • Experience choosing / modeling the right technique for the job (e.g. anomaly detection ranking / recommendation NLP) and knowing when a heuristic beats a model
  • Fluency with offline / online evals for AI systems; can build reliable golden sets and automatic regressions
  • Distributed systems & observability savvy

  • Experience with microservices performance : tracing latency breakdowns concurrency and resiliency patterns
  • Proficient in Go Java or Python; strong API / service design; production ops (monitoring alerting oncall rotation)
  • Nice to have

  • Handson with distributed tracing stacks (OpenTelemetry / Datadog APM) profilers and logs / metrics pipelines
  • Exposure to planning / agent frameworks tooluse orchestration RAG and retrieval / indexing for observability data
  • Familiarity with SLO / SLA practices and incident response
  • Benefits and Growth :

  • Get to build tools for software engineers just like yourself. And use the tools we build to accelerate our development.
  • Have a lot of influence on product direction and impact on the business.
  • Work with skilled knowledgeable and kind teammates who are happy to teach and learn.
  • Competitive global benefits.
  • Continuous professional development.
  • Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

    Required Experience :

    Senior IC

    Key Skills

    APIs,C / C++,Computer Graphics,Go,React,Redux,Node.js,AWS,Library Services,Assembly,GraphQL,High Voltage

    Employment Type : Full Time

    Experience : years

    Vacancy : 1

    [job_alerts.create_a_job]

    Senior Engineer Ai • New York City, New York, USA

    [internal_linking.related_jobs]
    Senior AI Engineer

    Senior AI Engineer

    AirOps • New York, New York, United States
    [filters.remote]
    [job_card.full_time]
    Today thousands of leading brands and agencies use.We’re building the platform and profession that will empower a million marketers to become modern leaders — not spectators — as AI reshapes how br...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior GenAI Platform Engineer - Enterprise AI

    Senior GenAI Platform Engineer - Enterprise AI

    Scale AI • New York, NY, US
    [job_card.full_time]
    Scale AI is seeking a Senior Software Engineer for its Generative AI platform, aiming to enhance enterprise-grade solutions across various sectors. The ideal candidate will possess a strong foundati...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI / ML Engineer

    Senior AI / ML Engineer

    Sigma Computing • New York, New York, United States
    [job_card.full_time]
    At Sigma, we’re not just adding AI—we’re building the future of how people work with data.Our platform already lets users explore billions of rows of data in seconds with a spreadsheet-like interfa...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Engineer — Platform & Agent Automation

    Senior AI Engineer — Platform & Agent Automation

    Imprint • New York, NY, United States
    [job_card.full_time]
    A financial technology company in San Francisco is seeking an AI Engineer to build and enhance its internal agent platform. The ideal candidate will possess strong programming skills, experience wit...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Engineer

    Senior AI Engineer

    Resonance • New York, NY, US
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Resonance is a technology company building a more sustainable and valuable fashion industry for designers, brands, manufacturers, consumers, and the planet. The company’s AI-powered operating system...[show_more]
    [last_updated.last_updated_30]
    Senior Manager, AI Engineer

    Senior Manager, AI Engineer

    SoFi • New York City, New York, USA
    [job_card.full_time]
    AI engineering role in SoFis growing independent risk organization.This is a critical senior role responsible for setting the technical direction driving execution and ensuring the successful deliv...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Sales Engineer AIML Solutions

    Senior Sales Engineer AIML Solutions

    Wiley Global Technology • Hoboken, New Jersey, USA
    [job_card.full_time]
    Senior Sales Engineer - AI / ML Solutions.Our mission is to unlock human potential.We welcome you for who you are the background you bring and we embrace individuals who get excited about learning.Br...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer, AI Agent

    Senior Software Engineer, AI Agent

    Astronomer • New York, New York, United States
    [job_card.full_time]
    Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Principal AI Engineer, Ads AI

    Principal AI Engineer, Ads AI

    LinkedIn • New York City, New York, USA
    [job_card.full_time]
    The Assets Formats & Placements team is part of our LinkedIn Marketing Solutions (Ads Technology) group.This team has a vision of building a content onboarding and asset generation platform inn...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Engineer

    Senior AI Engineer

    OneStream Software • New York, NY, United States
    [job_card.full_time]
    Senior AI Engineer (REACT / Front End).Hybrid, USA (Birmingham, MI, Chicago, IL, New York, NY, Fort Lauderdale, FL).Range applies to US candidates only) + Benefits / Variable Comp / Equity - Range may va...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    Senior AI Engineer - Cloud Engineering Focus (Hybrid)

    OneStream Software • New York, NY, United States
    [job_card.full_time]
    Senior AI Engineer - Cloud Engineering Focus.Hybrid, USA (Birmingham, MI, Chicago, IL, New York, NY, Fort Lauderdale, FL). Range applies to US candidates only) + Benefits / Variable Comp / Equity - Rang...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Engineer, Conversational AI & Agentic Systems

    Senior AI Engineer, Conversational AI & Agentic Systems

    Blooming Health • New York City, New York, USA
    [job_card.full_time]
    Blooming Health is a mission-driven venture-backed health tech company dedicated to transforming the social care landscape. We are building an intelligent platform to help individuals seamlessly nav...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    AI Engineer

    AI Engineer

    Oscar • New York, NY, US
    [job_card.full_time]
    AI Engineer - Healthcare Automation Platform.Healthcare AI | Hybrid (NYC preferred) or Remote.AI-powered automation platform. Our system processes complex, unstructured data to ensure time-sensitive...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Engineer

    Senior AI Engineer

    Tiger Analytics • Jersey City, NJ, US
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Tiger Analytics is seeking a highly skilled.In this role, you will design, develop, and implement cutting-edge AI solutions that drive business value for our clients. You will leverage advanced mach...[show_more]
    [last_updated.last_updated_30]
    Senior AI Solutions Engineer — Enterprise R&D Sales

    Senior AI Solutions Engineer — Enterprise R&D Sales

    Wiley • Hoboken, New Jersey, United States
    [job_card.full_time]
    A leading educational publisher in New Jersey is seeking a Senior Sales Engineer to enhance AI-powered research tool adoption. The role involves technical demonstrations, proof-of-concept designs, a...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Applied AI Engineer, Enterprise GenAI

    Applied AI Engineer, Enterprise GenAI

    Scale AI, Inc. • New York, NY, United States
    [job_card.full_time]
    AI is becoming vitally important in every function of our society.At Scale, our mission is to accelerate the development of AI applications. For 8 years, Scale has been the leading AI data foundry, ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior AI Engineer

    Senior AI Engineer

    Fonzi Ai • New York, New York, United States
    [job_card.full_time]
    We're a well-funded, fast-growing startup revolutionizing hiring through AI.Recently launched, we've already attracted thousands of top engineers to our platform. Our vision is to build an AI-powere...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior AI Solutions Engineer - Enterprise R&D Sales

    Senior AI Solutions Engineer - Enterprise R&D Sales

    Wiley • Hoboken, NJ, US
    [job_card.full_time]
    A leading educational publisher in New Jersey is seeking a Senior Sales Engineer to enhance AI-powered research tool adoption. The role involves technical demonstrations, proof-of-concept designs, a...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]