Talent.com
Senior Software Engineer, ML Systems Evaluation
Senior Software Engineer, ML Systems EvaluationA • Sunnyvale, California, United States
[error_messages.no_longer_accepting]
Senior Software Engineer, ML Systems Evaluation

Senior Software Engineer, ML Systems Evaluation

A • Sunnyvale, California, United States
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

WAYFINDER

Our Wayfinder team is building scalable, certifiable autonomy systems to power the next generation of commercial aircraft. Our team of experts is driving the maturation of machine learning and other core technologies for autonomous flight; we are creating a reference architecture that includes hardware, software, and a data-driven development process to allow aircraft to perceive and react to their environment. Autonomous flight is transforming the transportation industry, and our team is at the heart of this revolution.

The Opportunity / Role Description

As a Senior Software Engineer, ML Systems Evaluation on the Wayfinder team, you will play a critical role in developing and validating safety-critical AI-based systems that empower commercial aircraft to perceive, understand, and react to their surroundings in real-time. Your focus will be on developing, maintaining, and refining the evaluation framework and pipeline used to measure model accuracy, robustness, and real-world behavior in the complex, safety-critical world of commercial aviation.

Your primary responsibility will be to lead and contribute to the design, development, and implementation of the testing pipelines that assess model predictions on real and synthetic data, helping us identify, analyze, and resolve failure modes. This role requires strong software engineering fundamentals, as well as familiarity with modern pipeline architecture and tooling. You will collaborate with cross-functional teams of AI / ML experts, data engineers, and system architects to develop and execute comprehensive testing strategies, ensuring the highest levels of trust and confidence in our AI solutions.

You'll lead the design of automated evaluation pipelines that assess model predictions on real and synthetic image data taken from aircraft during landing approaches, helping us identify, analyze, and resolve failure modes.

Responsibilities

  • Lead the development of robust and scalable model evaluation frameworks for AI systems used in safety-critical environments.
  • Execute rigorous verification and validation strategies to ensure AI systems meet aviation certification standards.
  • Collaborate with systems engineers, AI / ML researchers and data engineers to develop tests, test, analyze, and improve the performance, robustness, and reliability of neural networks and other AI models.
  • Build automated testing pipelines to evaluate AI system behavior under a wide range of operational conditions, ensuring they perform safely and reliably in real-world scenarios.
  • Develop tools and metrics to assess the generalization, accuracy, and failure modes of AI models, identifying areas for improvement through better data sets, feature engineering, or architecture modifications.
  • Work closely with the systems and certification teams to ensure the AI systems meet all performance, embeddability and compliance requirements.
  • Conduct code reviews and mentor junior engineers to ensure best practices in testing, documentation, and code quality.
  • Stay informed about advancements in AI safety testing, aviation standards, and certification processes, and help shape the testing strategies for future AI-based aviation systems.

Requirements

  • Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field
  • Master’s Degree : 7 or more years of relevant experience
  • Bachelor’s Degree : 9 or more years of relevant experience
  • Proficiency in Python and experience with automation tools and CI / CD pipelines.
  • Familiarity with testing methodologies for neural networks.
  • Demonstrated ability to work in a collaborative, cross-functional environment.
  • Strong problem-solving skills and a commitment to quality and safety.
  • Strong sense of ownership.
  • Strong Plus

  • Experience with evaluation of ML models.
  • Experience developing evaluation frameworks.
  • Nice to have

  • Familiarity with :
  • MLOps tools such as MLFlow, DVC, Weights and Biases, or similar
  • Workflow orchestration tools (e.g. Dagster, Airflow, Dagster, Prefect)
  • Containerization tools (e.g. Docker)
  • ML / AI frameworks (TensorFlow, PyTorch, etc.)
  • Knowledge of AI safety concerns, including adversarial attacks, robustness, explainability, and uncertainty estimation.
  • Experience in developing safety-critical systems for the aviation industry or other highly regulated sectors (e.g., automotive or healthcare).
  • Experience with cloud platforms and distributed testing environments.
  • Strong understanding of data collection, augmentation, and synthetic data generation to improve testing outcomes.
  • Compensation :

    The estimated salary range for this position is $171,000 to $190,000 annually.  Enjoy comprehensive benefits : health insurance, paid time off, holidays, 401(k), Flexible Spending Account, Health Savings Account, Airbus Employee Share Ownership Plan, flight training, and more.

    Experience flexibility with our hybrid work model, which includes three days in the office to foster collaboration and innovation while allowing for remote work options. Additionally, employees can work remotely—inside or outside the U.S.—for up to 31 days per year.

    Why Join Us?

    Be a part of a dynamic team that values creativity, collaboration, innovation and problem solving.  At Acubed, your contributions will directly impact our digital future.  We welcome diverse perspectives and are committed to fostering an inclusive environment.

    Acubed is committed to creating a fair and equitable workplace for all. We seek applicants of all backgrounds and identities, across race, color, ethnicity, national origin or ancestry, age, citizenship, religion, sex, sexual orientation, gender identity or expression, veteran status, marital status, pregnancy or parental status, or disability. Applicants will not be discriminated against based on these or other protected categories or social identities.

    Acubed Requirements

    All job offers at Acubed are contingent upon the candidate passing references, background and export control checks.

  • Please Note that Acubed does not offer sponsorship of employment-based nonimmigrant visa petitions for this role.
  • [job_alerts.create_a_job]

    Senior Software Engineer ML Systems Evaluation • Sunnyvale, California, United States

    [internal_linking.similar_jobs]
    Senior Systems Engineer

    Senior Systems Engineer

    Calyxo, Inc. • Pleasanton, CA, United States
    [job_card.full_time]
    The company was founded in 2016 to address the profound need for improved kidney stone treatment.Kidney stone disease is a common, painful condition that consumes vast amounts of healthcare resourc...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer (Systems Level)

    Senior Software Engineer (Systems Level)

    Yubico • Santa Clara, California, United States
    [job_card.full_time]
    Our company’s mission is to make secure login easy and available for everyone.Yubico was founded in 2007 by Stina and Jakob Ehrensvard, and recently went public on Nasdaq First North Growth Market ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer, Data Platform - Experimentation & Evaluation

    Senior Software Engineer, Data Platform - Experimentation & Evaluation

    TikTok • San Jose, CA, United States
    [job_card.full_time]
    Senior Software Engineer, Data Platform - Experimentation & Evaluation.Get AI-powered advice on this job and more exclusive features. Team Introduction : Our mission in the experimentation and evalua...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Software Engineer, Systems Modelling

    Software Engineer, Systems Modelling

    Psiquantum • Milpitas, California, United States
    [job_card.full_time]
    PsiQuantum’s mission is to build the first useful quantum computers—machines capable of delivering the breakthroughs the field has long promised. Since our founding in 2016, our singular focus has b...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Software Engineer, Integration & System Validation (Starlink - Direct To Cell)

    Software Engineer, Integration & System Validation (Starlink - Direct To Cell)

    Spacex • Sunnyvale, California, United States
    [job_card.full_time] +1
    SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today SpaceX is actively developing the technolo...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer - ML / LLM Serving

    Senior Software Engineer - ML / LLM Serving

    Alldus • San Jose, CA, United States
    [job_card.full_time]
    Senior Software Engineer - ML / LLM Serving.This range is provided by Alldus.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Direct message the jo...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Software Engineer - ML Performance

    Software Engineer - ML Performance

    Baseten • San Ramon, California, United States
    [job_card.full_time]
    We’re a growing team of builders backed by top-tier investors, including.ML teams at enterprises and category-defining AI-native companies like. Baseten to power their core production workloads with...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer – Metrology Systems

    Senior Software Engineer – Metrology Systems

    KLA • Milpitas, CA, United States
    [job_card.full_time]
    A global leader in semiconductor technologies is seeking a Software Engineer to develop and maintain software for metrology solutions. You'll work collaboratively with international teams and partic...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer, Metrology Systems

    Senior Software Engineer, Metrology Systems

    KLA-Belgium • Milpitas, CA, United States
    [job_card.full_time]
    A global leader in semiconductor manufacturing is seeking a Software Engineer in Milpitas, California.This role involves developing and maintaining software for complex metrology systems.Candidates...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer — GenAI & RAG Platforms

    Senior ML Engineer — GenAI & RAG Platforms

    Cisco Systems • Santa Clara, CA, United States
    [job_card.full_time]
    A global technology leader seeks an experienced engineer to develop AI-driven services and APIs for its hybrid, multi-cloud environment. Ideal candidates will have 5+ years in backend or distributed...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer : Sampling & Evaluation Pipelines

    Senior Software Engineer : Sampling & Evaluation Pipelines

    Waymo • Mountain View, CA, United States
    [job_card.full_time]
    A leading autonomous technology company based in California seeks an experienced engineer to develop importance sampling techniques for evaluation pipelines. You will collaborate with teams to optim...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Senior ML Engineer (Level 5) — Deploy & Scale Models

    Senior ML Engineer (Level 5) — Deploy & Scale Models

    Minimal • Palo Alto, CA, United States
    [job_card.full_time]
    A leading technology company in Palo Alto is seeking a Machine Learning Engineer to build and deploy models for core products. Responsibilities include applying modern ML techniques and collaboratin...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior ML Engineer, Level 5 — Ranking & Recommendations

    Senior ML Engineer, Level 5 — Ranking & Recommendations

    Snap Inc. • Palo Alto, CA, United States
    [job_card.full_time]
    An innovative technology company is seeking a Machine Learning Engineer to join their dynamic team.In this role, you will leverage your expertise in machine learning to create impactful models that...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Principal AI / ML System Software Engineer

    Principal AI / ML System Software Engineer

    d-Matrix • Santa Clara, CA, United States
    [job_card.full_time]
    AI to power the transformation of technology.We are at the forefront of software and hardware innovation, pushing the boundaries of what is possible. We value humility and believe in direct communic...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer — Pricing Platform & Systems

    Senior ML Engineer — Pricing Platform & Systems

    Uber • Sunnyvale, CA, United States
    [job_card.full_time]
    A leading technology company is seeking a Senior Staff Machine Learning Engineer to lead the development of advanced ML systems for courier pricing. The role demands extensive experience in building...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer – Autonomy Evaluation Platform

    Senior Software Engineer – Autonomy Evaluation Platform

    General Motors • Mountain View, CA, United States
    [job_card.full_time]
    As a Senior Software Engineer within the Evaluation team under the Simulation, Evaluation, and Data organization, you will shape the design and delivery of the evaluation strategy for GM’s autonomo...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    ML Evaluation Engineer - Multimodal Foundation Models

    ML Evaluation Engineer - Multimodal Foundation Models

    Apple • Cupertino, CA, United States
    [job_card.full_time]
    A leading technology company is seeking a Machine Learning Engineer to develop user-facing conversational features.You will design evaluation processes, collaborate with researchers, and conduct st...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Hybrid ML Engineer — Large-Scale Recommender Systems

    Hybrid ML Engineer — Large-Scale Recommender Systems

    Pinterest • Palo Alto, CA, United States
    [job_card.full_time]
    A leading social media platform in the United States is seeking Machine Learning Engineers to innovate and enhance user experiences. Candidates will contribute to cutting-edge projects in machine le...[show_more]
    [last_updated.last_updated_1_day] • [promoted]