Talent.com
Senior Software Engineer, ML Systems Evaluation
Senior Software Engineer, ML Systems EvaluationA • Sunnyvale, California, United States
[error_messages.no_longer_accepting]
Senior Software Engineer, ML Systems Evaluation

Senior Software Engineer, ML Systems Evaluation

A • Sunnyvale, California, United States
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

WAYFINDER

Our Wayfinder team is building scalable, certifiable autonomy systems to power the next generation of commercial aircraft. Our team of experts is driving the maturation of machine learning and other core technologies for autonomous flight; we are creating a reference architecture that includes hardware, software, and a data-driven development process to allow aircraft to perceive and react to their environment. Autonomous flight is transforming the transportation industry, and our team is at the heart of this revolution.

The Opportunity / Role Description

As a Senior Software Engineer, ML Systems Evaluation on the Wayfinder team, you will play a critical role in developing and validating safety-critical AI-based systems that empower commercial aircraft to perceive, understand, and react to their surroundings in real-time. Your focus will be on developing, maintaining, and refining the evaluation framework and pipeline used to measure model accuracy, robustness, and real-world behavior in the complex, safety-critical world of commercial aviation.

Your primary responsibility will be to lead and contribute to the design, development, and implementation of the testing pipelines that assess model predictions on real and synthetic data, helping us identify, analyze, and resolve failure modes. This role requires strong software engineering fundamentals, as well as familiarity with modern pipeline architecture and tooling. You will collaborate with cross-functional teams of AI / ML experts, data engineers, and system architects to develop and execute comprehensive testing strategies, ensuring the highest levels of trust and confidence in our AI solutions.

You'll lead the design of automated evaluation pipelines that assess model predictions on real and synthetic image data taken from aircraft during landing approaches, helping us identify, analyze, and resolve failure modes.

Responsibilities

  • Lead the development of robust and scalable model evaluation frameworks for AI systems used in safety-critical environments.
  • Execute rigorous verification and validation strategies to ensure AI systems meet aviation certification standards.
  • Collaborate with systems engineers, AI / ML researchers and data engineers to develop tests, test, analyze, and improve the performance, robustness, and reliability of neural networks and other AI models.
  • Build automated testing pipelines to evaluate AI system behavior under a wide range of operational conditions, ensuring they perform safely and reliably in real-world scenarios.
  • Develop tools and metrics to assess the generalization, accuracy, and failure modes of AI models, identifying areas for improvement through better data sets, feature engineering, or architecture modifications.
  • Work closely with the systems and certification teams to ensure the AI systems meet all performance, embeddability and compliance requirements.
  • Conduct code reviews and mentor junior engineers to ensure best practices in testing, documentation, and code quality.
  • Stay informed about advancements in AI safety testing, aviation standards, and certification processes, and help shape the testing strategies for future AI-based aviation systems.

Requirements

  • Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field
  • Master’s Degree : 7 or more years of relevant experience
  • Bachelor’s Degree : 9 or more years of relevant experience
  • Proficiency in Python and experience with automation tools and CI / CD pipelines.
  • Familiarity with testing methodologies for neural networks.
  • Demonstrated ability to work in a collaborative, cross-functional environment.
  • Strong problem-solving skills and a commitment to quality and safety.
  • Strong sense of ownership.
  • Strong Plus

  • Experience with evaluation of ML models.
  • Experience developing evaluation frameworks.
  • Nice to have

  • Familiarity with :
  • MLOps tools such as MLFlow, DVC, Weights and Biases, or similar
  • Workflow orchestration tools (e.g. Dagster, Airflow, Dagster, Prefect)
  • Containerization tools (e.g. Docker)
  • ML / AI frameworks (TensorFlow, PyTorch, etc.)
  • Knowledge of AI safety concerns, including adversarial attacks, robustness, explainability, and uncertainty estimation.
  • Experience in developing safety-critical systems for the aviation industry or other highly regulated sectors (e.g., automotive or healthcare).
  • Experience with cloud platforms and distributed testing environments.
  • Strong understanding of data collection, augmentation, and synthetic data generation to improve testing outcomes.
  • Compensation :

    The estimated salary range for this position is $171,000 to $190,000 annually.  Enjoy comprehensive benefits : health insurance, paid time off, holidays, 401(k), Flexible Spending Account, Health Savings Account, Airbus Employee Share Ownership Plan, flight training, and more.

    Experience flexibility with our hybrid work model, which includes three days in the office to foster collaboration and innovation while allowing for remote work options. Additionally, employees can work remotely—inside or outside the U.S.—for up to 31 days per year.

    Why Join Us?

    Be a part of a dynamic team that values creativity, collaboration, innovation and problem solving.  At Acubed, your contributions will directly impact our digital future.  We welcome diverse perspectives and are committed to fostering an inclusive environment.

    Acubed is committed to creating a fair and equitable workplace for all. We seek applicants of all backgrounds and identities, across race, color, ethnicity, national origin or ancestry, age, citizenship, religion, sex, sexual orientation, gender identity or expression, veteran status, marital status, pregnancy or parental status, or disability. Applicants will not be discriminated against based on these or other protected categories or social identities.

    Acubed Requirements

    All job offers at Acubed are contingent upon the candidate passing references, background and export control checks.

  • Please Note that Acubed does not offer sponsorship of employment-based nonimmigrant visa petitions for this role.
  • [job_alerts.create_a_job]

    Senior Software Engineer ML Systems Evaluation • Sunnyvale, California, United States

    [internal_linking.similar_jobs]
    Senior Software Engineer, Data Platform - Experimentation & Evaluation

    Senior Software Engineer, Data Platform - Experimentation & Evaluation

    TikTok • San Jose, CA, United States
    [job_card.full_time]
    Senior Software Engineer, Data Platform - Experimentation & Evaluation.Get AI-powered advice on this job and more exclusive features. Team Introduction : Our mission in the experimentation and evalua...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer - ML / LLM Serving

    Senior Software Engineer - ML / LLM Serving

    Alldus • San Jose, CA, United States
    [job_card.full_time]
    Senior Software Engineer - ML / LLM Serving.This range is provided by Alldus.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.Direct message the jo...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer – Metrology Systems

    Senior Software Engineer – Metrology Systems

    KLA • Milpitas, CA, United States
    [job_card.full_time]
    A global leader in semiconductor technologies is seeking a Software Engineer to develop and maintain software for metrology solutions. You'll work collaboratively with international teams and partic...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer - Code Understanding & Transformers

    Senior ML Engineer - Code Understanding & Transformers

    ShareThis • Palo Alto, California, United States
    [job_card.full_time]
    A technology company in Palo Alto seeks a Sr.Machine Learning Engineer to design, train, and evaluate machine learning models for code understanding and generation. Responsibilities include developi...[show_more]
    [last_updated.last_updated_1_hour] • [promoted] • [new]
    Senior Software Engineer, Metrology Systems

    Senior Software Engineer, Metrology Systems

    KLA-Belgium • Milpitas, CA, United States
    [job_card.full_time]
    A global leader in semiconductor manufacturing is seeking a Software Engineer in Milpitas, California.This role involves developing and maintaining software for complex metrology systems.Candidates...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer — GenAI & RAG Platforms

    Senior ML Engineer — GenAI & RAG Platforms

    Cisco Systems • Santa Clara, CA, United States
    [job_card.full_time]
    A global technology leader seeks an experienced engineer to develop AI-driven services and APIs for its hybrid, multi-cloud environment. Ideal candidates will have 5+ years in backend or distributed...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer : Sampling & Evaluation Pipelines

    Senior Software Engineer : Sampling & Evaluation Pipelines

    Waymo • Mountain View, CA, United States
    [job_card.full_time]
    A leading autonomous technology company based in California seeks an experienced engineer to develop importance sampling techniques for evaluation pipelines. You will collaborate with teams to optim...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Senior ML Engineer (Level 5) — Deploy & Scale Models

    Senior ML Engineer (Level 5) — Deploy & Scale Models

    Minimal • Palo Alto, CA, United States
    [job_card.full_time]
    A leading technology company in Palo Alto is seeking a Machine Learning Engineer to build and deploy models for core products. Responsibilities include applying modern ML techniques and collaboratin...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior ML Engineer, Level 5 — Ranking & Recommendations

    Senior ML Engineer, Level 5 — Ranking & Recommendations

    Snap Inc. • Palo Alto, CA, United States
    [job_card.full_time]
    An innovative technology company is seeking a Machine Learning Engineer to join their dynamic team.In this role, you will leverage your expertise in machine learning to create impactful models that...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer, Observability

    Senior Software Engineer, Observability

    Expedia, Inc. • San Jose, CA, United States
    [job_card.full_time]
    Expedia Group brands power global travel for everyone, everywhere.We design cutting-edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners.Our...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer - AI / ML Infra

    Senior Software Engineer - AI / ML Infra

    GEICO • Palo Alto, CA, United States
    [job_card.full_time]
    At GEICO, we offer a rewarding career where your ambitions are met with endless possibilities.Every day we honor our iconic brand by offering quality coverage to millions of customers and being the...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal AI / ML System Software Engineer

    Principal AI / ML System Software Engineer

    d-Matrix • Santa Clara, CA, United States
    [job_card.full_time]
    AI to power the transformation of technology.We are at the forefront of software and hardware innovation, pushing the boundaries of what is possible. We value humility and believe in direct communic...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer — Pricing Platform & Systems

    Senior ML Engineer — Pricing Platform & Systems

    Uber • Sunnyvale, CA, United States
    [job_card.full_time]
    A leading technology company is seeking a Senior Staff Machine Learning Engineer to lead the development of advanced ML systems for courier pricing. The role demands extensive experience in building...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Software Engineer, Configuration & Experiments

    Senior Software Engineer, Configuration & Experiments

    Futureshaper.com • Mountain View, CA, United States
    [job_card.full_time]
    Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Software Engineer – Autonomy Evaluation Platform

    Senior Software Engineer – Autonomy Evaluation Platform

    General Motors • Mountain View, CA, United States
    [job_card.full_time]
    As a Senior Software Engineer within the Evaluation team under the Simulation, Evaluation, and Data organization, you will shape the design and delivery of the evaluation strategy for GM’s autonomo...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    ML Evaluation Engineer - Multimodal Foundation Models

    ML Evaluation Engineer - Multimodal Foundation Models

    Apple • Cupertino, CA, United States
    [job_card.full_time]
    A leading technology company is seeking a Machine Learning Engineer to develop user-facing conversational features.You will design evaluation processes, collaborate with researchers, and conduct st...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Software Engineer, Systems Modeling

    Software Engineer, Systems Modeling

    PsiQuantum • Milpitas, California, United States
    [job_card.full_time]
    PsiQuantum'smission is to build the first useful quantum computers-machines capable of delivering the breakthroughs the field has long promised. Since our founding in 2016, our singular focus has be...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Hybrid ML Engineer — Large-Scale Recommender Systems

    Hybrid ML Engineer — Large-Scale Recommender Systems

    Pinterest • Palo Alto, CA, United States
    [job_card.full_time]
    A leading social media platform in the United States is seeking Machine Learning Engineers to innovate and enhance user experiences. Candidates will contribute to cutting-edge projects in machine le...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]