Talent.com
Principal Engineer, Behavior Understanding and Evaluation
Principal Engineer, Behavior Understanding and EvaluationMotional • Boston, MA, US
Principal Engineer, Behavior Understanding and Evaluation

Principal Engineer, Behavior Understanding and Evaluation

Motional • Boston, MA, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Job Description

Job Description

Mission Summary :

The AV Behavior Design and Evaluation Team is responsible for owning the design and assessment of the AV behavior utilizing the concept of AV behavior competencies. The team relies on systematic decomposition of AV behavior along with data driven assessment utilizing data from on-road and simulation test modalities. We are responsible for building and operating robust, large-scale evaluation systems, alongside sophisticated simulation workflows and insightful diagnostic tools. By combining expert human judgment with advanced data driven techniques, we evaluate the system behavior performance.

As the team is looking to scale to support more and more behavioral competencies, we are looking for passionate Engineers to help build a robust framework to leverage the best of agile systems engineering and data driven performance assessment to achieve our ambitious goals in scaling our system design and evaluation processes. We are looking to work with people who are driven to design robust systems and transform complex data into measurable progress toward a safer, smarter transportation future.

This person will get to build relationships with the team as well cross-functional across organization to help build a cohesive understanding AV performance as applicable to the Behavioral Competencies.

What you'll do :

  • Decompose AV operational design domain (ODD) into AV capabilities
  • Define precise atomic system and subsystem behaviors and interfaces required to realize these capabilities.
  • Collaborate extensively with development teams (Autonomy, Infrastructure, Cloud Applications) to drive the design and implementation of new features that enable defined AV capabilities and enhance the overall product experience.
  • Strategically plan and execute system-level testing campaigns. This includes leveraging advanced simulation tools and coordinating tests on closed course tracks and proving grounds to ensure the system meets functional and performance requirements.
  • Effectively utilize existing metrics and propose new metrics to evaluate AV performance via simulation and on-road testing modalities.
  • Pioneer the definition and implementation of key evaluation metrics and reward functions, driving autonomy performance across safety, comfort, and assertiveness
  • Leverage a combination of learned methods and metrics along with classical metrics to stretch the horizon of performance evaluation
  • Characterize the performance target against parameters used to define AV capabilities to provide nuance in performance evaluation
  • Deliver strategic insights and recommendations based on AV capability testing by building meaningful visualizations
  • Develop and maintain comprehensive dashboards and reporting tools. Distill on-road performance data into actionable feedback for engineering teams and create high-level, insightful summaries for executive leadership.
  • Focus on automating simulation result analysis processes and gaps, removing manual processes where applicable
  • Partner cross-functionally with Behaviors, Actions, Simulation, System Engineering, Research, and other Onboard / Offboard Software Teams on evaluation strategies

What we're looking for :

  • Bachelor's degree and 7+ years of experience in one of Systems Eng, Data Science, Statistics or Computer Science with some form of experience in rest of the domains
  • At least 5 years of experience in leading and scaling data driven evaluation of autonomous driving behavior or similar safety critical emerging technology
  • Proven technical experience in establishing and executing evaluation strategies
  • A strong foundation in scientific and statistical methodologies
  • Fluency in SQL or similar query languages
  • Fluency in data visualization techniques
  • Fluency in Python
  • Proficiency in crafting data-driven reports and technical strategy documents for executive decision-making
  • Excellent interpersonal skills to build relationships with other teams
  • Excellent communication skills; effective in communicating complex concepts and driving resolutions
  • Self-starter who is motivated to set and achieve goals and takes initiative to drive ambiguous tasks / projects to completion
  • I deally, you also have :

  • Masters degree in Data Science or Computer Science
  • Experience in mentoring and leadership development
  • Experience collaborating in an agile development environment
  • A background in shaping product roadmaps
  • Familiarity with Reinforcement Learning (RL) with Human Feedback
  • Specialized expertise in creating expansive datasets
  • Experience collaborating in an agile development environment
  • Working with a global workforce that spans multiple timezones
  • We encourage a hybrid schedule with in-office time at one of our locations in Boston, Pittsburgh, or Las Vegas to support collaboration, or this role can be fully remote.

    The salary range for this role is an estimate based on a wide range of compensation factors including but not limited to specific skills, experience and expertise, role location, certifications, licenses, and business needs. The estimated compensation range listed in this job posting reflects base salary only. This role may include additional forms of compensation such as a bonus or company equity. The recruiter assigned to this role can share more information about the specific compensation and benefit details associated with this role during the hiring process.

    Candidates for certain positions are eligible to participate in Motional's benefits program. Motional's benefits include but are not limited to medical, dental, vision, 401k with a company match, health saving accounts, life insurance, pet insurance, and more.

    Salary Range

    $175,000—$234,250 USD

    Motional is a driverless technology company making autonomous vehicles a safe, reliable, and accessible reality. We're driven by something more.

    Our journey is always people first.

    We aren't just developing driverless cars; we're creating safer roadways, more equitable transportation options, and making our communities better places to live, work, and connect. Our team is made up of engineers, researchers, innovators, dreamers and doers, who are creating a technology with the potential to transform the way we move.

    Higher purpose, greater impact.

    We're creating first-of-its-kind technology that will transform transportation. To do so successfully, we must design for everyone in our cities and on our roads. We believe in building a great place to work through a progressive, global culture that is diverse, inclusive, and ensures people feel valued at every level of the organization. Diversity helps us to see the world differently; it's not only good for our business, it's the right thing to do.

    Scale up, not starting up.

    Our team is behind some of the industry's largest leaps forward, including the first fully-autonomous cross-country drive in the U.S, the launch of the world's first robotaxi pilot, and operation of the world's longest-standing public robotaxi fleet. We're driven to scale; we're moving towards commercialization of our technology, and we need team members who are ready to embrace change and challenges.

    Formed as a joint venture between Hyundai Motor Group and Aptiv, Motional is fundamentally changing how people move through their lives. Headquartered in Boston, Motional has operations in the U.S and Asia. For more information, visit www.Motional.com and follow us on Twitter, LinkedIn, Instagram and YouTube.

    Motional AD Inc. is an EOE. We celebrate diversity and are committed to creating an inclusive environment for all employees. To comply with Federal Law, we participate in E-Verify. All newly-hired employees are queried through this electronic system established by the DHS and the SSA to verify their identity and employment eligibility.

    [job_alerts.create_a_job]

    Principal Engineer • Boston, MA, US

    [internal_linking.similar_jobs]
    Principal Solutions Engineer

    Principal Solutions Engineer

    Freshworks • Boston, MA, United States
    [job_card.full_time]
    Lead executive alignment workshops across strategic and global accounts.Defines value messaging across C-suite buying committees. Lead discovery workshops, shape customer requirements, and influence...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior / Staff / Principal Engineer

    Senior / Staff / Principal Engineer

    Canonical • Boston, MA, United States
    [job_card.full_time]
    Senior / Staff / Principal Engineer.Be among the first 25 applicants.Senior / Staff / Principal Engineer.Get AI-powered advice on this job and more exclusive features. Canonical is a leading provider of ope...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Oliver Wyman - Associate / Engagement Manager / Principal – Tech Innovation Practice (Technical [...]

    Oliver Wyman - Associate / Engagement Manager / Principal – Tech Innovation Practice (Technical [...]

    Oliver Wyman • Boston, MA, United States
    [job_card.full_time]
    Oliver Wyman - Associate / Engagement Manager / Principal – Tech Innovation Practice (Technical Architecture & Delivery Focus). Job Title : Principal – Tech Innovation Practice (Technical Architecture...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal, Systems / Mechanical Engineering

    Principal, Systems / Mechanical Engineering

    HDS Global (HDS Mercury, Inc) • Boston, MA, United States
    [job_card.full_time]
    Principal, Systems / Mechanical Engineering.Principal, Systems / Mechanical Engineering.Principal, Systems / Mechanical Engineering. Be among the first 25 applicants.Principal, Systems / Mechanical Engineer...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Engineer, Enterprise Scalability

    Principal Engineer, Enterprise Scalability

    Klaviyo Inc. • Boston, MA, United States
    [job_card.full_time]
    Principal Engineer, Enterprise Scalability.At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Deputy Director - Program Design & Evaluation

    Deputy Director - Program Design & Evaluation

    Action for Boston Community Development • Boston, MA, US
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Deputy Director of Program Design and Evaluation.ABCD’s Head Start & Children’s Services program.This person is responsible for developing and monitoring the implementation of those standardized op...[show_more]
    [last_updated.last_updated_30]
    Principal AI Engineer

    Principal AI Engineer

    Validity • Boston, MA, US
    [job_card.full_time]
    [filters_job_card.quick_apply]
    About the Role We are looking for a Principal AI Engineer who moves fast, experiments relentlessly, and ships agentic AI systems that solve real problems. This is a hybrid, office-based position whi...[show_more]
    [last_updated.last_updated_30]
    Principal Product Development Engineer (Hybrid, Boston)

    Principal Product Development Engineer (Hybrid, Boston)

    Haemonetics Corp • Boston, MA, United States
    [job_card.full_time]
    We are constantly looking to add to our core talent.If you are seeking a career that is challenging and rewarding, a work environment that is diverse and dynamic, look no further - Haemonetics is y...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Principal Research Engineer

    Senior Principal Research Engineer

    Autodesk • Boston, MA, United States
    [job_card.full_time]
    Senior Principal Research Engineer.Senior Principal Research Engineer.You will work collaboratively to create and interpret design data that enhances design and engineering workflows, while providi...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Associate Principal — Energy Economics & Strategy

    Associate Principal — Energy Economics & Strategy

    Charles River Associates • Boston, MA, United States
    [job_card.full_time]
    A premier consulting firm is seeking an Associate Principal to work in the energy sector.This role mandates managing research teams, consulting on finance and strategy, and analyzing diverse econom...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Principal System Engineer

    Senior Principal System Engineer

    Cubic Corporation • Boston, MA, United States
    [job_card.full_time] +1
    Senior Principal System Engineer page is loaded## Senior Principal System Engineerlocations : Boston, Massachusetts : United Statestime type : Full timeposted on : Posted Yesterdayjob requisiti...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Product Development Engineer (Hybrid, Boston)

    Principal Product Development Engineer (Hybrid, Boston)

    Haemonetics Software Solutions • Boston, Massachusetts, United States
    [job_card.full_time]
    Principal Product Development Engineer to join a team of scientists and engineers located in our Boston, MA Research and Development Systems Engineering Laboratory contributing to the design and de...[show_more]
    [last_updated.last_updated_1_day] • [promoted]
    Principal Engineer, Senior Manager

    Principal Engineer, Senior Manager

    Verisk • Boston, MA, United States
    [job_card.full_time]
    We help the world see new possibilities and inspire change for better tomorrows.Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, mor...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Security Engineer

    Principal Security Engineer

    PTC • Boston, MA, United States
    [job_card.full_time]
    Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.PTC-Technical Recruiting Leader | Talent Acquisition Strategist | Global Recruitment Expert | AI...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Engineer

    Principal Engineer

    Sunrise Technologies, Inc. • Raynham, Massachusetts, United States
    [job_card.full_time]
    [filters_job_card.quick_apply]
    Reporting to the Engineering Manager, this position is responsible for promoting company growth through the design, development, and introduction of new products to the market by developing the eng...[show_more]
    [last_updated.last_updated_variable_days]
    Principal Engineer, Enterprise Scalability

    Principal Engineer, Enterprise Scalability

    Klaviyo • Boston, MA, United States
    [job_card.full_time]
    At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate ...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Principal, Systems / Mechanical Engineering

    Principal, Systems / Mechanical Engineering

    P2P • Boston, MA, United States
    [job_card.full_time]
    We are seeking a seasoned systems engineering leader to join our team to.Be a systems leader that serves as a guidepost for the engineering team as the materials handling industry transforms from m...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Supplier Engineer

    Principal Supplier Engineer

    Boston Scientific Gruppe • Boston, MA, United States
    [job_card.full_time]
    The Principal Supplier Engineer, External Operations, Global sourcing APAC, will lead the SFMD(Sourced Finished Medical Device) products on-boarding process in BSC and manage the engineering change...[show_more]
    [last_updated.last_updated_30] • [promoted]