About the Role
We are seeking a motivated individual to join our dynamic generative AI evaluation team. We focus on assessing and validating AI models that support healthcare decision-making tools. Our team ensures that our AI solutions meet rigorous standards for accuracy, fairness, and clinical relevance. The successful candidate will help evaluate model performance, analyze output quality, and ensure our AI solutions align with healthcare best practices.
The successful candidate has relevant experience with AI model evaluation, including performance metrics analysis, output validation, and data visualization. They enjoy collaborating with cross-functional teams including AI researchers, product managers, clinicians, and engineers to evaluate AI models, analyze their outputs, and improve their performance in healthcare contexts.
Responsibilities
- Leading requirements gathering and development of analytics frameworks for evaluating generative AI models in healthcare applications
- Leading and defining the strategic direction for AI evaluation methodologies in their product area
- Managing the analytics roadmap for model evaluation. Collaborating with internal and external stakeholders to manage evaluation timelines and deliverables
- Defining, developing, and deploying queries and metrics to assess AI model performance and output quality
- Leading data model creation and maintenance for AI evaluation frameworks and monitoring systems
- Conducting comprehensive QA of AI outputs and participating in risk analysis and mitigation
- Understanding how security and privacy laws such as HIPAA affect AI deployment in healthcare. Understanding how to handle sensitive data including PHI and PII in AI contexts
Requirements
- Extensive and relevant experience in data analytics
- Demonstrated thought leadership in analytics strategy and enterprise-wide project implementation
- Have current experience in a data analytics role
- Have expertise in using and querying relational database management systems such as Microsoft SQL Server, Snowflake.
- Have expertise in using BI tools such as Tableau or PowerBI
Working for you
We know that your wellbeing and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer
- Health Benefits: Comprehensive, multi-carrier program for medical, dental and vision benefits
- Retirement Benefits: 401(k) with match and an Employee Share Purchase Plan
- Wellbeing: Wellness platform with incentives, Headspace app subscription, Employee Assistance and Time-off Programs
- Short-and-Long Term Disability, Life and Accidental Death Insurance, Critical Illness, and Hospital Indemnity
- Family Benefits, including bonding and family care leaves, adoption and surrogacy benefits
- Health Savings, Health Care, Dependent Care and Commuter Spending Accounts
- Up to two days of paid leave each to participate in Employee Resource Groups and to volunteer with your charity of choice
U.S. National Base Pay Range: $102,800 - $171,300. Geographic differentials may apply in some locations to better reflect local market rates. This job is eligible for an annual incentive bonus. We know that your wellbeing and happiness are key to a long and successful career. These are some of the benefits we are delighted to offer: ● Health Benefits: Comprehensive, multi-carrier program for medical, dental and vision benefits ● Retirement Benefits: 401(k) with match and an Employee Share Purchase Plan ● Wellbeing: Wellness platform with incentives, Headspace app subscription, Employee Assistance and Time-off Programs ● Short-and-Long Term Disability, Life and Accidental Death Insurance, Critical Illness, and Hospital Indemnity ● Family Benefits, including bonding and family care leaves, adoption and surrogacy benefits ● Health Savings, Health Care, Dependent Care and Commuter Spending Accounts ● In addition to annual Paid Time Off, we offer up to two days of paid leave each to participate in Employee Resource Groups and to volunteer with your charity of choice
We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our or please contact 1-855-833-5120.
Criminals may pose as recruiters asking for money or personal information. We never request money or banking details from job applicants. Learn more about spotting and avoiding scams .
Please read our .