Talent.com

Program evaluation Jobs in San Francisco, CA

Create a job alert for this search

Program evaluation • san francisco ca

Last updated: 1 day ago

Medical Evaluation Specialist - Remote

YO AI LabsSan Francisco, CA, United States
RemoteRemote
Full-timeFull-time
Quick Apply

Job Title: Medical Evaluation Specialist.Medical Evaluation Specialists.In this role, you will create and validate challenging medical questions and answers designed to test advanced clinical reaso... Show more

Program Manager

AECOMOakland, California, United States
Full-time +1Full-time +1

AECOM is seeking an experienced.Oil and Gas business line in Oakland, California.This role will lead the successful delivery of complex environmental projects for clients in the refining, industria... Show more

Program Manager

RejuvenationSan Francisco, CA, United States
Full-timeFull-time

Program Manager, Inventory Planning Strategy.The Program Manager, Inventory Planning Strategy, is responsible for leading the assessment, design, documentation, and implementation of best-practice ... Show more

Program Manager

Omni InclusiveSan Francisco, CA, United States
Full-timeFull-time

Program/Project Manager Specializing In Mergers & Acquisitions (M&A).Strong Program/Project Manager specializing in Mergers & Acquisitions (M&A) with a proven track record in successfully managing ... Show more

Evaluation Manager

Children's Council of San FranciscoSan Francisco, CA, United States
Full-timeFull-time

For more than 50 years, Children's Council has been the heart of childcare and early education in San Francisco, advancing the belief that high-quality childcare can transform lives and communities... Show more

Director, Business Development - Search & Evaluation

Formation Bio (Formerly TrailSpark)San Francisco, CA, United States
Full-timeFull-time

Director, Business Development - Search & Evaluation.Formation Bio is a tech and AI driven pharma company differentiated by radically more efficient drug development.Advancements in AI and drug dis... Show more

Program Manager

Aditi ConsultingSan Francisco, CA, United States
$70.00–$73.00 hourly$70.00–$73.00 hourly
Full-timeFull-time

Summary: Biocompatibility PM is expected to have a good understanding of consumer electronics, analytical testing, demonstrated experience in program management and has worked in a collaborative cr... Show more

Evaluation and Learning Director

CYCSFSan Francisco, CA, US
$90,000.00–$105,000.00 yearly$90,000.00–$105,000.00 yearly
Full-timeFull-time
Quick Apply

JOB ANNOUNCEMENT For more than 54 years, Community Youth Center of San Francisco (CYC) has given San Francisco youth a sense of belonging and the tools and experiences they need to succeed in life.... Show more

Evaluation and Learning Director

CYCSan Francisco, CA, United States
$90,000.00–$105,000.00 yearly$90,000.00–$105,000.00 yearly
Full-timeFull-time

Evaluation And Learning Director.For more than 54 years, Community Youth Center of San Francisco (CYC) has given San Francisco youth a sense of belonging and the tools and experiences they need to ... Show more

Program Manager

Zortech SolutionsSan Francisco, CA, United States
Full-timeFull-time

Location: Denver CO or San Francisco CA (On Site couple of days a week).Develop and manage project governance plans, project schedules, success criteria, and project milestones.Own and drive end-to... Show more

Program Director

The Posse FoundationSan Francisco, CA, United States
$70,000.00–$75,000.00 yearly$70,000.00–$75,000.00 yearly
Full-timeFull-time

Posse Bay Area - San Francisco, CA 94104.The primary responsibilities of the program director are to oversee and manage the Posse program for up to six (6) university partners; manage a staff of up... Show more

Clinical Expert - Drug Evaluation

MercorSan Francisco, California, United States
RemoteRemote
Part-timePart-time
Quick Apply

Headquartered in San Francisco, our investors include.Disease-Area Clinician — Trial Endpoints & Prescribing.Create and critique rubrics that analyze commercial drugs and drug-development progr... Show more

Diagnostic Supervisor (Remote) - Veterans Evaluation Services

MaximusSan Francisco, US
$69,250.00 yearly$69,250.00 yearly
RemoteRemote
Full-timeFull-time

Essential Duties and Responsibilities: - Supervise, instruct, and support the Diagnostics team: Diagnostic Cappers, Diagnostic Facility Recruiters, Diagnostic Schedulers, Diagnostic Uploader... Show more

Senior Specialist / Manager, AI Safety Data and Evaluation Program

Lila SciencesSan Francisco, CA, United States
$88,000.00 yearly$88,000.00 yearly
Full-timeFull-time

We're building a talent-dense, high-agency AI safety team at Lila that will engage all core teams within the organization (science, model training, lab integration, etc.The initial focus of this te... Show more

Merchant Mariner Evaluation Specialist (Deck)

TSAOakland, CA, United States
Full-timeFull-time

Merchant Mariner Evaluation Specialist (Deck).You will serve as a Merchant Mariner Evaluation Specialist (Deck) and be responsible for providing advice and administration on the regulatory provisio... Show more

Program Manager

ArtechBrisbane, CA, United States
Full-timeFull-time

Location: Brisbane, CA Hybrid Duration: 12 Months Pay Range: $130-148/hr.At the Associate Director level, the Project Manager (PM) is accountable for one or more Global Program Teams (GPTs) and as... Show more

Agent Evaluation Infrastructure Engineer

MaxIT Consulting - Max Corporate GroupSan Francisco, California, United States
Full-timeFull-time
Quick Apply

San Francisco, California | Primarily On-site.Agent Evaluation Infrastructure Engineer.You will work on the engineering and research problems behind realistic agent environments, post-training syst... Show more

Program Manager

SilaAlameda, CA, United States
$168,000.00–$212,000.00 yearly$168,000.00–$212,000.00 yearly
Full-timeFull-time

We are Sila, a next-generation battery materials company.Our mission is to power the world's transition to clean energy.To create this future, our team is building a better lithium-ion battery from... Show more

Program Manager

MasTecOakland, CA, United States
$50.67–$76.01 hourly$50.67–$76.01 hourly
Full-timeFull-time

Inclusion & Belonging Program Manager, Principal.Intren West (IW) Professional Services Construction Management Solutions provides strategic program management, construction management, and operati... Show more

Remote Software Developer – AI Research & Evaluation

TuringSan Francisco, California, United States
RemoteRemote
Full-timeFull-time
Quick Apply

Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems.Turing support... Show more

People also ask
Medical Evaluation Specialist - Remote

Medical Evaluation Specialist - Remote

YO AI LabsSan Francisco, CA, United States
4 days ago
Job type
  • Full-time
  • Remote
  • Quick Apply
Job description

Job Title: Medical Evaluation Specialist

Role Type: Contractor
Location: Remote

Job Overview

We are seeking Medical Evaluation Specialists, including medical students, residents, physicians, and qualified biomedical professionals, to contribute clinical expertise to a project focused on evaluating and improving next-generation AI systems.

In this role, you will create and validate challenging medical questions and answers designed to test advanced clinical reasoning. Your work will help assess whether AI systems can accurately interpret medical evidence, synthesize complex information, and apply nuanced clinical judgment.

No prior AI experience is required—your medical knowledge, research skills, and clinical reasoning are what matter most.

Scope of Work

  • Create original, high-difficulty medical question-and-answer pairs covering areas such as diagnosis, pathophysiology, pharmacology, clinical guidelines, and clinical decision-making.
  • Develop questions that require synthesis, interpretation, and genuine medical reasoning rather than simple factual recall.
  • Research and verify answers using primary literature, clinical guidelines, systematic reviews, and authoritative medical references.
  • Document the rationale behind answers and provide appropriate supporting citations.
  • Evaluate AI-generated responses for clinical accuracy, completeness, reasoning quality, and adherence to available evidence.
  • Identify questions that are too straightforward and refine them to increase complexity while maintaining clinical validity.
  • Review questions and answers for ambiguity, unsupported assumptions, factual errors, and inconsistencies.
  • Ensure all deliverables are written with clarity, precision, and defensibility.
  • Incorporate reviewer feedback and adapt work to evolving project guidelines and quality standards.
  • Contribute insights that help establish rigorous benchmarks for medical AI evaluation.

Required Skills

  • Medical Training
  • Research & Source Triangulation
  • Attention to Detail
  • Written Precision
  • Analytical Thinking
  • Clinical Reasoning
  • Medical Literature Review
  • Evidence Synthesis
  • Self-Direction & Reliability
  • Medical Question Development

Preferred Qualifications

  • Current or recent medical training or clinical practice as a medical student, resident, or physician, or equivalent expertise in a relevant biomedical discipline.
  • Demonstrated ability to locate, interpret, compare, and synthesize information from primary research and clinical guidelines.
  • Strong written English skills with exceptional attention to detail.
  • Ability to create engaging, well-structured, challenging, and clinically nuanced questions.
  • Strong understanding of evidence-based medicine and clinical reasoning.
  • Ability to work independently, manage deadlines, and consistently deliver high-quality work in a remote environment.
  • Previous experience with medical question writing, peer review, clinical education, medical content evaluation, or research is a plus.
  • Familiarity with AI/ML systems, AI evaluation, or medical AI applications is advantageous but not required.

Compensation Structure

Compensation is output-based, with contributors paid per task that meets the applicable project specifications and quality standards. The time required to complete each task may vary depending on experience and individual workflow.

Minimum submission requirements apply, and selected contributors are expected to complete a minimum number of tasks per week.

Start Timeline & Availability

Roles are typically filled within 48 hours. Selected contributors should be prepared to begin their first assignments within 24–48 hours of completing onboarding.