QA Lead owns annotation quality : define gold standards, run audits, measure IAA, lead remediation and training for annotator panels. Critical to maintain high dataset reliability.
Responsibilities
Core responsibilities
- Design QA processes, gold datasets and adjudication workflows.
- Monitor IAA, gold check pass rates and annotator drift.
- Coach annotators and run calibration sessions.
- Author / update annotation guidelines; work with clients on edge cases.
Qualifications
Skills & experience
Must : strong analytical skills, prior QC / annotation experience.Preferable : familiarity with inter-rater stats and dataset QA tooling.Nice-to-have : domain knowledge for specialized datasets.