Talent.com
Programmers.io
Data Annotation EngineerProgrammers.io • Louisville, KY, United States
Search for other jobs
Data Annotation Engineer

Data Annotation Engineer

Programmers.io • Louisville, KY, United States
13 hours ago
Job type
  • Full-time
  • Quick Apply
Job description
  • Lead the day-to-day operation of the AI annotation program.
  • Manage a team of annotators responsible for reviewing customer conversations and AI interactions.
  • Develop annotation guidelines, procedures, and quality standards.
  • Establish calibration programs and quality assurance processes to ensure consistency across reviewers.
  • Partner with Data Scientists to create and maintain gold-standard datasets.
  • Monitor annotation accuracy, throughput, and quality metrics.
  • Coordinate reporting and deliverables for Legal, Compliance, Responsible AI, and executive stakeholders.
  • Manage annotation workflows and tooling including setting up annotation jobs, running data processing scripts, and testing annotation UIs for quality before job launch.
  • Plan capacity requirements to support seasonal increases in review volume.
  • Support future expansion into multilingual evaluation programs.
  • Identify process improvements that increase efficiency and annotation quality.
  • Bachelor's degree or equivalent experience in Linguistics/Psychology.
  • 5+ years of experience in program management, operations, quality assurance, data labeling, or related fields.
  • Experience leading teams and managing operational workflows.
  • Strong organizational and stakeholder management skills.
  • Ability to analyze quality metrics and drive continuous improvement initiatives.
  • Excellent written and verbal communication skills.
  • Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.
  • Strong organizational and stakeholder management skills.
  • Ability to analyze quality metrics and drive continuous improvement initiatives.
  • Excellent written and verbal communication skills.
  • Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.
  • Experience supporting AI, machine learning, or data annotation programs.
  • Familiarity with Responsible AI, compliance, legal review, or governance processes.
  • Experience developing operational standards and quality frameworks.
  • Experience managing vendor or contractor resources.
Create a job alert for this search

Data Annotation Engineer • Louisville, KY, United States