Program evaluator h1.location_city

Mercor is seeking detail-oriented writing experts to contribute to a high-impact AI research project with a leading lab. Freelancers will author prompt–golden answer pairs that train and evaluate advanced language models. This is a short-term, flexible opportunity for professionals with strong academic backgrounds and a knack for instructional clarity. Ideal for those who enjoy distilling complex concepts into well-crafted text.

• Job Details : - Design and Optimize Prompts : Create detailed prompts with multiple constraints and instructions. - Define and Document Evaluation Standards : Establish high-level expectations for correct responses in general consumer contexts, and develop comprehensive rubric. - Conduct Model Testing and Grading : Run prompts through models and assess preliminary outputs against expectations. - Support Benchmarking and Quality Assurance : Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor, maintaining consistency and reliability before integration into official benchmarks. ### Minimum Qualifications : - BS or BA from a reputable institution completed or in progress - Strong writing and critical thinking skills. - Ability to work independently and meet deadlines. - Significant familiarity with ChatGPT or similar tools for personal decision-making or hobbies / general interests. - US or Canada based. ### Preferred Qualifications : - Experience in teaching or research. ### Application & Onboarding Process : - Complete an AI-led interview, this should take around 15 minutes. - Complete a 45-minute written assessment that will guide you through writing rubrics. - If selected, you will be invited to work on the project. ### More Details About This Role : - Expect to contribute at least 20 hours per week . - Expect a commitment of around 2 month. - You’ll be working in a structured project environment with clear goals and tools.