Lead Specialist, Measurement

Lead Specialist, Measurement

This role aligns to industry level title of Lead Specialist, Applied AI Measurement & Learning Systems

Measurement & Learning | School Assessment

Role Overview

We are seeking a Lead Specialist, Measurement to support the design, validation, and continuous improvement of intelligent measurement systems and AI-assisted assessment and learning workflows.

This role sits within the Measurement & Learning team and operates at the intersection of Measurement, Content Development, Learning Science, Technology, and AI Science, with responsibility for helping ensure AI-assisted systems consistently produce high-quality, valid, scalable, and instructionally meaningful outputs across assessment and learning use cases.

The role primarily supports AI-assisted assessment item development workflows, while also contributing to adjacent applications including AI-generated practice items, learning materials aligned to standards, AI-powered learning and feedback experiences, instructional guidance systems, and emerging approaches to evidencing student skills and competencies.

This role also contributes to the advancement of next-generation assessment approaches, including innovative item types, AI-assisted measurement opportunities, and new ways of assessing reasoning, applied learning, and student competencies.

AI-Assisted Assessment Item Development

• Lead and support the design and continuous improvement of AI-assisted assessment item development workflows embedded within broader assessment and learning systems
• Translate SME and measurement intent into structured, reusable AI-assisted workflows and agentic frameworks that support scalable, consistent, and auditable content generation and enhancement
• Use AI-assisted tools and rapid prototyping approaches to explore, test, and iterate on emerging assessment, content, learning, and instructional workflow innovations
• Support assessment and learning content across Mathematics, English Language Arts, Science, Social Studies, K12 grade levels, TEIs, oral and constructed-response interactions, translation and multilingual content, and alignment to academic content standards
• Support fact checking, grammar, style refinement, distractor rationale strengthening, and key validation
• Ensure AI-assisted outputs meet expectations for validity, accuracy, fairness, accessibility, and instructional appropriateness

Assessment Innovation & Learning Applications

• Lead and support validation approaches for AI-assisted learning and assessment applications beyond core assessment item development workflows including
• Generation and evaluation of academically similar items and learning opportunities for student practice
• AI-assisted learning materials aligned to standards, learning goals, and instructional intent
• Innovative item types and new ways of assessing student skills, competencies, reasoning, communication, and applied learning
• Support validation and quality approaches for content and interactions used within AI-assisted online learning guides, instructional supports, and learner-facing experiences
• Help ensure AI-assisted assessment, feedback, and learning supports work together in educationally meaningful and instructionally aligned ways
• Contribute to the advancement of emerging intelligent measurement and learning system capabilities across assessment and instructional contexts

Validation, Monitoring & Quality Frameworks

• Design and oversee validation studies evaluating the quality, appropriateness, effectiveness, interpretability, and educational defensibility of AI-assisted systems, outputs, and workflows
• Define evaluation criteria in collaboration with psychometricians, learning scientists, and content leaders
• Establish processes for ongoing monitoring and continuous improvement
• Develop frameworks for ongoing monitoring, drift detection, and continuous improvement of AI-assisted systems over time
• Support research and validation frameworks grounded in human expertise, empirical evidence, transparency, and continuous improvement
• Support technical documentation and evidence generation related to the validity, quality, effectiveness, interpretability, and performance of AI-assisted systems and outputs

Research, Documentation & Thought Leadership

• Contribute to internal and external thought leadership related to intelligent measurement systems and AI-assisted learning technologies
• Support dissemination of research findings, validation approaches, and innovation frameworks through technical reports, presentations, and publications
• Help translate complex AI, learning, and measurement concepts into clear guidance for diverse stakeholder groups
• Provide guidance to teams on responsible and effective use of AI-assisted systems within assessment and learning workflows

Cross-Functional Collaboration

• Partner with Technology teams implementing AI infrastructure and intelligent systems
• Partner with AI Science teams building models and systems for scoring, reporting, feedback, guidance, and learning supports
• Partner with content teams authoring assessment and learning content
• Collaborate with research and measurement teams supporting learner models, recommender systems, and AI-guided instructional experiences
• Ensure alignment between AI-assisted content generation, downstream scoring systems, instructional guidance systems, and learner-facing experiences
• Support coherence across assessment, learning, scoring, feedback, and instructional systems

Required Qualifications

• Advanced degree in Educational Measurement, Psychometrics, Learning Sciences, Educational Technology, Artificial Intelligence, Educational Data Mining, Learning Analytics, Cognitive Science, Computer Science, Statistics, or a related field
• Experience working with assessment item development, learning systems, or educational content workflows
• Experience collaborating across cross-functional environments including content, measurement, technology, product, or AI-related teams
• Strong understanding of assessment quality principles, including validity, alignment, fairness, accessibility, and instructional appropriateness
• Ability to think systematically about AI-assisted workflows across content generation, scoring, feedback, instructional guidance, and learner-facing experiences
• Ability to evaluate where AI-assisted approaches can effectively support assessment and learning workflows and where human expertise remains essential
• Strong analytical and problem-solving skills with attention to quality, defensibility, and continuous improvement
• Ability to design or support rigorous validation and monitoring approaches for AI-assisted systems and workflows
• Comfort exploring, testing, and prototyping AI-assisted approaches using emerging tools and technologies
• Strong written and verbal communication skills, including the ability to translate complex AI and measurement concepts for diverse audiences

Preferred Qualifications

• Experience supporting AI-assisted assessment, scoring, feedback, tutoring, instructional guidance, or content-generation systems
• Experience designing or evaluating validation studies for AI-assisted educational systems or workflows
• Familiarity with psychometric principles and educational measurement concepts
• Experience with innovative item types, technology-enhanced items (TEIs), oral or constructed-response assessment, or competency-based assessment approaches
• Experience contributing to technical documentation, dissemination materials, presentations, publications, or thought leadership initiatives
• Familiarity with recommender systems, learner models, domain models, or AI-guided instructional experiences
• Experience identifying workflow risks, edge cases, drift concerns, or unintended consequences within AI-assisted systems
• Experience leveraging AI-assisted tools to accelerate workflow exploration, prototyping, innovation, or analysis
• Curiosity about emerging approaches to assessing skills, competencies, reasoning, and applied learning through AI-assisted systems
• Experience operating in fast-moving innovation environments where systems and workflows continuously evolve

Design Philosophy

• Human expertise as a core system foundation
• Empirical validation over assumed performance
• Reuse, transparency, and system coherence
• Ongoing monitoring and improvement of AI-assisted workflows
• Innovation grounded in educational usefulness and defensible measurement principles

Why This Role Matters

This role is foundational to building intelligent assessment and learning systems that educators, partners, and learners can trust. By anchoring AI-assisted systems in rigorous validation, ongoing monitoring, educational usefulness, and human expertise, this leader will help ensure AI-assisted approaches empower assessment and learning without compromising quality, validity, fairness, or instructional integrity.