Senior Applied Researcher 

Learning & Assessment Sciences

Team: Human Data

 

Prolific

Prolific isn’t just powering AI development—we’re redefining it. While foundational AI technologies are becoming commoditized, Polific’s human data infrastructure delivers the high-quality, diverse data needed to differentiate the next generation of AI models. Through our platform, we empower researchers and companies to access a global pool of participants, ensuring cutting-edge AI research and training grounded in ethical and inclusive practices.

 

The role

We’re looking for a forward-thinking TBD to build and deploy proprietary assessment, profiling, and training programs for Prolific participants. You’ll design scalable and validated solutions, delivered via our platform, to comprehensively assess and evaluate  the relevant behavioral and psychometric qualities of our participants, with a view to developing bespoke participant pools that can be leveraged for a variety of research use cases.

This role demands expertise in psychometric and instructional design, content development, and project management, alongside the ability to manage multiple stakeholders and collaborate cross-functionally. You’ll be pivotal in developing a deeper understanding of our global participant pool and ensuring that our participants are equipped with the skills to deliver high-quality data while staying agile to meet the evolving demands of our customers.

 

What you’ll be doing

• Assisting with the development of robust, reliable, and validated assessment frameworks to accredit participants as AI taskers and domain experts, ensuring high task quality.
• Creating adaptable assessment frameworks for skill and expertise verification and competency certification, tailored to client specifications.
• Developing multi-modal, interactive training content to upskill participants and engage specialist participants, optimized for deployment via Prolific’s platform.
• Build scalable, role-specific skill development pathways for onboarding, accreditation, and ongoing learning.
• Partnering with platform developers to seamlessly integrate training solutions into existing workflows, ensuring a cohesive and intuitive participant experience.
• Collaborate with cross-functional teams to align programs with operational goals and customer requirements.
• Monitor participant performance through platform analytics, identifying skill gaps and refining training content accordingly.
• Regularly update training programs at pace to meet evolving AI customer needs and industry standards.
• Develop scalable solutions for continuous upskilling, leveraging emerging technologies to enhance participant readiness.
• Experiment with cutting-edge learning methodologies and technologies to push the boundaries of instructional design.
• Integrate innovative assessment modalities, such as adaptive testing and real-time feedback systems, to enhance participant engagement and outcomes.
• Lead efforts to ensure that all assessment training programs are highly engaging, scalable, and accessible to a diverse, global participant base.
• Manage relationships with internal stakeholders, including data, science, operations, and customer success teams, to align training content with task complexity and project needs.
• Act as the primary point of contact for subject matter experts (SMEs) who will be used to assist with participant grades and assessments, driving the creation of accurate and relevant content.
• Facilitate train-the-trainer (T3) sessions and oversee live-led training initiatives when required.

    

What You'll Bring

• Experience in psychometric and instructional testing design in high-growth, tech-driven environments.
• Expertise in assessment and instructional design for competency-based assessments and certifications.
• Experience in LLMs/ML and AI, including a deep understanding of assessment and grading systems for AI research, training, and development.
• Experience deploying learning programs through proprietary platforms or similar ecosystems.
• Content Creation: Demonstrable proficiency with e-learning and training platforms.
• Psychometric assessment: Deep understanding of how to design, administer, and interpret psychometric tests to evaluate cognitive abilities, personality traits, and behavioral tendencies effectively.
• Assessment Design: Strong understanding of assessment frameworks for skill verification, with experience designing task-based evaluations.
• Data Analytics: Ability to leverage data-driven insights to refine and improve learning programs.
• Collaboration & project management: Skilled at project management when tasks involve cross-functional stakeholders.
• Innovation: Passion for testing new learning methodologies and modalities to drive participant engagement and success

Preferred Qualifications

• Postgraduate degree in a subject related to social or behavioral science alongside exposure to cognitive and behavioral testing strategies and measures.
• Experience in training solutions tailored for AI/ML lifecycle tasks, such as data labeling, annotation, and model evaluation.
• Familiarity with ethical AI practices and the challenges of scaling human data operations.
• Knowledge of Prolific’s platform and participant ecosystem, including task complexity and curation processes.

 

Why Prolific is a great place to work

At Prolific, you’ll have the chance to shape the future of human-AI collaboration by empowering participants to contribute to cutting-edge AI research and training. You’ll work in a fast-paced, purpose-driven environment that values impact, inclusion, and innovation.

We offer competitive compensation, benefits, and flexible remote working options. Be part of a team that’s pioneering a responsible AI landscape, ensuring that the technology of tomorrow understands and respects the full spectrum of human experience.

 

Links to more information on Prolific

Benefits 

External Handbook 

Website 

Youtube 

 

Privacy Statement

By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.