Training Specialist

The future of AI — whether in training or evaluation, classical ML or agentic workflows — starts with high-quality data.

At HumanSignal, we’re building the platform that powers the creation, curation, and evaluation of that data. From fine-tuning foundation models to validating agent behaviors in production, our tools are used by leading AI teams to ensure models are grounded in real-world signal, not noise.

Our open-source product, Label Studio, has become the de facto standard for labeling and evaluating data across modalities — from text and images to time series and agents-in-environments. With over 250,000 users and hundreds of millions of labeled samples, it’s the most widely adopted OSS solution for teams working on building AI systems.

Label Studio Enterprise builds on that traction with the security, collaboration, and scalability features needed to support mission-critical AI pipelines — powering everything from model training datasets to eval test sets to continuous feedback loops.We started before foundation models were mainstream, and we’re doubling down now that AI is eating the world. If you're excited to help leading AI teams build smarter, more accurate systems — we’d love to talk.

Level: Individual Contributor Compensation: $60,000 – $125,000 Location: San Francisco, CA

About the Role

HumanSignal specializes in operationally complex, multimodal data collection and annotation — delivering the datasets that frontier AI research requires and remote workforce marketplaces can't. We own projects end-to-end, from scoping and protocol design through final delivery, running on-site and distributed expert workforces across 50+ knowledge domains, 30+ languages, and 75+ countries. Our work spans RLHF, evals, red-teaming, and custom multimodal data creation, all powered by Label Studio Enterprise and built on a foundation of rigorous quality workflows, ethical sourcing, and full data security. This role sits at the operational core of that delivery engine — responsible for ensuring our clients get the highest-quality data on time, every time.

This role is not for everyone. HumanSignal Services operates at the intersection of frontier AI research and large-scale human data delivery — and the work is fast, demanding, and unforgiving of dropped balls. You'll own complex, high-stakes data programs end-to-end, managing expert workforces, navigating shifting customer requirements, and holding quality and delivery timelines simultaneously. There is no playbook handed to you. You will build it, break it, and rebuild it better. If you thrive under pressure, take personal pride in operational excellence, and don't quit when a project gets hard — this is the role for you

__

We're building the future of human data for AI, and the quality of our output depends on how well we enable our contributors. As a Training Specialist, you'll design, develop, and deliver training materials that onboard and upskill our expert network — ensuring every contributor understands task requirements, quality standards, and best practices from day one. You'll work closely with Strategic Project Leads, Data Science, and Operations to close quality gaps quickly and at scale.

What You'll Do

Design and deliver project-specific training collateral within tight timelines (often 24–48 hours) for new AI data labeling programs

Develop training materials — written guides, annotated examples, rubrics, and assessments — that translate complex technical task requirements into clear contributor instructions

Analyze learner performance data, error patterns, and quality issues to identify training gaps and design targeted interventions

Train and coach operations staff (SPLs, Ops Associates) on instructional design principles, feedback techniques, and quality assessment methods

Build reusable content templates, frameworks, and self-serve resources to maintain training consistency across projects and customers

Continuously track training effectiveness via quality metrics, completion rates, and satisfaction scores — and iterate quickly

Collaborate with Product and Engineering to surface tooling improvements that support contributor learning at scale

Required Qualifications

3+ years in instructional design, training, or technical content creation

Proven ability to simplify complex concepts for non-expert audiences

Experience developing training materials under tight timelines

Strong written communication and structured thinking

Data-driven: comfortable analyzing performance metrics to improve programs

Preferred Qualifications

AI/ML background or strong understanding of AI training methodologies (SFT, RLHF, model evaluation)

STEM degree or technical fluency (CS, Data Science, or related field)

Experience with LMS platforms or annotation tooling (Scale, Labelbox, Prodigy, etc.)

Background in customer success enablement, sales enablement, or technical consulting