Expert data, made canonical.

Verified training data for coding and business-software agents.

Pilot work is written by vetted US experts, checked by machine and approved by a person: every coding task is sandbox-verified, and every grader is tested against reward hacking.

Open samples, verified by machine

55
RL environment tasks, each verified
17
sandbox-verified coding tasks
4
reward hacks every environment grader must block
92
sample records you can download today

Our team wrote them with AI help to show the format.

Inspect every record, no sales call →

See how it works

Two minutes of real product screens, with demo data: how experts are vetted, how a project runs, and how every answer is checked.

Chapters

Read the transcript

For AI teams 0:02

Canonset makes training data for AI labs, written by vetted experts. Labs can see the format and the checks before talking to anyone. Every coding task comes with hidden tests, proven in a sandbox. And a business app as an RL environment, each task scored by a grader.

Joining as an expert 0:22

Experts apply for a role, with the pay shown up front. They sign in with an emailed code, then take a short test. Next, an ID check, with their written consent, or a video call instead. They photograph both sides of a US driver's license or state ID. Then the camera takes three pictures as they turn their head. Our own software matches the face to the ID; if in doubt, a person looks. Verified on the spot; the photos are deleted once it's decided.

The interview, three ways 1:00

Then an interview, taken one of three ways. Live, with an AI interviewer.

AI interviewer: To start, tell me briefly about your background and the kind of work you've been doing most in the last few years.

Applicant · synthetic voice: Hi! I'm a backend engineer with about eight years of experience, mostly Python and Go, building payment and data services.

A written interview, where Claude asks and follows up. Or a call with a person on our team. Claude assesses each transcript against a rubric; a person decides.

A paid trial task 1:32

A strong test earns a paid trial task, once a client project needs their field. Here the checks fail: no test proves the fix. One more test, and they pass.

Every answer, checked 1:46

Every answer is reviewed, with an AI second opinion where the client allows it. Coding tasks show how often a frontier model solves them, in one setting. Then our team approves each expert, field by field.

A project, from request to data 2:01

A client starts a project and uploads the data. Personal data is caught before it becomes a task. Experts only ever see a codename. Here, an expert picks the better answer, and says why. The client tracks quality week by week, and gets the data by API.

Running it 2:19

Behind it all: one queue for our team, weekly payouts and an audit log. Start a pilot at canonset.com/pilot.

What we deliver

The public samples show the format. Written by our team with AI help, and verified by machine (the sandbox or the grader). Pilot work is different: vetted experts write it under our no-AI rule.

How we check quality

No answer reaches you until it passes automatic and human checks.

1

Sandbox-verified tests

Each coding task runs in an isolated container: at least one test must fail on the starting code, and all must pass twice on the reference solution.

2

Graders that block reward hacking

Before an environment task ships, four known exploits, such as approving everything or doing nothing, must fail its grader.

3

Known-answer tasks

Gold tasks are mixed in unannounced; misses lower an expert's quality score.

4

Expert agreement

Up to five experts can answer each task; for preference and rubric work, their agreement is in your quality report.

5

A person approves every answer

Senior reviewers approve, return or reject; nobody reviews their own work.

6

No AI-written answers

Timing, paste, overlap and writing-pattern flags put suspect answers first in review; a confirmed case ends the expert's access.

Security and independence

  • Your data stays inside the platform: no shared documents, no public links.
  • Experts see only the task in front of them, never your company name.
  • Submitted code runs in our own locked-down containers, not a third-party sandbox.
  • Canonset is independent of every AI lab.
Read our security overview →

For experts

Paid, flexible project work that uses what you know. Apply once, see the pay before you start, get paid every Monday. No fees, ever.