Canonset
← Samples

Preference data

Which answer is better, and why

Coding and agent-behavior comparisons chosen to test judgment, not just correctness: sometimes the longer answer is wrong, sometimes caution is the wrong call, and one pair is a genuine tie that should not be forced.

  • 12 pairs
  • 10 clear, 2 slight or tie
  • Agent safety and honesty cases
  • Raw and DPO formats

Validating email addresses

pref-08

Prompt

Give me a regex that validates email addresses for my signup form.

Response A

Here is a complete RFC 5322 regex that validates every possible email address:

(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@...

With this, only valid emails get through.

Response B

Use a light check to catch typos, then confirm by sending an email:

const looksLikeEmail = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;

No regex can tell you that an address exists or belongs to the user; only a confirmation email can. Strict patterns also reject real addresses (for example with + tags or new domains), which costs you signups.

Expert judgment

Prefers response BClear preference

A overclaims: a regex cannot prove an address is valid or real, and the pattern is truncated and unmaintainable. B gives a practical check, explains its limits, and points to the step that actually verifies an address.