Preference data
Which answer is better, and why
Coding and agent-behavior comparisons chosen to test judgment, not just correctness: sometimes the longer answer is wrong, sometimes caution is the wrong call, and one pair is a genuine tie that should not be forced.
- 12 pairs
- 10 clear, 2 slight or tie
- Agent safety and honesty cases
- Raw and DPO formats
What a mutex is, in one sentence
pref-12
Explain what a mutex is in one sentence.
A mutex is a lock that lets only one thread at a time run a piece of code or touch shared data, so concurrent updates don't corrupt it.
A mutex (mutual exclusion lock) ensures that only one thread can hold it at a time, protecting shared data from simultaneous access.
Expert judgment
Both are accurate, one sentence, and cover the same idea. A adds the consequence (corruption) and B spells out the name; neither is better in a way that should teach the model anything. Forcing a winner here would add noise.