Verified coding tasks

Tasks that prove their own tests

Each task is what a coding agent would receive: a problem statement and starting code, plus hidden tests and a reference solution. Our sandbox ran every one: the tests fail on the starting code, pass twice on the reference, and have no network access.

12 tasks
Python 3.12, TypeScript, JavaScript
33 fail→pass tests
Bug fixes, features, two security fixes

Download JSONL

Keep once-listeners from skipping others

code-11 · what the agent sees is the problem statement and the starting code

mediumJavaScript · Node.js 24 · node:test

Problem statement

emitter.js is a small event emitter used by our job runner. Two bugs: 1. When a once() listener runs, the listener registered right after it is skipped for that event. 2. off(event, fn) cannot remove a listener that was added with once(event, fn). Fix both without changing the API: on() returns an unsubscribe function, once() registers a listener that runs at most once, off() removes a listener, emit() calls the listeners registered at the moment of the call.

Starting code

emitter.js

export class Emitter {
  #listeners = new Map();

  on(event, fn) {
    const list = this.#listeners.get(event) ?? [];
    list.push(fn);
    this.#listeners.set(event, list);
    return () => this.off(event, fn);
  }

  once(event, fn) {
    const wrapper = (...args) => {
      this.off(event, wrapper);
      fn(...args);
    };
    return this.on(event, wrapper);
  }

  off(event, fn) {
    const list = this.#listeners.get(event);
    if (!list) return;
    const i = list.indexOf(fn);
    if (i >= 0) list.splice(i, 1);
  }

  emit(event, ...args) {
    for (const fn of this.#listeners.get(event) ?? []) fn(...args);
  }
}

Reference solution

emitter.js

export class Emitter {
  #listeners = new Map();

  on(event, fn) {
    const list = this.#listeners.get(event) ?? [];
    list.push(fn);
    this.#listeners.set(event, list);
    return () => this.off(event, fn);
  }

  once(event, fn) {
    const wrapper = (...args) => {
      this.off(event, wrapper);
      fn(...args);
    };
    wrapper.original = fn;
    return this.on(event, wrapper);
  }

  off(event, fn) {
    const list = this.#listeners.get(event);
    if (!list) return;
    const i = list.findIndex((l) => l === fn || l.original === fn);
    if (i >= 0) list.splice(i, 1);
  }

  emit(event, ...args) {
    for (const fn of [...(this.#listeners.get(event) ?? [])]) fn(...args);
  }
}

Tests (hidden from the agent)

emitter.test.js

import assert from "node:assert/strict";
import { test } from "node:test";
import { Emitter } from "./emitter.js";

test("on and emit", () => {
  const e = new Emitter();
  const seen = [];
  e.on("job", (id) => seen.push(id));
  e.emit("job", 7);
  assert.deepEqual(seen, [7]);
});

test("unsubscribe returned by on", () => {
  const e = new Emitter();
  const seen = [];
  const stop = e.on("job", (id) => seen.push(id));
  stop();
  e.emit("job", 1);
  assert.deepEqual(seen, []);
});

test("once runs a single time", () => {
  const e = new Emitter();
  let count = 0;
  e.once("job", () => count++);
  e.emit("job");
  e.emit("job");
  assert.equal(count, 1);
});

test("once does not skip the next listener", () => {
  const e = new Emitter();
  const seen = [];
  e.once("job", () => seen.push("first"));
  e.on("job", () => seen.push("second"));
  e.emit("job");
  assert.deepEqual(seen, ["first", "second"]);
});

test("off removes a once listener", () => {
  const e = new Emitter();
  let count = 0;
  const listener = () => count++;
  e.once("job", listener);
  e.off("job", listener);
  e.emit("job");
  assert.equal(count, 0);
});

Notes

The skipped-listener bug comes from removing an element from an array while iterating over it.

Sandbox run

Recorded by pnpm samples:verify. Our tests fail the build if this stops matching the task.

Checks passed5 tests pass on the reference solution; 2 of them fail on the starting code.

Test	Starting code	Reference	Second run
emitter.test.js::on and emit	pass	pass	pass	pass → pass
emitter.test.js::unsubscribe returned by on	pass	pass	pass	pass → pass
emitter.test.js::once runs a single time	pass	pass	pass	pass → pass
emitter.test.js::once does not skip the next listener Expected values to be strictly deep-equal: + actual - expected [ 'first', - 'second' ]	fail	pass	pass	fail → pass
emitter.test.js::off removes a once listener Expected values to be strictly equal: 1 !== 0	fail	pass	pass	fail → pass

JavaScript · Node.js 24 · node:test · canonset-sandbox-node:1 · 0.8 s · checked 2026-09-29 23:30 UTC

Output: Starting code (exit 1, 0.1 s)

✔ on and emit (1.542507ms)
✔ unsubscribe returned by on (0.21217ms)
✔ once runs a single time (0.206964ms)
✖ once does not skip the next listener (2.173943ms)
✖ off removes a once listener (0.385842ms)
ℹ tests 5
ℹ suites 0
ℹ pass 3
ℹ fail 2
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 89.200763

✖ failing tests:

test at emitter.test.js:31:1
✖ once does not skip the next listener (2.173943ms)
  AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
  + actual - expected
  
    [
      'first',
  -   'second'
    ]
  
      at TestContext.<anonymous> (file:///work/0-starter/emitter.test.js:37:10)
      at Test.runInAsyncScope (node:async_hooks:227:14)
      at Test.run (node:internal/test_runner/test:1402:25)
      at Test.processPendingSubtests (node:internal/test_runner/test:974:18)
      at Test.postRun (node:internal/test_runner/test:1542:19)
      at Test.run (node:internal/test_runner/test:1467:12)
      at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
    generatedMessage: true,
    code: 'ERR_ASSERTION',
    actual: [ 'first' ],
    expected: [ 'first', 'second' ],
    operator: 'deepStrictEqual',
    diff: 'simple'
  }

test at emitter.test.js:40:1
✖ off removes a once listener (0.385842ms)
  AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
  
  1 !== 0
  
      at TestContext.<anonymous> (file:///work/0-starter/emitter.test.js:47:10)
      at Test.runInAsyncScope (node:async_hooks:227:14)
      at Test.run (node:internal/test_runner/test:1402:25)
      at Test.processPendingSubtests (node:internal/test_runner/test:974:18)
      at Test.postRun (node:internal/test_runner/test:1542:19)
      at Test.run (node:internal/test_runner/test:1467:12)
      at async Test.processPendingSubtests (node:internal/test_runner/test:974:7) {
    generatedMessage: true,
    code: 'ERR_ASSERTION',
    actual: 1,
    expected: 0,
    operator: 'strictEqual',
    diff: 'simple'
  }

Output: Reference solution (exit 0, 0.2 s)

✔ on and emit (1.486438ms)
✔ unsubscribe returned by on (0.214723ms)
✔ once runs a single time (0.211372ms)
✔ once does not skip the next listener (0.174671ms)
✔ off removes a once listener (12.366935ms)
ℹ tests 5
ℹ suites 0
ℹ pass 5
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 87.351444

Output: Reference, second run (exit 0, 0.1 s)

✔ on and emit (1.483563ms)
✔ unsubscribe returned by on (0.248393ms)
✔ once runs a single time (0.215294ms)
✔ once does not skip the next listener (0.186131ms)
✔ off removes a once listener (0.307514ms)
ℹ tests 5
ℹ suites 0
ℹ pass 5
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 99.554707