Verified coding tasks

Tasks that prove their own tests

Each task is what a coding agent would receive: a problem statement and starting code, plus hidden tests and a reference solution. Our sandbox ran every one: the tests fail on the starting code, pass twice on the reference, and have no network access.

12 tasks
Python 3.12, TypeScript, JavaScript
33 fail→pass tests
Bug fixes, features, two security fixes

Download JSONL

Merge calendar intervals correctly

code-05 · what the agent sees is the problem statement and the starting code

easyPython 3.12 · pytest

Problem statement

scheduling/merge.py merges busy intervals before we show free slots. Users see a meeting from 10 to 11 and one from 11 to 12 as two blocks with a zero-length gap, and busy times disappear when the calendar API returns intervals out of order. Fix merge so that it accepts intervals in any order, merges intervals that overlap or touch (one ends exactly when the next starts), and returns them sorted by start. An empty list returns an empty list.

Starting code

scheduling/merge.py

def merge(intervals):
    result = []
    for start, end in intervals:
        if result and start < result[-1][1]:
            result[-1] = (result[-1][0], max(result[-1][1], end))
        else:
            result.append((start, end))
    return result

Reference solution

scheduling/merge.py

def merge(intervals):
    result = []
    for start, end in sorted(intervals):
        if result and start <= result[-1][1]:
            result[-1] = (result[-1][0], max(result[-1][1], end))
        else:
            result.append((start, end))
    return result

Tests (hidden from the agent)

tests/test_merge.py

from scheduling.merge import merge


def test_sorted_overlaps():
    assert merge([(1, 3), (2, 5), (7, 8)]) == [(1, 5), (7, 8)]


def test_contained_interval():
    assert merge([(1, 10), (2, 3)]) == [(1, 10)]


def test_empty():
    assert merge([]) == []


def test_unsorted_input():
    assert merge([(7, 8), (1, 3), (2, 5)]) == [(1, 5), (7, 8)]


def test_touching_intervals_merge():
    assert merge([(10, 11), (11, 12)]) == [(10, 12)]

Notes

Touching intervals are the case users actually hit: back-to-back meetings.

Sandbox run

Recorded by pnpm samples:verify. Our tests fail the build if this stops matching the task.

Checks passed5 tests pass on the reference solution; 2 of them fail on the starting code.

Test	Starting code	Reference	Second run
tests/test_merge.py::test_sorted_overlaps	pass	pass	pass	pass → pass
tests/test_merge.py::test_contained_interval	pass	pass	pass	pass → pass
tests/test_merge.py::test_empty	pass	pass	pass	pass → pass
tests/test_merge.py::test_unsorted_input assert [(7, 8)] == [(1, 5), (7, 8)] At index 0 diff: (7, 8) != (1, 5) Right contains one more item: (7, 8) Use -v to get more diff	fail	pass	pass	fail → pass
tests/test_merge.py::test_touching_intervals_merge assert [(10, 11), (11, 12)] == [(10, 12)] At index 0 diff: (10, 11) != (10, 12) Left contains one more item: (11, 12) Use -v to get more diff	fail	pass	pass	fail → pass

Python 3.12 · pytest · canonset-sandbox-python:1 · 1.2 s · checked 2026-09-29 23:30 UTC

Output: Starting code (exit 1, 0.3 s)

...FF                                                                    [100%]
=================================== FAILURES ===================================
_____________________________ test_unsorted_input ______________________________

    def test_unsorted_input():
>       assert merge([(7, 8), (1, 3), (2, 5)]) == [(1, 5), (7, 8)]
E       assert [(7, 8)] == [(1, 5), (7, 8)]
E         
E         At index 0 diff: (7, 8) != (1, 5)
E         Right contains one more item: (7, 8)
E         Use -v to get more diff

tests/test_merge.py:17: AssertionError
________________________ test_touching_intervals_merge _________________________

    def test_touching_intervals_merge():
>       assert merge([(10, 11), (11, 12)]) == [(10, 12)]
E       assert [(10, 11), (11, 12)] == [(10, 12)]
E         
E         At index 0 diff: (10, 11) != (10, 12)
E         Left contains one more item: (11, 12)
E         Use -v to get more diff

tests/test_merge.py:21: AssertionError
=========================== short test summary info ============================
FAILED tests/test_merge.py::test_unsorted_input - assert [(7, 8)] == [(1, 5),...
FAILED tests/test_merge.py::test_touching_intervals_merge - assert [(10, 11),...
2 failed, 3 passed in 0.03s

Output: Reference solution (exit 0, 0.3 s)

.....                                                                    [100%]
5 passed in 0.02s

Output: Reference, second run (exit 0, 0.3 s)

.....                                                                    [100%]
5 passed in 0.02s