Verified coding tasks

Tasks that prove their own tests

Each task is what a coding agent would receive: a problem statement and starting code, plus hidden tests and a reference solution. Our sandbox ran every one: the tests fail on the starting code, pass twice on the reference, and have no network access.

12 tasks
Python 3.12, TypeScript, JavaScript
33 fail→pass tests
Bug fixes, features, two security fixes

Download JSONL

Parse accounting amounts without float errors

code-01 · what the agent sees is the problem statement and the starting code

easyPython 3.12 · pytest

Problem statement

parse_amount in ledger/amounts.py turns amounts from bank exports into numbers. Finance reports two bugs: 1. Amounts in parentheses, the accounting notation for negatives such as "(1,234.50)", crash the import. 2. Totals drift by fractions of a cent because the function returns floats. Fix parse_amount so that it returns a decimal.Decimal, treats parentheses as a negative sign, accepts a leading "$", thousands separators and surrounding spaces, and raises ValueError for anything that is not an amount.

Starting code

ledger/amounts.py

def parse_amount(text: str) -> float:
    return float(text.replace("$", "").replace(",", ""))

Reference solution

ledger/amounts.py

from decimal import Decimal, InvalidOperation


def parse_amount(text: str) -> Decimal:
    cleaned = text.strip().replace("$", "").replace(",", "").replace(" ", "")
    negative = cleaned.startswith("(") and cleaned.endswith(")")
    if negative:
        cleaned = cleaned[1:-1]
    try:
        value = Decimal(cleaned)
    except InvalidOperation:
        raise ValueError(f"not an amount: {text!r}") from None
    return -value if negative else value

Tests (hidden from the agent)

tests/test_amounts.py

from decimal import Decimal

import pytest

from ledger.amounts import parse_amount


def test_thousands_separator():
    assert parse_amount("1,234.50") == Decimal("1234.50")


def test_dollar_sign_and_spaces():
    assert parse_amount(" $7.00 ") == Decimal("7.00")


def test_returns_decimal():
    assert isinstance(parse_amount("1.10"), Decimal)


def test_no_float_drift():
    assert parse_amount("0.10") + parse_amount("0.20") == Decimal("0.30")


def test_parentheses_are_negative():
    assert parse_amount("(1,234.50)") == Decimal("-1234.50")


def test_rejects_text():
    with pytest.raises(ValueError):
        parse_amount("abc")

Notes

Three tests fail on the starting code; the thousands-separator, spaces and error tests guard existing behavior.

Sandbox run

Recorded by pnpm samples:verify. Our tests fail the build if this stops matching the task.

Checks passed6 tests pass on the reference solution; 3 of them fail on the starting code.

Test	Starting code	Reference	Second run
tests/test_amounts.py::test_thousands_separator	pass	pass	pass	pass → pass
tests/test_amounts.py::test_dollar_sign_and_spaces	pass	pass	pass	pass → pass
tests/test_amounts.py::test_returns_decimal AssertionError: assert False + where False = isinstance(1.1, Decimal) + where 1.1 = parse_amount('1.10')	fail	pass	pass	fail → pass
tests/test_amounts.py::test_no_float_drift AssertionError: assert (0.1 + 0.2) == Decimal('0.30') + where 0.1 = parse_amount('0.10') + and 0.2 = parse_amount('0.20') + and Decimal('0.30') = Decimal('0.30')	fail	pass	pass	fail → pass
tests/test_amounts.py::test_parentheses_are_negative ValueError: could not convert string to float: '(1234.50)'	fail	pass	pass	fail → pass
tests/test_amounts.py::test_rejects_text	pass	pass	pass	pass → pass

Python 3.12 · pytest · canonset-sandbox-python:1 · 1.2 s · checked 2026-09-29 23:30 UTC

Output: Starting code (exit 1, 0.3 s)

..FFF.                                                                   [100%]
=================================== FAILURES ===================================
_____________________________ test_returns_decimal _____________________________

    def test_returns_decimal():
>       assert isinstance(parse_amount("1.10"), Decimal)
E       AssertionError: assert False
E        +  where False = isinstance(1.1, Decimal)
E        +    where 1.1 = parse_amount('1.10')

tests/test_amounts.py:17: AssertionError
_____________________________ test_no_float_drift ______________________________

    def test_no_float_drift():
>       assert parse_amount("0.10") + parse_amount("0.20") == Decimal("0.30")
E       AssertionError: assert (0.1 + 0.2) == Decimal('0.30')
E        +  where 0.1 = parse_amount('0.10')
E        +  and   0.2 = parse_amount('0.20')
E        +  and   Decimal('0.30') = Decimal('0.30')

tests/test_amounts.py:21: AssertionError
________________________ test_parentheses_are_negative _________________________

    def test_parentheses_are_negative():
>       assert parse_amount("(1,234.50)") == Decimal("-1234.50")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/test_amounts.py:25: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

text = '(1,234.50)'

    def parse_amount(text: str) -> float:
>       return float(text.replace("$", "").replace(",", ""))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       ValueError: could not convert string to float: '(1234.50)'

ledger/amounts.py:2: ValueError
=========================== short test summary info ============================
FAILED tests/test_amounts.py::test_returns_decimal - AssertionError: assert F...
FAILED tests/test_amounts.py::test_no_float_drift - AssertionError: assert (0...
FAILED tests/test_amounts.py::test_parentheses_are_negative - ValueError: cou...
3 failed, 3 passed in 0.04s

Output: Reference solution (exit 0, 0.3 s)

......                                                                   [100%]
6 passed in 0.02s

Output: Reference, second run (exit 0, 0.3 s)

......                                                                   [100%]
6 passed in 0.02s