RL environment · business software
Expense approvals
A practice copy of Alder & Vale Freight's expense-review app (a fictional company), a written policy, and 50 tasks an agent is scored on automatically. Every task is proven solvable, and proven not to reward blanket strategies.
Scored tasks
50
18 easy · 17 medium · 15 hard
Verified
50/50
Reference passes, 4 of 4 attacks fail
Policy rules
22
Version 3.2, in 7 sections
World
15 people
10 budgets · 5 currencies
What the agent does
- Works a queue of expense reports as a finance analyst, with the same actions a person has: approve, reject, request information, escalate, and fix individual lines.
- Applies a numbered policy with an order of precedence: escalations, then missing information, then line rules.
- Deals with what real queues contain: foreign currencies, duplicates across reports, purchases split under the receipt limit, a shared budget spent oldest-first, and employee notes that try to give it instructions.
- Is scored on the final state of the app: what was approved, adjusted, rejected, escalated or left alone, and whether each reason cites the rule.
How we know the tasks are sound
- Every task has a reference solution written by a person, and it must pass every check.
- Four strategies that ignore the policy must each fail: do nothing, approve everything, reject everything, ask for receipts everywhere.
- Reports outside a task's scope must end untouched, so an agent cannot score by acting on everything.
- A test fails the build if any duplicate or split purchase exists that the task author did not intend.
Skills covered
How many tasks exercise each skill
adjust · 9lodging limit · 8software · 7escalation · 7partial approval · 5personal items · 5meals · 5currency · 5receipts · 4airfare · 4timeliness · 4missing information · 4client entertainment · 4duplicates · 4instructions in data · 4alcohol · 3budget · 3categories · 3rule order · 3conflict of interest · 3submission order · 3split purchases · 2scope · 2triage · 2approve · 1history · 1dates · 1cross-report · 1reject report · 1