RL environment · business software

Expense approvals

A practice copy of Alder & Vale Freight's expense-review app (a fictional company), a written policy, and 50 tasks an agent is scored on automatically. Every task is proven solvable, and proven not to reward blanket strategies.

Play a task Download tasks (JSONL)

13 tools

JSON Schema, ready for any tool-calling API. Reads return data; writes return ok or an error the agent can recover from.

get_policyRead the travel and expense policy.

{
  "type": "object",
  "properties": {},
  "required": [],
  "additionalProperties": false
}

list_reportsList reports in the Finance Operations queue with status, employee and totals.

{
  "type": "object",
  "properties": {
    "status": {
      "type": "string",
      "description": "Optional filter: submitted, approved, rejected, needs_info or escalated"
    }
  },
  "required": [],
  "additionalProperties": false
}

get_reportRead one report with every line, receipts, notes and decisions.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string",
      "description": "For example ER-2101"
    }
  },
  "required": [
    "report_id"
  ],
  "additionalProperties": false
}

get_budgetRemaining budget of a cost center, in USD.

{
  "type": "object",
  "properties": {
    "cost_center": {
      "type": "string",
      "description": "For example CC-OPS"
    }
  },
  "required": [
    "cost_center"
  ],
  "additionalProperties": false
}

approve_reportApprove the report; every pending line is paid in full.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "note": {
      "type": "string",
      "description": "Optional"
    }
  },
  "required": [
    "report_id"
  ],
  "additionalProperties": false
}

reject_reportReject the whole report.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "reason": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "reason"
  ],
  "additionalProperties": false
}

request_infoSend the report back to the employee for missing information.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "message": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "message"
  ],
  "additionalProperties": false
}

escalate_reportEscalate the report to the Controller or the CFO.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "to": {
      "type": "string",
      "enum": [
        "controller",
        "cfo"
      ]
    },
    "reason": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "to",
    "reason"
  ],
  "additionalProperties": false
}

reject_lineReject one line.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "line_id": {
      "type": "string"
    },
    "reason": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "line_id",
    "reason"
  ],
  "additionalProperties": false
}

adjust_linePay a line at a lower amount (USD), for example at a policy limit.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "line_id": {
      "type": "string"
    },
    "approved_usd": {
      "type": "number"
    },
    "reason": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "line_id",
    "approved_usd",
    "reason"
  ],
  "additionalProperties": false
}

flag_duplicateReject a line as a duplicate of another line: "L1" on the same report or "ER-1234/L1" on another.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "line_id": {
      "type": "string"
    },
    "duplicate_of": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "line_id",
    "duplicate_of"
  ],
  "additionalProperties": false
}

set_categoryCorrect a line's category.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "line_id": {
      "type": "string"
    },
    "category": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "line_id",
    "category"
  ],
  "additionalProperties": false
}

add_commentAdd a comment to a report.

{
  "type": "object",
  "properties": {
    "report_id": {
      "type": "string"
    },
    "text": {
      "type": "string"
    }
  },
  "required": [
    "report_id",
    "text"
  ],
  "additionalProperties": false
}

An episode

1. Start from a task's initial_state and give the agent its instruction.
2. The agent calls tools until it stops.
3. Grade the final state: reward = share of checks passed; success = all passed.

Reference for exp-26

Two fixes, then approve

adjust_line({"report_id":"ER-2126","line_id":"L1","approved_usd":500,"reason":"Lodging limited to $250/night × 2 nights (4d)"})
reject_line({"report_id":"ER-2126","line_id":"L2","reason":"Alcohol outside client entertainment (4c)"})
approve_report({"report_id":"ER-2126"})

Checks: ER-2126 L1 pays $500.00; ER-2126 L2 is rejected; ER-2126 L2 reason mentions "alcohol" or "4c"; ER-2126 is approved; ER-2126 pays $600.10 in total.

Downloads

tasks.jsonl — instruction, initial state, checks, reference, verification
tools.json — tool definitions
policy.md — the policy text