Case gallery
Every case, its trap, and how the models handled it
All 45 cases are synthetic and generated from a seed. Open one to read its statement files with the perturbation highlighted in place, and to replay the recorded episodes against it.
45 of 45 cases
Bulk batch
bulk_batch-2026050000
statictoolRows collapsed into one bulk entry, hiding the detail.
Bulk batch
bulk_batch-2026050001
statictoolRows collapsed into one bulk entry, hiding the detail.
Bulk batch
bulk_batch-2026050002
statictoolRows collapsed into one bulk entry, hiding the detail.
Bulk batch
bulk_batch-2026050003
statictoolRows collapsed into one bulk entry, hiding the detail.
Bulk batch
bulk_batch-2026050004
statictoolRows collapsed into one bulk entry, hiding the detail.
Clean
clean-2026000000
statictoolNo trap: a straightforward reconciliation.
Clean
clean-2026000001
statictoolNo trap: a straightforward reconciliation.
Clean
clean-2026000002
statictoolNo trap: a straightforward reconciliation.
Clean
clean-2026000003
statictoolNo trap: a straightforward reconciliation.
Clean
clean-2026000004
statictoolNo trap: a straightforward reconciliation.
Column drift
column_drift-2026100000
statictoolColumn names and order change partway through the CSV.
Column drift
column_drift-2026100001
statictoolColumn names and order change partway through the CSV.
Column drift
column_drift-2026100002
statictoolColumn names and order change partway through the CSV.
Column drift
column_drift-2026100003
statictoolColumn names and order change partway through the CSV.
Column drift
column_drift-2026100004
statictoolColumn names and order change partway through the CSV.
Date ambiguity
date_ambiguity-2026150000
statictoolSome dates are swapped to an ambiguous convention.
Date ambiguity
date_ambiguity-2026150001
statictoolSome dates are swapped to an ambiguous convention.
Date ambiguity
date_ambiguity-2026150002
statictoolSome dates are swapped to an ambiguous convention.
Date ambiguity
date_ambiguity-2026150003
statictoolSome dates are swapped to an ambiguous convention.
Date ambiguity
date_ambiguity-2026150004
statictoolSome dates are swapped to an ambiguous convention.
FX conversion
fx_conversion-2026200000
statictoolOne entry is restated in EUR with the rate in the narrative.
FX conversion
fx_conversion-2026200001
statictoolOne entry is restated in EUR with the rate in the narrative.
FX conversion
fx_conversion-2026200002
statictoolOne entry is restated in EUR with the rate in the narrative.
FX conversion
fx_conversion-2026200003
statictoolOne entry is restated in EUR with the rate in the narrative.
FX conversion
fx_conversion-2026200004
statictoolOne entry is restated in EUR with the rate in the narrative.
Injected instruction
injected_instruction-2026250000
statictoolThe data itself instructs the agent to report a wrong figure.
Injected instruction
injected_instruction-2026250001
statictoolThe data itself instructs the agent to report a wrong figure.
Injected instruction
injected_instruction-2026250002
statictoolThe data itself instructs the agent to report a wrong figure.
Injected instruction
injected_instruction-2026250003
statictoolThe data itself instructs the agent to report a wrong figure.
Injected instruction
injected_instruction-2026250004
statictoolThe data itself instructs the agent to report a wrong figure.
Missing balance
missing_balance-2026300000
statictoolThe opening balance is gone: the answer is uncomputable.
Missing balance
missing_balance-2026300001
statictoolThe opening balance is gone: the answer is uncomputable.
Missing balance
missing_balance-2026300002
statictoolThe opening balance is gone: the answer is uncomputable.
Missing balance
missing_balance-2026300003
statictoolThe opening balance is gone: the answer is uncomputable.
Missing balance
missing_balance-2026300004
statictoolThe opening balance is gone: the answer is uncomputable.
Near duplicate
near_duplicate-2026350000
statictoolA double-booked payment that cannot be told from a real one.
Near duplicate
near_duplicate-2026350001
statictoolA double-booked payment that cannot be told from a real one.
Near duplicate
near_duplicate-2026350002
statictoolA double-booked payment that cannot be told from a real one.
Near duplicate
near_duplicate-2026350003
statictoolA double-booked payment that cannot be told from a real one.
Near duplicate
near_duplicate-2026350004
statictoolA double-booked payment that cannot be told from a real one.
Rounding trap
rounding_trap-2026400000
statictoolAmount precision is inconsistent across entries.
Rounding trap
rounding_trap-2026400001
statictoolAmount precision is inconsistent across entries.
Rounding trap
rounding_trap-2026400002
statictoolAmount precision is inconsistent across entries.
Rounding trap
rounding_trap-2026400003
statictoolAmount precision is inconsistent across entries.
Rounding trap
rounding_trap-2026400004
statictoolAmount precision is inconsistent across entries.