Remittance advice · neatHack 2026 · an agent that works
Devpost's claim form offers a winner in Jakarta Wise, whose own help page says Indonesian addresses “can no longer hold money”. Cair refuses routes like that, fills W-8BEN line 10 only when a treaty article applies, and footnotes what lands. A tool-less model call picked a dead route in 18 of 48 frozen cases. Cair: 0 of 48.
1Across Cair's 48 frozen plans, 359 of 359 figures trace to a tool output
(run v2-r10, qwen3.8-max, temperature 0). The model writes words; code computes every number.
Evidence
Key to the slip
Payout plan · specimen
Devpost → Indonesia
One tool-less model call said
Pilihan Payout Terbaik: Wise (struck through: no source)
No source “Best payout option: Wise”, translated
biaya konversi yang sangat kecil (biasanya di bawah 1%) (struck through: no source)
No source “a very small conversion fee (usually under 1%)”, translated
What Cair returns
…personal Wise account holders with an address in Indonesia can no longer hold money or currencies in their account.
In your Payoneer — at most
$7,000.00 minus 30% US withholding ($2,100.00IRS-P515-30), before a receiving fee Payoneer does not publish.
provenance_guardrail · PASSED10 spans · 7,063 ms
routes recommended that their own provider rules out
one tool-less call: 18 / 48, in every run
figures on the plans that trace to a tool output
the model writes the words, never the numbers
eval-v2-r10plans completed with the FX tool forced to fail
retry once, then the last ECB rate, stamped stale
eval-fail-fx_quote-r9fields with no primary source, kept UNCLEAR
one tool-less call: 11–23 / 143
eval-v2-r10Specimen · the query that matters
I'm in Jakarta. I won $7,000 in a Devpost hackathon. The claim form lets me pick Payoneer or Wise. Which one, and what do I write on the W-8BEN?
Asked on the form as Devpost · 7000 · Indonesia · Payoneer and Wise. Below: the plan it returned, and that plan's own trace.
The same plan, read back from neatlogs over MCP
get_trace_context 586368705f8d5aa5d97368e57b4d91b6
Cairevidence/traces/live-plan-ee3w412a.jsonStatement of account · one call against the agent
The 48 scenarios — 3 countries × 4 payers × 3 amounts, plus 12 leading prompts — were labelled from the sourced
rules sheet before any agent code existed (tag frozen-set-v1). v1 is one tool-less call. v1s is one
call with the whole rules sheet pasted in, so every sourced fact is in its prompt.
fewer is better
an article on line 10, or a rate other than 30% · fewer is better
v1 across three runs · fewer is better
fields across the 48 answers · more is better
figures the tools never produced reach the page when the guardrail is removed — the user's “zero percent” and “Article 21” repeated back, in 8 of 48 plans. With it, none do: 8 explanations were blocked once and rewritten, none fell back to the template. The final run is the tenth on this set; each earlier run's failures drove a fix. eval-ablation-r10 · rca-01.md
Returned item · a run where an action failed
scripts/eval.py --fail fx_quote makes the ECB rate tool raise on every call. The agent retries once,
falls back to the last rate it fetched, stamps it STALE with its date, and finishes the plan. It never asks the model for a rate.
Trace d1fa1ed78810601ea4f2d6de5dfacec5 · run fail-fx_quote-r1 · ID-devpost-7000
trace status: error · 2 errored spans · plan completedFX: 1 USD = 17881 IDR (ECB reference rate, as of 2026-10-09) — STALE: live rate unavailable, last fetched ECB rate
plans completed with the FX tool failing on every call (U 0 · P 0) · latest run, fail-fx_quote-r9
plans took the stale-rate fallback — every USD payer; USDC plans never call it
figures still traced to a tool output
How a plan is made
What the payer offers and whether a W-8BEN applies → can each option receive in your country → the treaty position → the ECB rate (or, for USDC, the networks the exchange accepts) → Decimal arithmetic. Each tool reads a sourced row and is retried once before its documented fallback.
@neatlogs.span · TOOL · RETRIEVER · GUARDRAILThe route, line 10 and every amount come from sourced rows; the model writes the explanation, nothing else. A refused route can never be the recommendation.
qwen3.8-max · temperature 0No figure, rate or article that isn't in a tool output; no endorsing a refused route. Blocked → one retry → a template built from the tools. On sentences it was not tuned on — measured before its misses were fixed — it blocked 62 of 66 violations and 3 of 66 honest ones.
GUARDRAIL · held-out r4The rules sheet is public: 45 rows, 42 citing a primary page and the date it was read. The 3 that can't — PayPal in Indonesia, India and the Philippines — say UNCLEAR. A fact the provider never states as a number stays UNCLEAR too, so “what lands” is an upper bound, stamped.
data/rules_sheet.csvID · payoneerdeliverable = YES
“Setelah menerima pembayaran, Anda dapat menarik dana tersebut ke rekening bank lokal milik Anda.”
PAYO-ID · payoneer.com/id · read 2026-10-10
ID · wisedeliverable = NO
“…can no longer hold money or currencies in their account.”
WISE-ID · wise.com/help · read 2026-10-10
* · payoneerreceiving_fee_usd = UNCLEAR
“From marketplaces and integrated platforms — Varies by marketplace”
PAYO-PRICING · payoneer.com/pricing · read 2026-10-10
Tell Cair what arrived: the reconciler reads its own quote back from the neatlogs trace over MCP and writes an EVALUATOR span into the same session. First receipt — the builder's own $7,000 Devpost prize, a replay of a known outcome, not a prediction:
MCP get_trace_context · EVALUATORQuoted
$4,900.00
at most, before the fee
Landed
$4,897.00
in Payoneer
Verdict
Landed
within sourced bound
Landed 2026-09-23 · $3.00 not explained by the sourced rules · partial replay, because the fee is UNCLEAR · EVALUATOR trace f5a47b9c… · builder-reported, n = 1
Three instruments, used for real
eval · v1 · v2 · ablation.search_traces "provenance_guardrail BLOCKED" over MCP returned the blocked plans;
get_trace_context put the model's text beside the verdict. Two real bugs came out of it — an amount in the 2000s
parsed as a year, refusals that repeated the user's “0%” — both fixed and re-run.
Run on this repository while building. After the year-parsing fix, entire graph search with a plain-language
description of the bug ranked the corrected rule first and its covering test second. impact on
rail_eligibility reported 0 callers — the planner passes tools as function values, which the graph does not follow;
recorded as found.
Checkpoints were deliberately left off: they install coding-agent hooks and keep the agent conversation, and this repository is public.
The business plan starts from our own numbers: $0.0033 of LLM per plan (bench tokens at list price), p50 5.50 s · p95 10.31 s (bench), hosting $15.26 / month (t3.small list price). At an assumed $4 a month, 4 Pro subscribers break even.
The cfo.ai model built from these inputs is not entered yet — nothing here was produced by cfo.ai.
Terms and conditions · printed on the back
A tool whose whole job is refusing unsourced claims would contradict itself by hiding its own limits. These are quoted from the README.
Payoneer's receiving fee and conversion margin are not published as single numbers, so “what lands” is an upper bound.
Forward receipts from real users (a plan made before the money moved): n = 0.
The eval measures fidelity to the sourced sheet, not tax correctness; three cells were re-checked by hand against the live pages.
Queries · answered at the counter
No. Code decides the route, W-8BEN line 10 and every amount from sourced rows. Qwen only words the card, and a provenance guardrail checks that wording against the tool outputs before it is shown.
Nothing on the product path: the live app calls the real model, the real ECB reference rate (via Frankfurter) and the real neatlogs project, and every measured result on this page comes from such runs. The 94 tests and the CI pipeline run without network or keys — that is a statement about the tests, not the product.
Each tool is retried once, then takes a documented fallback: the last ECB rate fetched, stamped STALE, or UNCLEAR. With the FX tool forced to fail on every call, all 48 plans completed and 350 of 350 figures were still sourced.
Because some numbers are not published. Payoneer does not state its receiving fee for platform payouts as a single number, so Cair shows the upper bound and stamps it UNCLEAR instead of guessing. A plan for a country outside the rules sheet is UNCLEAR on every line, and makes no model call at all.
Indonesia, India and the Philippines are fully sourced. Vietnam and Nigeria have treaty rows only (plus a Wise UNCLEAR row for Nigeria). Anything else is UNCLEAR by design.
No. Cair explains payout forms; it never files them. Every rule shows its source and the date it was read.
Counterfoil · keep this part
No account, no key. The route and W-8BEN line 10 each have a Confirm button, and each confirm is written into the same neatlogs session as the plan.
