Coding tasks and graders · synthetic data
How often does a first-draft grader pass wrong code?
The tasks
- 01Payments API
Make payment creation idempotent
A payment endpoint must charge a card once per Idempotency-Key, no matter how the client retries.
- 02Interest
Accrue CAD interest at posting boundaries
Accrue daily interest on a CAD balance and post it at month ends, using Actual/365 and banker's rounding.
- 03FX transfers
Quote transfers with currency-specific minor units
Quote a currency transfer with the fee taken in the source currency and each currency's own decimal places.
- 04Reconciliation
Reconcile synthetic statements deterministically
Match bank statement lines to ledger entries with exact, split, refund and duplicate rules, deterministically.
- 05Ledger
Enforce an atomic double-entry journal
Run a double-entry journal where every post balances per currency and is all-or-nothing.
- 06Card holds
Track active authorization holds and available balance
Compute available balance as posted funds minus active card holds, with capture, void and expiry.
- 07Statements
Select transactions by the account calendar month
Select a month's transactions using midnight in the account's own time zone, converted to UTC.
- 08Refunds
Limit refunds to the captured amount
Allow partial refunds up to the captured amount, which may be less than what was authorized.
- 09Fees
Apply a non-marginal tiered fee schedule
Price a payment at one tier selected by an inclusive upper bound, then round and clamp the fee.
- 10Batch files
Validate an entire SBF-80 payment batch
Validate an invented 80-character fixed-width payment file and reject the whole batch on any error.
- 1 TicketObservable rules and starter code.
- 2 SolutionsCorrect and plausible wrong code.
- 3 Naive graderFirst-draft tests.
- 4 Hardened graderTests for the missed bugs.
Task 01 · idempotent payments
Example: duplicate payment retries
Results
Independent check
The hardened graders reject 46 of 46 held-out wrong solutions and accept 6 of 6 correct ones.
Before the gaps were closed, the hardened graders rejected 6 of 44 of these.