The packet supports the guide to counterfactual task variants, where an invoice held for a missing approval and paid after resubmission forms the pair. Partners do not assemble it. It is described so you know what a company's records must be able to produce later.
Packet contents at a glance
| Order | Document | What it shows | Prepared by |
|---|---|---|---|
| 1 | Cover sheet | Headcount, AP system and years, migration dates, authorized contact | Introducer with the owner |
| 2 | Task specification | The decision rule the pairs test, written as an instruction | Controller |
| 3 | Pair inventory | Counts of rejected-then-resubmitted cases per year, by reason code | AP manager |
| 4 | Field comparison template | Which fields are compared to prove only one thing changed | AP system administrator |
| 5 | Threshold register | Each approval threshold with effective dates | Controller |
| 6 | Authority and outcome summary | Approval limits by role and year; counts of holds, releases, payments, credits | Finance |
| 7 | Rights notes | Supplier terms on reuse, personal or bank data present, consents needed | Legal or procurement |
Document notes, one at a time
Cover sheet. Peak full-time headcount (contractors excluded), the AP or ERP tool and the years it covers, any migration and its date, and the name of the CEO (in an owner-run company, usually the owner), who can authorize a licensing discussion. It names no suppliers.
Task specification. The rule written as an instruction a model or a new clerk could follow. For the fictional case: "Process this vendor invoice. If it exceeds $10,000 and has no signed purchase approval, place it on hold and route it to the budget owner. Otherwise, release it for payment once the purchase order matches."
Pair inventory. A small table: year, reason code, number of rejected cases, number resubmitted, and whether the first version was kept. It tells the reviewer how many candidate pairs exist and which reasons dominate, without opening one.
Field comparison template. The list of fields that must match for two versions to count as one task with one change: invoice number, vendor, amount, purchase order, cost center, service date, and the attachment field. The template is what lets a reviewer reject a pair where the resubmission also corrected the amount.
Threshold register. One row per threshold value, with the date it took effect and the date it ended. If the threshold moved from $10,000 to $15,000 in a given year, every pair straddling that date is flagged.
Authority and outcome summary. Approval limits by role for each year, plus counts of holds, releases, payments and later credits or disputes. Counts only.
Rights notes. Whether supplier agreements limit reuse of invoice data, whether bank details or personal data appear on the documents, and whether a parent company must consent.
The pair checklist and acceptance check (Illustrative)
Once a review is under way, the reviewer works through one candidate pair at a time. The checklist below is what each pair must eventually be able to produce from the company's systems.
- ☐Both submissions with timestamps, the shared invoice number and the revision marker
- ☐A field-by-field comparison showing exactly one material difference
- ☐The threshold in force on both dates
- ☐The hold and release events, with who acted and their approval limit
- ☐Payment records, plus any credit or dispute that followed
- ☐A control case under the threshold with the same missing item
Acceptance check. The pair passes only if all four hold: the two versions differ in the attachment and nothing else material; the threshold was the same on both dates; variant A was held and variant B released, each by someone with authority; and variant B was paid without a later credit. A pair where the resubmission also corrected the amount fails, because two things changed at once.
Here is how the check reads against three fictional candidates from a 180-person industrial parts distributor.
| Candidate | Difference between versions | Threshold same on both dates | Release authorized | Paid without credit | Result |
|---|---|---|---|---|---|
| Pair 1, $18,400 repair invoice | Attachment only | Yes | Yes | Yes | Pass |
| Pair 2, $12,750 parts invoice | Attachment and amount | Yes | Yes | Yes | Fail: two changes |
| Pair 3, $16,200 service invoice | Attachment only | No, threshold changed between submissions | Yes | Yes | Fail: rule moved |
Pair 3 is a useful reminder of why the threshold register sits in the packet. The records looked perfect until the dates were set against the policy.
Why reviewers value pairs, and where that stops
Research shows why evaluators want pairs like this. The ICLR 2020 paper on counterfactually-augmented data had people revise documents minimally so a different label applied, and found that training on originals and revisions together generally helped. The contrast-sets paper found models scored significantly lower on small edits that change the answer. That evidence concerns evaluation in general. It says nothing about demand for a particular company's records, which only SourceX's review can establish.
Real pairs also carry something generated ones lack: they show how the company's process actually responded, including exceptions nobody wrote down. That is the argument in whether synthetic data will replace real data. Longer multi-step cases, where a hold leads to several follow-up actions, connect to why task length matters for agents.
Before you suggest a packet
Check three things in conversation first. The company needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license the records and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. The AP lead should be able to say that rejected versions are kept, at least for recent years. And the owner should understand that nothing leaves the company before an agreement: no invoices, no exports, no sample pairs.
If those hold, the counterfactual variants qualification scorecard gives a quick read on fit, and the gap analysis for counterfactual variants shows how to describe what is missing. Accounts payable exception handling covers the wider record family, and other categories are under data types.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the controller could produce a pair inventory and a threshold register from the company's own system, register as a partner, confirm who qualifies and read the terms before you introduce the decision-maker.