This gap analysis sits under the guide to counterfactual task variants. That guide shows one invoice held and then paid. This page asks what happens when the records behind such a pair are incomplete, and how to say so.
What a pair review expects to find
A pair is two versions of one task that differ in one meaningful detail. The contrast-sets paper in Findings of EMNLP 2020 built such pairs deliberately, with small edits that change the correct answer, and found models scored significantly lower on them. A business pair can serve the same purpose only if a reviewer can prove the two versions are the same task, see the single difference and confirm what happened to each.
| # | Evidence | Half of the pair it protects | What a reviewer proves with it |
|---|---|---|---|
| 1 | First submission kept after rejection | First version | That the rule fired at all |
| 2 | Resubmission with timestamp | Resubmission | That the fix happened, and when |
| 3 | Shared case number or revision link | Both | That A and B are the same task |
| 4 | Coded rejection reason | Both | Which single detail differed |
| 5 | Rule or threshold in force on both dates | Both | That the rule did not change in between |
| 6 | Approver identity and limit on the release | Resubmission | That the release was valid |
| 7 | Outcome of each version: held, paid, credited, disputed | Both | That the pair ended differently for the stated reason |
Expected versus usually held
| Evidence | What companies typically keep | What usually goes missing | Why |
|---|---|---|---|
| First submission | A hold record | The rejected document itself | Many AP tools replace the image with the corrected one |
| Resubmission | Always, as the paid invoice | Rarely missing | It is the record of payment |
| Link | Same invoice number | A revision marker when the resubmission is keyed as new | Manual re-entry under a new internal ID |
| Coded reason | A hold flag | The reason as a fixed field | Free-text notes such as "see email" |
| Dated rule | The current threshold | When it changed | Policies updated without effective dates |
| Approver authority | The approver's name | Their limit on that date | Limit tables overwritten |
| Outcomes | Final payment | Later credits and disputes linked back | Credits raised as separate documents |
Two failure patterns deserve a second look. If the approval threshold moved from $10,000 to $15,000 between the two submissions, the pair confounds two changes; this is the problem the temporal context guide deals with. And if the resubmission corrected the amount as well as supplying the missing document, the pair has two differences and cannot isolate either.
A building services contractor, gap by gap (Illustrative)
A fictional 230-person building services contractor rejects and resubmits a steady stream of subcontractor invoices. Its AP manager was asked, for every 10 resubmitted invoices, roughly how many keep each item. These are estimates from a conversation, not counts from the system.
| Evidence | Expected per 10 | Available per 10 | Missing per 10 | Half damaged |
|---|---|---|---|---|
| First submission kept | 10 | 3 | 7 | First version |
| Resubmission | 10 | 10 | 0 | None |
| Link between versions | 10 | 8 | 2 | Both |
| Coded reason | 10 | 4 | 6 | Both |
| Rule dated | 10 | 5 | 5 | Both |
| Approver limit | 10 | 7 | 3 | Resubmission |
| Outcome of each version | 10 | 9 | 1 | Both |
The pattern is clear. Payment and linkage are strong, so the company can show what was paid and tie most resubmissions to an invoice number. But in roughly seven of ten cases the rejected version is gone, which means most pairs survive only as a hold flag and a paid invoice. Those are still useful task-and-outcome records, as described in task and outcome pairs, but only about three in ten can serve as true counterfactual pairs. The AP manager added that the tool began keeping rejected images after a 2023 configuration change, so recent years are stronger.
Wording a gap for the introduction
Describe the gap, the years it affects and the reason, in a sentence each. Avoid both understatement and alarm.
- Overwritten first version: "Rejected invoice images were replaced by the corrected version until mid-2023; since then both are kept."
- Uncoded reason: "Hold reasons are coded for price and quantity mismatches; missing-approval holds are noted in free text."
- Undated rule: "The approval threshold changed once, in 2022; the controller can confirm the date but it is not recorded in the AP tool."
- Approver limit: "Approvers are recorded on every release; the limit table for years before 2022 was not kept."
A statement like this helps SourceX set the review scope at once. A claim that "we keep everything" is usually discovered to be untrue later, which costs everyone time.
When a gap stops the introduction
Record gaps narrow the review. These conditions stop the introduction until they change, and none of them depends on the records' quality.
| Condition | Why it stops | What to do |
|---|---|---|
| Fewer than 50 full-time employees at peak (contractors excluded) | Fails eligibility | Do not introduce |
| A parent company, client or outsourced AP provider controls the records | Rights are not the company's to license | Find who can authorize, then reassess |
| No documented operating history | Fails eligibility | Do not introduce |
| No authorized contact | Nobody can open a licensing discussion | Identify the CEO or owner-CEO first |
Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
What stays unknown until SourceX reviews it: how many pairs differ in exactly one detail, buyer interest in the process, the de-identification needed for subcontractor details and any offer. To screen the company first, use the counterfactual variants qualification scorecard; to see what the company would prepare later, read the review packet contents list. Process mining explains why event histories with rework steps are informative, the proprietary data moat guide covers why real exception records are hard to copy, and the wider catalog is under data types.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the company passes the gates and you can name its pair gaps in a few plain sentences, register as a partner, check who qualifies and read the terms before you introduce the owner.