This scorecard is part of the guide to counterfactual task variants, which explains why the same invoice held once and paid once is a natural pair. Here the narrower job is deciding whether a given company holds such pairs in a usable form, and how confident to be when you introduce it.
How the points add up
Each of the seven criteria scores 2 for a strong answer, 1 for borderline and 0 for weak, for a maximum of 14. Gates are pass or fail and sit outside the points.
| Total | Fit | Meaning for the introduction |
|---|---|---|
| 12 to 14 | Strong | Pairs likely exist in quantity, linked and coded; introduce with the basics confirmed |
| 7 to 11 | Borderline | Pairs exist but some break on linkage, reasons or dating; introduce and name the weak rows |
| 0 to 6 | Weak | The opportunity is mostly theoretical; resolve the open questions first |
Why pairs are worth scoring at all: researchers had to build them by hand. In the ICLR 2020 study on counterfactually-augmented data, people revised documents minimally so a different label applied, and models trained on originals and revisions together generally did better than models trained on comparable amounts of original data. The contrast-sets paper found models scored significantly lower on small, meaningful edits. A company that kept its rejections holds pairs of that kind as a by-product of its controls.
Gates first: size, rights, authority and English share
Check these before scoring anything. A failed gate ends the scorecard for now, however rich the records.
| Gate | Pass | Fail |
|---|---|---|
| Size | 50+ full-time employees at peak, contractors excluded | Under 50, or the headcount is mostly contractors |
| Rights to license | The company owns its AP, claims or request history, and supplier or client terms do not forbid reuse | A parent, client or outsourced provider controls the records |
| Authorized contact | A named CEO, or owner-CEO, who can open a licensing discussion | Nobody with that authority identified |
English share applies only outside the U.S., Canada and the EU. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. For a company in those other regions, treat the 70% English rule as a fourth gate. Documented operating history is required too; it is scored below because its depth matters, but a company with none fails outright.
The seven criteria, strong to weak
| Criterion | Strong (2) | Borderline (1) | Weak (0) | Question that settles it |
|---|---|---|---|---|
| Operating history | Five or more years in one AP or case system | History split by a migration | Under two years, or rebuilt records | Which years does each system cover? |
| Rejected versions kept | First submission stays in history after correction | Kept for some document types | Overwritten by the corrected version | When an invoice is rejected and resubmitted, are both versions kept? |
| Reason coded | Fixed reason codes on every hold | Codes for some holds, free text for others | "See email" or nothing | Is the reason for each rejection a fixed field? |
| Versions linked | Same case number, new revision marker | Linked by a typed reference | Resubmission gets an unrelated new ID | Can the two versions be tied by a number? |
| Rule dated | Thresholds and policies kept with effective dates | Current policy known, past dates reconstructable | Nobody knows when thresholds changed | When did the approval threshold last change? |
| Outcome kept | Hold, release, payment and credit history | Final payment only | No outcome stored with the case | Is each version's outcome kept: held, paid, credited? |
| Control cases | Similar cases below the threshold are in the same system | Some, in a different system | None identifiable | Are small invoices processed in the same tool? |
The questions come in this order on purpose. If rejected versions are overwritten, the next four answers matter much less. For the general screen that sits behind these rows, see the company qualification checklist.
Three Illustrative companies, scored
All three companies are fictional, and the answers came from a conversation with each owner or finance lead.
| Criterion | 180-person industrial parts distributor | 240-person property management firm | 90-person engineering consultancy |
|---|---|---|---|
| Operating history | 2 | 1 | 2 |
| Rejected versions kept | 2 | 2 | 0 |
| Reason coded | 2 | 1 | 0 |
| Versions linked | 2 | 1 | 1 |
| Rule dated | 1 | 1 | 1 |
| Outcome kept | 2 | 2 | 1 |
| Control cases | 1 | 1 | 0 |
| Total | 12 of 14, strong | 9 of 14, borderline | 5 of 14, weak |
The parts distributor keeps every rejected supplier invoice in its AP tool under the same invoice number with a revision marker and a coded hold reason. It drops a point on rule dating because the approval threshold changed in 2022 and only the controller remembers when, and another on controls because small invoices go through a corporate card process.
The property management firm keeps rejected maintenance invoices and their outcomes, but a 2021 move to a new accounting system split the history, and hold reasons are coded only for some vendors. Every row is fixable through review, so this is a borderline introduction with named open items.
The engineering consultancy overwrites rejected expense claims with the corrected version, and reasons live in email. Even with a solid outcome record, the pairs do not survive. It passes the gates, so an introduction is possible, but it should say plainly that resubmissions are not kept.
What the score leaves open
The score summarizes what a company says it holds. It cannot show how many pairs differ in exactly one material detail, because that needs a field-by-field comparison inside the company's systems after a discussion starts. It cannot show SourceX's current data needs for a given process, the de-identification required for supplier names and bank details, or what the company would be offered.
Two companion pages pick up where the scorecard stops. The evidence gaps page for counterfactual variants shows how to word the weak rows honestly, and the review packet contents list shows what the company prepares later. For related record families, see task and outcome pairs and accounts payable exception handling; the introducer's guide to operational data licensing covers the wider process, and the full list sits under data types.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the company passes all three gates and scores in the strong or borderline range, register as a partner, confirm the details on who qualifies and read the program terms before you introduce its finance lead.