This worksheet is part of the benchmark contamination guide. The workflow example shows the same controls running inside one business process, and the review packet shows what a company assembles later.
Why private is a starting point, not a verdict
Benchmark builders work from a cautious default. The BIG-bench documentation asks task authors to embed a canary string so their data can be filtered out of training corpora, and it states that anything accessible online and not marked with the canary should be treated as potentially part of a model's training data. The burden is on showing a record is absent, not on showing it is present.
The cost of getting this wrong is measurable. When researchers built GSM1k, a fresh set of grade-school math problems matched to the public GSM8k benchmark, several model families scored lower on the new problems, which the authors read as overfitting to the public set. Fresh material revealed what familiar material had hidden.
Detection methods help buyers, but they point one way. Oren and colleagues showed that a statistical test can flag contamination of a known dataset without access to a model's training data, by checking whether the model prefers the dataset's original ordering to shuffled versions. A test like that needs the dataset to exist publicly in some form. For a never-published business record, there is nothing to test against, so the owner's documentation of where the record has been is the main evidence available.
The worksheet: one line per candidate task
Each candidate task gets one line. A task is a single case with a checkable outcome: a service call with a confirmed diagnosis, a claim with a final decision, an order exception with a recorded fix.
| Column | What to enter | Why it is there |
|---|---|---|
| Task ID | The record's own ID in the system of record | Lets every later check be audited |
| Created | The date the case opened | Places the task against a model's data cutoff |
| Systems | Every system that held any part of the case | Shows where copies could exist |
| Outside recipients | Vendors, outsourcers or partners that received it | Each is a route to check |
| Six exposure checks | Clear, flagged or unknown for each check below | The core of the worksheet |
| Outcome recorded | Yes or no, and where | No outcome means no checkable right answer |
| Band | Held-out candidate, training only, or excluded | The decision the line supports |
Every check starts as unknown. A task moves to clear only when someone has looked and recorded what they found.
Six exposure checks, and what each can and cannot show
| Check | How a private record escapes | What a check can show | What it cannot show |
|---|---|---|---|
| Public twin | The company's own help article, manual or case study walks through the same scenario | A phrase search of the company's public pages finds or misses a close match | Whether an old, since-deleted page was captured by a crawler |
| Customer post | The other party describes the case on a forum or review site | A search on distinctive phrases or part numbers turns it up | Posts in closed communities or under different wording |
| Vendor copy | An outsourced provider or software vendor received the records | The vendor list and each contract's data-use terms | What the vendor actually did with the copies |
| Shared pool | Records went into an industry benchmarking consortium | Membership and contribution records; see data shared with an industry benchmark consortium | How the consortium's members used the pooled data |
| Prior owner | Records came with an acquired business that had shared them | Deal files and the seller's disclosures; see who owns startup data after an acquisition | Sharing the seller never disclosed |
| Date known | The case cannot be dated, so its timing against a cutoff is unknown | A creation date in the system of record | Whether the date survived a past migration intact |
The right-hand column is the honest part. Each check narrows the uncertainty without closing it, which is why the worksheet records findings rather than conclusions.
A worked line (Illustrative)
Fictional and invented for this page. A 230-person commercial HVAC service contractor considers its service-call diagnoses as possible evaluation material. One candidate: "Rooftop unit trips on high head pressure on hot afternoons only; find the cause."
| Column | Entry |
|---|---|
| Task ID | Service call from the field service system |
| Created | July 2025 |
| Systems | Field service tool, technician photos, parts order in the ERP |
| Outside recipients | Equipment maker's warranty portal received the failure code |
| Public twin | Clear: the company blog covers seasonal maintenance, not this fault |
| Customer post | Clear: no match on the unit model and symptom wording |
| Vendor copy | Flagged: the maker's warranty portal received the code and a short description |
| Shared pool | Clear: the company joins no benchmarking pool |
| Prior owner | Clear: the branch was not acquired |
| Date known | Clear: creation date kept in the field service tool |
| Outcome recorded | Yes: condenser fan motor replaced, callback-free for 90 days |
| Band | Training only, pending the rights review of the warranty portal's terms |
One flag moved this task out of the held-out band. It may still be valuable for training, and the flag may fall away if the portal's terms bar reuse. The worksheet keeps the record of why.
Qualification: who answers, who decides, what stays open
The partner's role is to find out whether the company could fill in a worksheet like this, not to fill it in. Ask at metadata level: which workflow produces cases with a recorded outcome, whether case material has ever been published or shared, which outside parties received copies, whether records can be dated, and who can authorize a licensing discussion. The operations lead usually knows the first four; the CEO, or the owner-CEO in an owner-run company, decides the last.
The company needs 50+ full-time employees at peak (contractors excluded), a documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
What stays unknown until SourceX reviews it: vendor contract terms, how each buyer weighs a flag, whether a buyer runs its own overlap checks, demand for the record type and any offer. Nobody at the introduction stage should promise that an archive is uncontaminated. For how evaluation tasks are used on the buyer side, see AI agent benchmarks and what GDPval is. SourceX funds any referral reward separately; it never comes out of the company's payout.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If a client's operations lead could answer the worksheet's columns for a workflow with recorded outcomes, register as a partner, check the company against who qualifies and read the program terms before you introduce its decision-maker.