This is a thesis about where AI development effort may move, turned into a network decision. Whether any current SourceX program wants evaluation sets in a given domain is decided in SourceX's review and is not something a fund can know in advance.
What changes when a model is tested rather than taught
Training changes a model; evaluation measures it. SourceX's own guide to converting an evaluation license into a training deal draws the same line from the contract side: the grant has to be rewritten, because training changes a model permanently and evaluation does not.
Three findings show why the companies worth calling can differ under each use:
- The outcome is the test. Anthropic's engineering guide to agent evals describes a task as one test with its own defined inputs and success criteria, and judges the outcome by the final state: a flight-booking agent passes only if a reservation actually exists in the database, whatever the agent claims. A record is evaluation material when its final state was captured.
- Small sets count. The same guide says teams wait too long because they think they need hundreds of tasks, when 20-50 simple tasks drawn from real failures is a great start. A company with a modest archive of well-documented cases, including its failures, can matter.
- Published tests leak. Zhang and colleagues wrote GSM1k, 1,205 new math problems matched to the public GSM8k benchmark, and found accuracy drops of up to 8% for some models, with several model families showing systematic overfitting. Cases that were never public are harder for a model to have seen.
Translate the evaluation thesis into a company question
The training question a fund usually asks is about volume: how many records of this kind, across how many years, and how varied? The evaluation question is narrower:
Does this company keep cases whose final outcome was recorded and checked by someone accountable, in a system whose contents were never published or shared with vendors?
Record types where the answer is often yes include closed claims with final settlement and adjuster notes, support tickets with a confirmed resolution and a reopen flag, invoice exceptions with the approver's decision, inspection reports with pass, fail and rework, and dispatch exceptions with what actually happened. For the vocabulary, see what a golden dataset is; for explaining the two uses to an owner in two lines, training versus evaluation, briefly.
Ten training-versus-evaluation referral questions
Ask them of the relationship owner first, then of the CEO if the CEO wants the conversation. Every question stays at metadata level.
| # | Question (metadata only) | Points to training value | Points to evaluation value |
|---|---|---|---|
| 1 | How many records of the core type exist, across how many years? | Large counts over many years | Count matters less than question 2 |
| 2 | Is the final outcome stored with each case? | Helpful, not required | Yes, as a field rather than a buried note |
| 3 | Who confirmed the outcome? | Anyone, or no one | A named accountable role, such as an approver or inspector |
| 4 | Were failures, reversals and reopened cases kept? | Useful variety | Essential; evaluation sets are often built from real failures |
| 5 | Has any of it been published, given to a vendor or used in a public benchmark? | Matters less | Never shared is the point |
| 6 | Is expert judgment written down as reasons, not only a code? | Adds richness | Lets a grader check the reasoning |
| 7 | When did the process last change? | Variety across eras helps | Stable periods give comparable cases |
| 8 | Is each case a self-contained record of inputs and result? | Not required | Needed to build a task with defined inputs and success criteria |
| 9 | Do customer contracts limit reuse of these records? | A rights question under either lens | A rights question under either lens |
| 10 | Could a set stay held out, with access logged? | Not relevant | Shows the cases stayed unseen |
Method note. Questions 2, 4 and 8 rest on Anthropic's description of evaluation tasks, outcomes and starting sets; question 5 on the GSM1k contamination finding; questions 9 and 10 on SourceX's rights criteria and the held-out records discussed in its license guide, all read on 2026-10-26. The split between the two columns is our framework, not SourceX data and not any AI developer's stated requirement. It ranks companies for calling order only; it does not measure demand, record quality or value.
One call list, re-ranked (Illustrative)
Illustrative. Six generic companies a fund's network already knows, with invented record counts.
| Company (Illustrative) | Records described by the owner | Training-lens rank | Evaluation-lens rank | Why it moved |
|---|---|---|---|---|
| 1,200-person industrial distributor | 20 million order lines over 15 years, no case outcomes | 1 | 5 | Volume without recorded outcomes |
| 800-person B2B software company | 2.4 million support tickets over 9 years; resolution code stored, customer confirmation not | 2 | 4 | Outcomes recorded but not confirmed |
| 400-person freight brokerage | 1.1 million loads, plus 30,000 exceptions with logged outcomes | 3 | 2 | The exception log is a ready set of outcome-checked cases |
| 260-person specialty insurer | 40,000 closed claims over 12 years with final settlement and adjuster notes | 4 | 1 | Every case has a checked final outcome and written reasons |
| 150-person inspection firm | 18,000 inspection reports with pass, fail and rework | 5 | 3 | Clear outcomes; whether reports belong to clients needs checking |
| 300-person accounting outsourcer | Close workpapers prepared for clients | Off the list | Off the list | Records belong to clients under either lens |
Same network, same relationships, a different order of calls. The accounting outsourcer shows what the lens never changes: client-owned material is usually not a fit.
Relationship and ownership facts that stay fixed under either lens
The lens reorders the list; it does not loosen anything. The CEO is the person who authorizes a licensing discussion, and the relationship owner, not the platform team, asks first. A company needs 50+ full-time employees at peak (contractors excluded), years of records it created itself, the rights to license them and a location the program covers. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. Funding status plays no part.
Credit is the same too: one partner per company, for the introduction that was first, valid and made within the 90-day window, counted only after the company submits a verified application, and any earlier lead, account or email exchange between SourceX and the company makes it a duplicate. Personal consumer data is usually not a fit under either lens. What only SourceX can settle: whether a current program needs evaluation sets in the company's domain, whether customer contracts allow reuse, and what any license would cover. The comparison of evaluation-only and training licenses covers the terms that follow.
A two-week test of the re-ranking
Test the lens before trusting it. Take the companies that moved two or more places (the distributor, the insurer and the inspection firm in the example) and ask their relationship owners three questions only: 2, 3 and 5. Record yes, no or unknown for each.
- If the evaluation-lens leaders answer yes to questions 2 and 3 more often than the training-lens leaders, call them first, through their owners, and only with each CEO's consent.
- If they do not, the owner's first description overstated the outcomes; return to the fund's default order.
- Either way, report the counts and the reasons. No reward estimate belongs in the note.
Whoever submits an introduction must first sign the SourceX Channel Partner Agreement; partners who signed it as the Channel Partner Non-Circumvention and Non-Disclosure Agreement stay bound by that signed text. A sector-specific version of this test is laid out in a referral pilot for a fund exploring vertical AI, and the portfolio view in the page for venture platform teams.
What would undo the evaluation thesis
- Experts can write the tests. OpenAI's GDPval builds its tasks from the representative work of industry professionals averaging about 14 years of experience, across 44 occupations, and grades them by head-to-head comparison with expert work. If developers commission expert-written tasks, they may not need company archives for evaluation at all.
- The frontier may not be contaminated. The same GSM1k study found that frontier models showed minimal signs of overfitting. If the strongest models are not memorizing public tests, the premium on never-published cases may be smaller than the thesis assumes.
- A small set needs a small slice. If 20-50 well-chosen tasks can start an evaluation, a developer may want only a narrow part of a company's archive. That is our inference, and SourceX has not published how evaluation use affects any license.
- Rights bind both uses. Client-owned records and personal consumer data are usually not a fit whether a model is trained or tested.
- Demand is decided in review. SourceX checks each company against its current programs, and those change.
Methods note for this section. Each risk comes from the cited research or SourceX's published criteria as read on 2026-10-26. Nothing here measures how much evaluation demand exists, which sectors it favors or how often a re-ranked call leads to a credited introduction; those figures are not published.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
When a re-ranked company's owner confirms stored, checked outcomes and the CEO wants the conversation, register for a partner link, confirm the fit facts against who qualifies and read the program terms; related guides sit in the guides library.
Re-ordering a call list is a referral task, and weighing SourceX as an investment is another; anyone doing the second can ask for investor materials, and the specific diligence items they need, through the SourceX investor overview.
This article is for information only and is not an offer to sell or a solicitation of an offer to buy securities. Any offering by SourceX is made only to verified accredited investors through its offering documents.