Does a client's long-horizon task data qualify? A scorecard with a seven-join trace test

Short answer

Rate the company on four company criteria (size, operating history, rights, location and English share) and three case criteria (years of complete cases, exception reasons, a recorded final outcome). Then run a trace test: for one typical procurement case, ask which reference links each of eight steps to the next. Six or seven confirmed joins out of seven, with no weak criterion, is a strong fit. Four or five is borderline. Three or fewer means the cases break apart.

Visual briefFull-size diagram
A horizontal gauge from 0 to 7 joins with weak, borderline and strong zones; as seven links between eight procurement steps are checked beneath it, the needle slides and settles at 6 of 7, in the strong zone, with the return-to-credit link flagged as missing.
Illustrative trace test for a fictional 320-person food manufacturer. The test uses answers from an operations lead, not records.

This scorecard belongs to the guide on long-horizon task data in an ordinary workflow, which traces one replacement-parts order across 41 days and four systems. That guide shows what a long case is and which fields hold it together. This page turns that into a rating an introducer can complete in one call with an operations or finance lead, using answers rather than files.

Company criteria: size, history, rights, location

These decide whether the company can be introduced at all. A weak mark on any of them stops the rating.

CriterionStrongBorderlineWeak
Size50 or more full-time employees at peak, contractors excludedNear 50, headcount unconfirmedClearly under 50 full-time
Operating historyDocumented history across several years of tradingA recent merger or carve-out complicates the recordNo documented history
Rights to licenseA named executive could authorize a discussion; no known restriction on supplier recordsA named executive, but supplier terms unreviewedRecords controlled by a parent or customer
Location and English shareU.S., Canada or EU company, or another covered country with at least 70% of records in EnglishCovered country, mixed languages, share unknownNot in a covered country, or mostly another language outside the U.S., Canada and the EU

Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

Case criteria: depth, reasons and outcome

These decide whether the records hold whole cases with something to check at the end.

CriterionStrongBorderlineWeak
Record depthFive or more years of complete cases, closed and disputed ones includedTwo to four years, or only open cases survived a migrationUnder two years
Exception reasonsMatch exceptions, returns and rejections carry a written cause in the systemReasons live in emailExceptions fixed with no note
Recorded outcomeSettlement, credit and replacement are all recorded and datedSettlement recorded; replacements raised separatelyNobody can say how a case ended

Exception reasons matter more than they seem. ERP matching catches the symptom automatically: Microsoft's documentation describes three-way matching as comparing invoiced quantity with received quantity and treating a difference as a matching error. What the match cannot record is why the quantities differ. That explanation is what turns a flag into a step a reviewer can follow.

The trace test: count the joins in one case

This is the heart of the scorecard. Ask the lead to picture an ordinary order that went wrong, then walk through eight steps and ask, for each link, "Which number on the later record points back to the earlier one?" Count a join only if the answer names a reference the system stores, not one a person would have to look up.

JoinFrom stepTo stepReference that confirms it
1Work orderRequisitionWork order number on the requisition
2RequisitionApprovalRequisition number in the approval log
3ApprovalPurchase orderRequisition number on the PO
4Purchase orderShipment and receiptPO number on receipts and inspection results
5ReceiptReturnReceipt or PO number on the return authorization
6Purchase orderInvoice and match exceptionPO number on the invoice
7Exception and returnSettlement and creditInvoice or return number on the payment and credit memo

Read the count this way:

Joins confirmedFitWhat it usually means
6 or 7StrongA reviewer could follow a case end to end
4 or 5BorderlineCases exist, but one or two joins need matching by hand
0 to 3WeakThe records are many short fragments, not long cases

The result is the lower of the trace test and the weakest criterion above. A company with seven joins and a weak rights mark is not a strong fit.

Why so strict? Long cases are where AI agents still struggle. METR measures capability as the length of tasks, in human expert time, that agents complete half the time, and reports that this length has been doubling roughly every seven months; the background is in our explainer on METR's task-length measure. A long task can only be checked end to end if every join between its steps holds.

Three manufacturers rated (Illustrative)

All three companies are fictional, rated from a conversation.

Criterion320-person food manufacturer450-person packaging converter180-person machine shop
Company criteriaAll strongAll strongRights borderline: supplier terms unreviewed
Record depthStrong: seven years on one ERPBorderline: ERP cutover three years ago kept open items onlyStrong: six years
Exception reasonsStrongStrongWeak: fixed by phone
Recorded outcomeStrongStrongBorderline
Trace test6 of 7: credits quote the invoice but not the return number7 of 73 of 7: returns by email, quality log keyed by date
ResultStrongBorderline: limited by record depthWeak

The food manufacturer's missing join is worth naming in the introduction rather than hiding: credits can still be matched to returns through the invoice, but not directly. The converter has perfect joins and a short usable history, which only SourceX's review can weigh.

What a rating cannot settle

The scorecard decides whether to make an introduction. It cannot tell anyone how many cases survive a full review, whether supplier contracts limit use of pricing, what de-identification is needed, or whether any buyer wants this task type. Those are measured by SourceX with the company. Partners stay at metadata level: which systems hold each step, roughly how many orders a year, how many years, whether returns and credits carry references, and who owns the records. Operations or finance leads know the systems; the CFO or owner decides on a licensing discussion.

For what the company's team prepares once a review starts, see the long-horizon task data review packet. The record family is covered more broadly in procurement and vendor workflows and the guide to assessing procurement approvals and exception workflows; long-horizon tasks and project records gives a project-based example. Before any of this, the company qualification checklist and the questions to ask before licensing company data still apply.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If a client's operations lead confirms six or more joins on the trace test and no criterion rates weak, register as a partner, check who qualifies, and read the terms before introducing the CFO or owner.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Why a trace test instead of asking about data volume?

Because long-horizon value sits in the joins between steps. A company can have millions of purchase orders and still be unable to say which return and credit belong to which order. Counting joins in one typical case tests that directly.

Does the trace test need a real case file?

No. The operations or finance lead answers from how the process works: which number the receiving team writes on a return, whether credits quote the invoice, and so on. Nobody opens a record.

What if the company uses a different workflow than procurement?

Use the same method on its own long process, such as a warranty claim or an equipment repair. List the steps, then ask which reference links each step to the one before.

Sources

  1. arxiv.org/abs/2503.14499
  2. learn.microsoft.com/en-us/dynamics365/finance/accounts-payable/t…
  3. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment