# Trajectory-level evaluation fit scorecard | SourceX

Canonical: https://refer.sourcex.si/questions/does-trajectory-level-evaluation-data-qualify
Last updated: 2026-11-04

Pass two gates first: 50 or more full-time employees at peak, and English share where the geography rule needs it. Then score five criteria from 0 to 3: operating history, rights and authority, step attribution (counted double), dated rules, and outcome linkage, for 18 points. Fourteen or more is strong, nine to thirteen borderline, eight or under weak for now. The deciding question is whether each approval is logged under the person who actually gave it.

The scorecard extends the guide to [trajectory-level evaluation in an ordinary workflow](/guides/concept-trajectory-level-evaluation), which follows a $312 no-receipt refund that was correct in amount but approved with a shared override code. That guide explains why the path matters. This page helps an introducer judge whether a particular client's records could support grading the path at all.

## Gates before any points

Two conditions are pass or fail. If either fails, stop; no score on the record criteria changes the answer.

| Gate | Pass | Fail |
|---|---|---|
| Headcount | 50+ full-time employees at peak, contractors excluded | Under 50, or mostly seasonal contractors |
| Geography and English | Company in the U.S., Canada or the EU, or 70%+ of records in English elsewhere | Under 70% English outside those regions |

The rule in full: Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

## The five scored criteria

| Criterion | 3 points | 2 points | 1 point | 0 points |
|---|---|---|---|---|
| Operating history | Five or more years in the same point-of-sale or ERP system | Three to five years, or one migration with history carried over | One to two years | Months, or history discarded at a migration |
| Rights and authority | The company owns its transaction logs; a named CEO or owner can authorize a licensing discussion | Ownership clear; vendor or processor terms unchecked | A franchise or parent may share ownership | Records belong to a franchisee, processor or client |
| Step attribution (x2) | Every step logged with the individual who took it, no shared codes | Shared codes ended on a known date; later years are clean | Some steps logged per person, others under shared or role accounts | Approvals on paper, or under shared logins throughout |
| Dated rules | Thresholds, approver lists and policy revisions carry effective dates | Major changes dated; minor ones not | Changes known from memory only | Only the current policy exists |
| Outcome linkage | Reversals, chargebacks and re-refunds link to the original transaction | Linkable by card, amount and date | Outcomes visible only in a processor portal | No outcome record |

Step attribution is scored 0 to 3 and then doubled, so it can carry 6 of the 18 points. That reflects how trajectory grading works. Google Cloud's agent evaluation tools, for example, compare an agent's sequence of tool calls with a reference sequence, with exact, in-order and any-order matching. A business archive supports the same kind of comparison only when each step is attributable. The research benchmark τ-bench, by contrast, grades by comparing the final database state with a goal state, an outcome check that would pass a correctly sized refund approved by the wrong person.

**Reading the total.** With both gates passed: 14 to 18 is strong, 9 to 13 borderline (introduce with the low rows named), 8 or under weak for now. A zero on step attribution makes the result weak for now, whatever the total, because without attributable steps there is nothing for a trajectory grader to check that an outcome grader cannot.

## Five questions that settle the score

Ask the head of operations, loss prevention or finance. None of them requires a screen or a file.

| # | Question | Criterion |
|---|---|---|
| 1 | Which routine task has a rule about how it is done, not just the result? Roughly how many a year? | Sets the task; context for all rows |
| 2 | Is every step logged with the person who took it, and in which system? | Step attribution |
| 3 | Can anyone approve under someone else's identity, through shared codes or assistant entries? If that changed, when? | Step attribution |
| 4 | Are thresholds and approver lists changed with effective dates recorded? | Dated rules |
| 5 | Who owns these records, and who can authorize a licensing discussion? | Rights and authority |

Add two follow-ups for the remaining rows: "How far back does the system go?" and "When a refund is later reversed or disputed, can you find the original transaction?"

Question 3 is the one that moves the score most, and it deserves a plain, unhurried tone. Security auditing has long treated shared accounts as an attribution problem: NIST's audit-content enhancement AU-3(1) lists the individual identities of group account users among the extra details a system may need to record. Many retailers know about shared override codes and have fixed them at some date, and that date is itself useful to record.

## Three scored fits (Illustrative)

All three companies are fictional. The numbers illustrate the method only.

**Strong: a 450-person outdoor gear retailer with 22 stores.** No-receipt refunds over a threshold need a manager's own badge scan, logged in the point-of-sale system for eight years. Shared codes were removed in 2021, on a date the loss prevention lead knows. Threshold changes are dated in the policy library; chargebacks link to transactions by card, amount and date only. The company owns its transaction logs and the CEO can authorize a licensing discussion. Scores: history 3, rights 3, attribution 2 (x2 = 4, because the years before 2021 carry shared codes), rules 3, linkage 2, for a total of 15. Strong, with the pre-2021 period named.

**Borderline: a 190-person furniture retailer.** Delivery-damage credits need a manager signature on a paper slip that is scanned into the order later. The ERP records the credit under the clerk's login. Rule changes are announced by email. Scores: history 3, rights 3, attribution 1 (x2 = 2), rules 1, linkage 2 for 11. Worth introducing, with the paper approvals named as a gap.

**Weak: an 85-person franchise group of quick-service restaurants.** Each franchisee owns its point-of-sale data under the franchise agreement, voids use a shared manager code, and the menu-pricing rules carry no dates. Scores: history 2, rights 0, attribution 0, rules 0, linkage 1 for 3. Not ready.

## Who owns the records, and what stays open after scoring

Store operations or loss prevention usually knows the approval process; finance owns refund and chargeback records; IT runs the point-of-sale and ERP systems. A licensing discussion is authorized by the CEO, or by the owner in an owner-run company. The company also needs documented operating history and an authorized contact.

The score cannot tell anyone how many cases are gradable, how card and employee data would be handled, what the point-of-sale vendor's terms allow, or whether a buyer wants the task type. Those questions belong to SourceX's review with the company. Partners stay at metadata level and never ask for receipts, logs or screenshots.

For what step records look like in general, see [agent trajectory data](/questions/agent-trajectory-data); for company basics, the [company qualification checklist](/resources/company-qualification-checklist). Low rows can be described with [trajectory-level evaluation evidence gaps](/questions/trajectory-level-evaluation-evidence-gaps), and after an introduction the company prepares the [trajectory-level evaluation review packet](/resources/trajectory-level-evaluation-review-packet). Advisors in diligence work may find [MCP for due diligence Q&A](/resources/mcp/mcp-due-diligence-qanda) useful, and the [introducer guide to operational data licensing](/guides/a-business-introducer-guide-to-operational-data-licensing) covers the full process.

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If both gates pass and the records score nine or more with step attribution above zero, [register as a partner](/register), confirm the basics on [who qualifies](/who-qualifies), and read the [terms](/terms) before you introduce the COO, CFO or owner.

## Sources

- https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/evaluation-agents
- https://arxiv.org/abs/2406.12045
- https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf
- https://refer.sourcex.si/terms
