This packet is part of the guide to trajectory-level evaluation in an ordinary workflow, which follows one store refund that was the right amount but approved with a shared code. The packet is how a company shows that thousands of such refunds could be graded step by step. It is assembled by the company after an introduction, under SourceX's agreement with the company. Partners never prepare or handle it; they need to know it exists so their first questions point the right way.
How the packet is organized
A reviewer reads in the order a grader works. First, what the task is and what a correct run looks like. Second, whether the steps can be attributed and timed. Third, whether outcomes can be confirmed and the records can be licensed. Each group depends on the one before: there is no point mapping log fields for a task nobody has defined, and no point checking rights for records that cannot be attributed.
The structure mirrors how agent evaluations are built. Google Cloud's agent evaluation tools compare a run with a reference trajectory using exact, in-order or any-order matching; that needs documents 1 to 3. Research benchmarks such as TheAgentCompany split each task into checkpoints and give partial credit for the checkpoints passed; that needs each checkpoint visible in documents 4 to 6.
Packet contents, in order
| Order | Group | Document | What it shows | Who prepares it |
|---|---|---|---|---|
| 1 | Task | Task statement and rough volume | The routine task in one paragraph and about how often it occurs a year | Head of store operations |
| 2 | Task | Policy history with effective dates | Each version of the rule: thresholds, who may approve, required order of steps | Store operations with compliance |
| 3 | Task | Reference trajectory | The correct sequence of steps, numbered, with what each step must record | Store operations |
| 4 | Steps | Role and permission matrix | Which roles may approve which refunds, and since when | Loss prevention with HR |
| 5 | Steps | Log field dictionary | For each step, the fields the system records: user, role, terminal, timestamp, approval method, override reason | IT, point-of-sale administrator |
| 6 | Steps | Credential practice history | Whether shared codes, shared logins or assistant entries were used, and the dates they ended | Loss prevention |
| 7 | Proof | Outcome linkage note | How a reversal, chargeback or repeat refund links to the original transaction, and where it does not | Finance |
| 8 | Proof | Rights and contract summary | Who owns the logs; what point-of-sale vendor and processor terms say; who can authorize licensing | Legal, signed off by the CEO |
| 9 | Proof | Case scorecard and acceptance check | Four described case types graded both ways, and the written check a correct run must pass | Store operations, drafted with SourceX |
Documents 2 and 6 are the ones companies most often underestimate. Without effective dates, a refund graded against today's threshold may be graded wrong; without the credential history, nobody can say which years have trustworthy approver identities.
The reference trajectory (Illustrative)
For the running example, a no-receipt return refunded to the original card under the returns policy, document 3 would read:
- Look up the purchase by card; record the cashier ID and the amount found.
- Inspect the item; record a condition code.
- If the amount exceeds $250, a manager approves with their own credential; record approver ID, role and time.
- Refund the purchase amount less anything already refunded, to the original card.
- Restock or dispose of the item; record the inventory adjustment.
Each numbered step names what it must record, which is what makes document 5 checkable: every field in the dictionary should trace back to a step here.
The case scorecard: outcome check versus trajectory check (Illustrative)
Document 9 shows how the two kinds of grading differ on real case types. Four fictional refunds:
| Case | What happened | Outcome check | Trajectory check |
|---|---|---|---|
| A | $312 refund, manager approves with own code, then refund | Pass | Pass |
| B | $312 refund, shared override code used | Pass | Fail: approval not by an authorized person |
| C | $312 refund issued first, manager approval entered ten minutes later | Pass | Fail on order: approval must precede refund |
| D | Manager approves, but refund issued as $336 including tax already refunded on a prior partial return | Fail: amount wrong | Fail at step 4 |
Cases B and C are the valuable ones, because an outcome-only grader would call them successes. Appen, which sells agent trajectory analysis, describes (checked 2026-10-18) reviewers labeling each step as correct, suboptimal, incorrect or unnecessary to find where a run breaks down; cases like B and C are what such labels are for.
Acceptance check, written before outcomes are examined. A case passes only if all five hold: the refund amount equals the purchase amount less anything already refunded; an approval event exists for any no-receipt refund over $250; the approver's role permits approval under the policy version in force that day; the approval timestamp precedes the refund timestamp; and the approver is not the cashier.
What the partner checks, and what stays with the company
Before introducing, a partner can confirm that documents 1, 4 and 8 could be written: is there a task with a rule about how it is done, do approvals carry individual identities, and who can authorize licensing? The trajectory-level evaluation scorecard turns those answers into a fit score, and the evidence gap analysis shows how to describe what is missing.
The partner never asks for receipts, register logs, approval reports or screenshots. Partners sign the Channel Partner Agreement before submitting introductions. The company needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
For how graded examples are used, see what model evaluation is and rubric-based evaluation. For what step records look like, see agent trajectory data and tool-use training data.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the operations lead could write the task statement and the loss prevention lead can date the credential history, register as a partner, check who qualifies, and read the terms before you introduce the CEO, who would sign document 8.