This gap analysis belongs to the guide to trajectory-level evaluation in an ordinary workflow, which grades a single store transaction against its reference path. The examples below stay with a related task so the gaps are easy to compare: a return over $250 without proof of purchase needs a supervisor's personal sign-off before money goes back to the customer.
What step-level grading expects to find
Grading a trajectory means comparing what happened with a reference path and the rules in force. Google Cloud's agent evaluation tooling, for example, compares an agent's sequence of tool calls with a reference sequence and offers exact, in-order and any-order matching. Appen's trajectory analysis page (checked 2026-10-18) describes reviewers labeling each step of an agent's run as correct, suboptimal, incorrect or unnecessary. For a business archive to support either approach, each step must leave five kinds of evidence.
| Evidence | Question it lets a grader answer |
|---|---|
| Approver identity | Did an authorized person take the approval step, personally? |
| Step context | Was the required check done, and what did it find? |
| Rule version | Which threshold and approver list applied on that date? |
| Sequence and time | Did approval come before the refund? |
| Final outcome | Did the refund stand, or was it reversed, disputed or repeated? |
NIST's audit-content control for information systems (AU-3) covers much of the same ground from a security angle: the type of event, when and where, its source, its outcome and the identity of who was involved. The trajectory version adds one demand of its own, the rule version, because the reference path changes when the policy does.
Expected versus what retailers typically keep
The pattern below is a description of common shapes, not measured data.
| Evidence | Typically present | Usually missing | How the gap shows up |
|---|---|---|---|
| Refund amount and tender | Almost always: the payment record is complete | Rarely missing | No gap; this is why outcome-only checks look easy |
| Approver identity | An approval event exists | The real approver, when shared override codes, "manager on duty" logins or assistant entries are used | The log names a supervisor who was not on shift |
| Step context | A return line with item and price | The condition code, an optional field left blank | The grader cannot tell whether the item was inspected |
| Rule version | Today's threshold in the policy library | When the threshold changed from $200 to $250 | Refunds between $200 and $250 from before the change are graded against the wrong threshold |
| Final outcome | The refund transaction | A later reversal held in the processor's portal | A "correct" outcome that was not final |
Sequence and time are left out of the table because point-of-sale timestamps are usually present; the trouble comes when the approval is entered into a different system with its own clock, which turns into a version of the context gap.
Four gaps, worked through (Illustrative)
Missing permissions. Most common and most damaging. A pooled sign-off code records an approval without a real approver, so money returned by a clerk acting alone looks identical to money a supervisor signed off. NIST's enhancement AU-3(1) names this problem in security terms: the individual identities of group account users. In grading terms, the failure case is indistinguishable from the success case, which removes exactly the examples a trajectory grader is for.
Missing context. The return line has a condition code field, but it is optional and blank on most rows. The step happened, or did not; nobody can say. The graded path then skips step 2, and only an any-order or in-order match that tolerates missing steps can use the case at all.
Unclear versions. The no-receipt threshold was $200 until last spring and $250 since. Without the change date, every $230 refund from last winter is graded against the wrong rule: one properly approved looks like an unnecessary approval, and one issued without approval looks compliant. Both are graded wrong.
Unproven outcomes. A card network later reverses a refund as a duplicate, but the reversal sits in the payment processor's portal with no link to the store transaction. The path may be perfect and the result still wrong.
Which gaps narrow the archive, and by how much
Not every gap costs the same. A rough way to think about it, before SourceX measures it in review:
- Permissions gap with a known end date: the years after the fix remain fully gradable; earlier years support outcome checks only.
- Permissions gap with no end: the approval step cannot be graded, which removes the main reason to grade trajectories in this task.
- Context gap: cases with the field filled remain usable; the share matters.
- Version gap: cases near a threshold become ambiguous; cases far from it are unaffected.
- Outcome gap: steps can be graded, but the record cannot show whether a correct-looking path produced a result that held.
Stating the gap plainly in an introduction
Ask about shared codes early, plainly and without judgment. Many retailers know about them and fixed them at some date. Then record what the company told you, in its words:
Task: no-receipt refunds over $250, about 9,000 a year across stores, per the loss prevention lead. Present: approval events in the point-of-sale system since 2017; refunds and tender. Gap, per the company: shared manager codes in use until March 2021; condition codes optional. Unknown: whether threshold changes before 2022 were dated; whether processor reversals can be matched. Who decides: CFO, with the loss prevention lead as systems contact.
Illustrative content. Do not estimate gradable share, and do not promise a gap can be repaired. Before writing it, the trajectory-level evaluation scorecard helps decide whether to introduce at all; afterwards, the company prepares the trajectory-level evaluation review packet.
The company needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. On English share, the rule is: Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. For help judging it, see how to assess whether business documentation is primarily in English. For what step records look like, see agent trajectory data; for how graded examples are used, see what model evaluation is; for tracing evidence in diligence, MCP for due diligence Q&A.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the company can date when shared approvals ended, or never used them, register as a partner, check who qualifies, and read the terms before you send the introduction with its gaps named.