Starting-state fidelity review packet: seven metadata documents a distributor prepares, in order

Short answer

A starting-state fidelity review packet holds seven metadata-only documents, in this order: a task specification, an event reconstruction note, a request record description, a list of allowed actions, an acceptance check written before outcomes are examined, a volume and span summary, and a rights and sensitivity note. Together they tell SourceX whether each order-change task can be rebuilt at the minute it began and graded fairly. The operations lead usually coordinates. The packet describes records and fields; it holds none of them until an agreement is in place.

Visual briefFull-size diagram
A binder on the right fills with seven tabbed sheets that slide in from the left in order: task specification, event reconstruction, request record, allowed actions, acceptance check, volume and span, and rights note, each labelled with the role that prepares it.
Illustrative packet order for a distributor. The packet describes records and fields; it holds no orders until an agreement is in place.

This list belongs to starting-state fidelity, which uses an order partly shipped before a change request arrived. The introducer's part comes earlier: the starting-state fidelity scorecard decides whether to introduce, and the starting-state fidelity evidence gaps analysis explains what the packet should admit openly.

Packet order and owners

The order follows the logic of a simulation. AI developers begin each episode from a defined state, as Gymnasium's reset() does, and grade the end against a goal, as τ-bench does. The packet describes the same two ends using the company's records.

OrderDocumentWhat it showsPrepared by
1Task specificationThe job each case representsCOO or VP operations
2Event reconstruction noteWhich events rebuild the order at any momentERP administrator with warehouse manager
3Request record descriptionWhere change requests arrive and whether their time is keptCustomer service lead
4Allowed actionsWhat a responder may do, and who may approve whatCOO with sales operations
5Acceptance checkThe test a correct response passes, fixed in advanceCOO
6Volume and spanCounts, years and migration datesOperations analyst
7Rights and sensitivity noteIdentifiers present, contract limits, the authorizerCFO or counsel

Documents 1 to 3: defining the starting point

1. Task specification. "Respond to a customer change request on an open order, applying only the changes the order's state allows, and tell the customer what cannot be changed." One sentence is enough; it sets what every case will be graded on.

2. Event reconstruction note. The event types needed to rebuild an order at the request's received time: order creation, line edits, picks, shipments, address changes and invoice issue. For each, state whether it carries a timestamp, a user and an order line reference, and which system holds it. Warehouse and carrier systems often keep better times than the order desk, so name all of them. Toloka's description (checked 2026-10-18) of versioned environments and audit logs for exact reproduction is a fair benchmark for the detail a buyer will look for.

3. Request record description. Where change requests arrive (email, portal, phone notes, EDI) and whether the received time is stored on the order or only in an inbox. Give rough shares by channel if known.

Documents 4 and 5: what was allowed, and what counts as correct

4. Allowed actions. The moves available to a responder: cancel open quantity, add lines, change the address for unshipped lines, offer a return authorization for shipped goods, escalate pricing changes to a named role. Order systems usually enforce part of this; Shopify, for instance, lets merchants edit unfulfilled items but not remove fulfilled items or change their quantity. Record who may approve returns and credits, and up to what amount.

5. Acceptance check. Written before any outcome is examined, so nobody fits the rule to the result. Illustrative, using the main guide's 10:42 order:

  • ☐
    No change is applied to quantity already shipped at the received time (the 150 glove cartons stay shipped).
  • ☐
    Open quantity is adjusted as requested (the 50 open cartons are cancelled).
  • ☐
    The address change applies only to unshipped lines (the first-aid kits).
  • ☐
    The customer is told in writing which parts could not change.
  • ☐
    Any return offer, such as for 30 cartons, is approved by a permitted role.

A response that edits the shipped line to 120 fails the first condition whatever the customer later accepted.

Documents 6 and 7: scale, span and rights

6. Volume and span. Change requests per year; the share arriving after partial shipment, if known; years of timed event history; and the date of any ERP or warehouse system migration, with a note on whether timestamps and edit history survived it.

7. Rights and sensitivity note. Customer names, site addresses, pricing and contact details appear throughout. List customer contracts that limit reuse, any third-party logistics provider that holds shipment logs, and the executive who can authorize a licensing discussion. Owners sometimes ask whether licensing such records makes them a data broker; the explainer on what a data broker is under state law answers that, and data licensing and cyber insurance coverage covers a question CFOs raise next.

Illustrative cover sheet

Fictional, for a 460-person foodservice equipment distributor.

FieldEntry
TaskRespond to change requests on open orders within what the order state allows
Timed eventsPicks and shipments timed in the warehouse system for five years
Edit historyOrder-line change log active for three years, with old values
RequestsAbout half via portal with received time; the rest by email
Allowed actionsReturns approved by branch managers up to a set limit; pricing to sales operations
MigrationWarehouse system replaced four years ago; earlier history dated only
AuthorizerCEO, with the COO preparing the case

Browser-based order portals produce a related kind of record, described in browser agent training data. Eligibility still applies: 50 or more full-time employees at peak, documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. Other record families are under data types.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the operations lead could fill the event reconstruction note and acceptance check from systems already in place, register as a partner, confirm the company against who qualifies and read the terms, then introduce the COO and let the company prepare the packet with SourceX.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Who in the company leads the packet?

Usually the COO or VP operations, with the warehouse manager and ERP administrator supplying event and change-log facts. The CFO or counsel prepares the rights note, which names the CEO as the person who can authorize licensing.

Does the packet need example orders?

No. It names event types, fields and counts. Example records are exchanged only later, under an agreement and SourceX's de-identification process.

What if a logistics provider holds the shipment events?

Name the provider in the event reconstruction document and flag its contract in the rights note. Whether those logs can be licensed is settled in SourceX's review with the company.

Sources

  1. gymnasium.farama.org/api/env
  2. arxiv.org/abs/2406.12045
  3. help.shopify.com/en/manual/fulfillment/managing-orders/editin…
  4. toloka.ai/agent-data
  5. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment