# Partial-credit evaluation review packet | SourceX

Canonical: https://refer.sourcex.si/resources/partial-credit-evaluation-review-packet
Last updated: 2026-11-04

A partial-credit evaluation review packet holds eight metadata documents, assembled in this order: a cover sheet, the task specification, the checkpoint list with weights, the procedure versions, an evidence map per checkpoint, the exact outcome check, volume and span counts, and rights notes. Together they let a reviewer score stepwise work, such as a vendor onboarding that stalled before final approval, without seeing a single vendor record. The company's finance, ERP and legal staff prepare them.

This page belongs to the guide on [partial-credit evaluation in a vendor onboarding](/guides/concept-partial-credit-evaluation), which scores one stalled onboarding three ways. That guide explains the scoring. This page is the shopping list: what a controller's team would put on the table once a licensing discussion begins, so an owner can judge the effort before committing to it.

## Packet contents in assembly order

| Order | Document | What it shows the reviewer | Prepared by |
|---|---|---|---|
| 1 | Cover sheet | ERP name, years covered, onboardings per year, records owner | Controller |
| 2 | Task specification | The onboarding job in two or three sentences, as the company defines it | AP manager with the controller |
| 3 | Checkpoint list and weights | Each step in order, its points, and which steps are mandatory | AP manager |
| 4 | Procedure versions | Which written procedure applied on which dates | Finance operations or internal audit |
| 5 | Evidence map per checkpoint | The system, record and date and approver fields that prove each step | ERP administrator |
| 6 | Exact outcome check | The test that decides whether the whole task finished | Controller, refined with SourceX |
| 7 | Volume and span counts | Onboardings per month, share that stalled, months retained, system changes | ERP administrator with finance |
| 8 | Rights and sensitivity notes | What is personal, financial or contract-restricted | General counsel or outside counsel |

The order is deliberate. Documents 1 to 3 define the work; 4 and 5 show whether the records can prove it; 6 makes it gradable; 7 and 8 say how much exists and what limits its use.

## What each document contains

**1. Cover sheet.** One page: the ERP or procurement tool in use and when it was adopted, any earlier system, the vendor types onboarded (subcontractors, suppliers, service providers), rough onboarding counts per year, and the executive who owns the records. A reviewer sizes the archive from this page before reading anything else.

**2. Task specification.** The job written as the company would brief a new AP clerk. Illustrative: "Onboard a new subcontractor so it can be paid, following the written procedure in force on the request date. The task is complete when the vendor is active in the ERP." Short and exact beats long and vague.

**3. Checkpoint list and weights.** The steps in order, with the points each is worth and a flag on any step without which the task cannot finish. The Illustrative sheet below extends the main guide's four-step onboarding to the six steps a fictional firm uses. Weights are the company's view of effort and importance; a buyer may reweight them, but it needs the company's starting point.

**4. Procedure versions.** Every version of the onboarding procedure in the covered years, with effective dates. If an insurance step was added in 2023, attempts from 2022 must not lose credit for skipping it. Where no written procedure existed for part of the period, say so plainly.

**5. Evidence map per checkpoint.** For each step: the system that holds the proof, the record type, and the field that carries the date and the approver. This is the document that turns a procedure into something checkable from records alone.

**6. Exact outcome check.** The pass or fail test for the whole task, written before anyone samples cases. It is covered in detail in its own section below.

**7. Volume and span counts.** Onboardings started per month, the share that stalled before approval, months of workflow history still retained, and the dates of any ERP or procurement tool changes. Stalled attempts matter more here than in most packets; see the next section.

**8. Rights and sensitivity notes.** Categories, not cases: taxpayer identification numbers, bank details, insurance documents, employee names in approvals, and any vendor contract terms that limit reuse. Any request that got as far as a W-9 holds a taxpayer identification number, since giving a payer that number for information returns is what the IRS says the form is for, and it needs de-identification.

## An Illustrative checkpoint sheet (document 3)

**Illustrative and fictional.** A 230-person electrical contracting firm fills in document 3 for subcontractor onboarding. Its written procedure has six steps, two more than the main guide's example, and the firm weights them by effort and risk rather than one point apiece:

| Step | Checkpoint | Points | Blocks completion if missing | Proof field named in document 5 |
|---|---|---|---|---|
| 1 | Prequalification questionnaire returned | 1 | No | Received date on the vendor request |
| 2 | Signed W-9 received | 1 | No | Attachment date on the vendor request |
| 3 | Insurance certificate names the firm as additional insured | 2 | No | Certificate attachment with expiry date |
| 4 | Safety record reviewed by the safety manager | 2 | No | Review note with reviewer and date |
| 5 | Bank account approved in its own workflow | 1 | Yes | Bank account workflow status and approver |
| 6 | Controller activates the vendor record | 3 | Yes | Vendor workflow approval, approver and time |
| All | Six checkpoints | 10 | Steps 5 and 6 | Five dated system fields and one review note |

Two steps block completion here, not one. As Microsoft documents the Dynamics 365 Finance vendor workflow, proposed changes to approval-flagged fields reach the vendor record only once every approval step has finished, so step 6 is where onboarding takes effect. Bank details follow a workflow of their own, in which an approver may reject, delegate or cancel a submission, so step 5 can stop payment by itself even after the controller signs.

## How the weights move a stalled attempt's score

The weights are the reason document 3 exists. Applied to the sheet above, they rank stalled attempts by how much real work each one did, which a simple count of steps would not. Illustrative attempts, scored with the benchmark formula described in the next section:

| Attempt stopped after | Points earned | Share of points | Weighted partial score |
|---|---|---|---|
| Step 2, tax form | 2 of 10 | 20% | 10% |
| Step 4, safety review | 6 of 10 | 60% | 30% |
| Step 5, bank approval | 7 of 10 | 70% | 35% |
| Step 6, vendor active | 10 of 10 | 100% | 100% |

An attempt that cleared the insurance and safety steps scores three times one that stopped at the tax form, and the finished onboarding still sits far above all of them. If the firm had weighted every step equally, stopping after step 4 would score 4 of 6 points instead, and the safety review would count for no more than the questionnaire. A buyer may reweight, but the company's own weights are its statement of where the effort lies.

## The outcome check, and why stalled attempts belong in the packet

Document 6 is the heart of the packet. Illustrative wording for the firm above: "Onboarding is complete only when the vendor record is active, its bank account has passed its own approval, and steps 1 to 4 each carry a dated record. An inactive vendor means an incomplete attempt, whatever documents are attached."

Fixed in writing before anyone samples a case, that sentence lets anyone holding the records decide which checkpoints passed and whether the task finished. The TheAgentCompany benchmark is built the same way: a task earns a full completion score of 1 only when all of its checkpoints pass, and its partial score averages the share of checkpoint points earned with that full score. Our explainers on [what model evaluation is](/proof/what-is-model-evaluation) and [verifiable rewards](/guides/reinforcement-learning-verifiable-rewards) explain why developers want tasks with a clear end state.

The same logic explains why document 7 counts stalled onboardings. If the ERP purges abandoned requests, or its reports list only active vendors, every surviving case scores 100% and the score table above cannot be filled in. The [record gaps page](/questions/partial-credit-evaluation-evidence-gaps) describes how that loss usually happens.

## What the packet leaves open

The packet describes and sizes the archive. It does not decide whether the records qualify or are wanted. The company must have 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

Still unknown once the packet exists: how many onboardings pass sampling against the evidence map, how tax and bank details are de-identified, whether vendor contracts restrict use, whether any buyer wants this task type, and any offer. Partners never request exports, screenshots or sample files. To check fit before any of this, use the [discovery questions for partial-credit evaluation](/questions/does-partial-credit-evaluation-data-qualify). For the surrounding approval records, see [how to assess procurement approvals and exception workflows](/guides/how-to-assess-procurement-approvals-and-exception-workflows) and [vendor bank account change verification](/data-types/vendor-bank-account-change-verification-process); the [M&A contract review resource](/resources/mcp/mcp-manda-contract-review) shows records described at metadata level in a deal setting, and more record families are in [data types](/data-types).

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the controller can picture the team filling in these eight documents, including counts of stalled onboardings, [register as a partner](/register), confirm the basics on [who qualifies](/who-qualifies), and read the [program terms](/terms) before you make the introduction.

## Sources

- https://arxiv.org/abs/2412.14161
- https://learn.microsoft.com/en-us/dynamics365/finance/accounts-payable/vendor-workflow
- https://learn.microsoft.com/en-us/dynamics365/finance/accounts-payable/vendor-bank-account-workflow
- https://www.irs.gov/forms-pubs/about-form-w-9
- https://refer.sourcex.si/terms
