What does partial-credit evaluation mean in an ordinary business workflow?
Partial-credit evaluation scores a task by its steps. Instead of a single pass or fail, the task is split into checkpoints, and an attempt earns credit for each checkpoint it completes, usually with extra weight for finishing the whole thing.
The idea matters to AI developers because most real work is long and fails late. A software agent that gathers every document for a new supplier and then forgets to route the file for approval has done most of the work and still left the company unable to pay that supplier. A pass-or-fail score treats it the same as an agent that did nothing. A step-by-step score shows exactly where it stopped. Our explainer on what model evaluation is covers why developers need test material like this, and the guide to AI agent benchmarks shows how published tests are built.
Research benchmarks already publish the arithmetic. TheAgentCompany, a benchmark of workplace tasks, scores each task with checkpoints. Its full completion score is 1 only if every checkpoint passes. Its partial completion score is half the share of checkpoint points earned plus half the full completion score, so an attempt that finishes earns an extra 50% that an unfinished one cannot. Business processes with recorded, separately approved steps are the raw material for scoring of this kind.
Scoring one vendor onboarding three ways (Illustrative)
This case is fictional. A 210-person facilities services company adds a new HVAC subcontractor. Its onboarding procedure has four checkpoints, each worth one point, and the company runs Dynamics 365 Finance with vendor approvals switched on.
| Checkpoint | What counts as done | Record it leaves | Attempt status |
|---|---|---|---|
| 1. Tax form | Signed Form W-9 with legal name and taxpayer identification number on file | Document attached to the vendor request | Done |
| 2. Insurance | Certificate of insurance on file, valid on the start date | Document with expiry date | Done |
| 3. Bank details | Account submitted to the bank account approval workflow | Workflow entry with status | Done |
| 4. Final authorization | Controller approves the proposed vendor record, which makes the vendor active | Workflow approval with approver and time | Not done |
The IRS describes Form W-9 as the way a payee gives a payer its correct taxpayer identification number for information returns, which is why it is the first checkpoint. Checkpoint 4 is where the process actually takes effect. Microsoft's documentation for the vendor approval workflow says edits to approval-flagged fields are held as proposed changes, and the proposed values are written to the vendor record only after every approval step is complete. Bank details have their own separate workflow, in which approvers can reject, delegate or cancel. So after this attempt, three documents sit in the request and the vendor still cannot be paid.
Here is the same attempt scored three ways:
| Scoring rule | Arithmetic | Score |
|---|---|---|
| Pass or fail | Not every checkpoint passed | 0% |
| Share of checkpoints | 3 of 4 checkpoints | 75% |
| Weighted partial (TheAgentCompany formula) | 0.5 × 3/4 + 0.5 × 0 | 37.5% |
| Weighted partial, if the controller had approved | 0.5 × 4/4 + 0.5 × 1 | 100% |
The share-of-checkpoints row overstates progress, since the one missing step is the one that matters. The pass-or-fail row hides the progress entirely. The weighted rule sits between them and keeps a finished onboarding far ahead of an unfinished one. Which rule a buyer uses is the buyer's choice. What a company's records must supply is the evidence to apply any of them: each checkpoint recorded separately, with a date and an approver.
Evidence fields each checkpoint needs
A checkpoint can only earn credit if a reviewer can tell, from the record alone, that it happened, when, and on whose authority. For the onboarding above, that means four fields per checkpoint plus one for the task as a whole.
| Field | Why the score depends on it | Onboarding example (Illustrative) |
|---|---|---|
| Checkpoint identity | Credit is assigned per step, so steps must be distinguishable | Tax form, insurance, bank details, final authorization |
| Completion evidence | A step counts only if its output exists | W-9 attached, certificate attached, bank workflow entry |
| Timestamp | Order and timing matter when a later step depends on an earlier one | Certificate valid on the start date |
| Actor and approver | Shows the step was done by someone allowed to do it | Controller named on the approval, or missing |
| Final state of the task | Decides the finishing bonus | Vendor proposed, not active |
The fifth field is the one most often lost. When a request is abandoned, some systems delete it or leave it in a draft state that reports never show. An archive that keeps only finished onboardings can be scored pass or fail, but not for partial credit, because every attempt in it passed.
One example cannot show how many attempts stalled, how many were later completed under a new request, or whether stalled requests were purged. Those are archive-level questions that SourceX answers in its review with the company.
Does the company qualify, and what stays unknown?
The company needs 50 or more full-time employees at peak (contractors excluded), documented operating history, rights to license its records, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. The company qualification checklist covers the basics.
Unknown until SourceX reviews it: how many onboardings carry complete step records, how tax identifiers and bank details are removed or de-identified, whether software vendor terms limit use, and whether any buyer wants this task type. Partners never request exports, screenshots or sample files. Related concepts are in the data types hub.
Go deeper on partial-credit evaluation
- Questions to ask a client about partial-credit evaluation: a discovery list for a controller or operations lead, kept at metadata level.
- Record gaps that make partial-credit evaluation hard to assess: the missing step records that most often prevent scoring, and how to spot them early.
- Partial-credit evaluation review packet: what an evidence packet for SourceX's review should contain, item by item.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the controller confirms that each onboarding step is recorded with its own date and approver and that stalled requests stay in the system, register as a partner, check the company against who qualifies, and read the terms before you introduce the CFO or COO.