This analysis belongs to how to assess human review records and correction workflows, which walks one disputed case through four gates. This page does a different job: it compares what a review expects against what companies typically keep, piece by piece, so an introducer can describe an incomplete archive accurately.
Expected versus available: the gap table
The middle columns are typical patterns from QA and support operations, not measurements. Every company differs, which is why the last column is a question to ask, not a conclusion.
| Expected evidence | Why a review wants it | What companies typically have | What is usually missing | Question for the QA lead |
|---|---|---|---|---|
| Case material | The input both reviewers judged | Ticket thread while retention allows | Closed tickets past the deletion window | How long are closed tickets kept? |
| First score | One reasonable reading | Stored in the QA tool | Overwritten when a dispute succeeds | Is the original score kept after a dispute? |
| Second score | The competing reading | Sometimes, in a calibration or appeal view | Never exported with the main QA data | Where do second opinions live? |
| Reviewer comments | The reasoning behind each score | Free-text fields, unevenly used | Blank or one-word comments | Are comments required on disputed reviews? |
| Rubric version | The rule in force that day | Current rubric only | Old wording after in-place edits | Can last year's rubric be shown with its date? |
| Ruling and reason | How the disagreement was settled | Final score | The reason, given verbally or in chat | Where is a lead's ruling written down? |
| Rubric change | Whether the ruling changed the rule | Rarely linked | No reference from rubric change to case | Do rubric edits cite the case that prompted them? |
The pattern is consistent: evidence near the start of the chain survives better than evidence at the end. A company often holds thousands of scored tickets and very few rulings with reasons, and that difference decides how the archive should be described.
Worked gap profile: a disputed escalation at an IT service desk
Illustrative and fictional. A 520-person managed IT services company runs a service desk for client firms. An agent escalates a password-reset request to the security team instead of resetting it on the call, because the caller could not answer the second verification question.
- Reviewer A scores the escalation unnecessary: the knowledge base allowed a reset after one failed question if the caller's phone number matched.
- Reviewer B scores it correct: the client's contract required escalation for any failed verification on administrator accounts, and this was one.
- The QA lead rules for Reviewer B and adds a line to the rubric: check the account type before applying the general reset rule.
Now the gap profile, as the company's QA lead might describe it at metadata level:
| Evidence | Status at this company |
|---|---|
| Case material | Kept: tickets retained five years |
| First score | Kept, but replaced by the ruling in reports |
| Second score | Partial: in the calibration view, not in the main export |
| Reviewer comments | Partial: Reviewer B wrote two lines, Reviewer A none |
| Rubric version | Missing: rubric edited in a shared document with no dated history kept |
| Ruling and reason | Kept: lead's note on the review |
| Rubric change | Missing: the new line does not cite the case |
This archive still has value: case, scores and rulings with reasons. What it cannot show is the rule as it read before the change. That is a fair, specific sentence for an introduction.
Where the gaps come from in ordinary tools
Most gaps are produced by normal settings, not neglect.
Case material and deletion schedules. In Zendesk, a ticket deletion schedule continuously deletes archived tickets that were closed and not modified for a set period, and deleted tickets cannot be restored. A company with a short schedule may have rich QA data with no source to check it against.
Second scores in a separate view. In Zendesk QA, calibration reviews, comments and scores appear in a separate Calibration dashboard, not on the main dashboard. A team that reports only from the main view may not realize it holds paired scores at all.
Rubric versions. Where a rubric lives decides whether old wording survives. Confluence Cloud, for example, keeps page history and lets a user select two versions and compare them or restore an earlier one. A rubric kept in a wiki with page history may be reconstructable; one pasted into slides each quarter usually is not.
Agent-data vendors describe this kind of evidence too: Appen's agentic AI page, checked on October 18, 2026, lists failure-mode taxonomies and golden trajectories among its agent data services. Disagreements settled with a reason are close to that material. A vendor's service list is not evidence that any buyer wants a given archive, or that it qualifies.
Writing the gap into the introduction
The honest sentence is short and names what is missing. Three patterns cover most cases:
- Strong core, missing rules: "Disputed reviews with both scores and written rulings for four years; rubric changes were not versioned."
- Scores without reasoning: "Paired scores exist in the calibration view; comments and rulings are sparse."
- Outcomes only: "Final QA scores with ticket history; second opinions were not kept."
Avoid words that imply completeness, such as "full audit trail", unless the QA lead has confirmed every piece. An overstated introduction costs the owner time when SourceX's review finds the gap. Related record types are described under human feedback and review records and call center QA scorecards, and the wider idea is explained in what human-in-the-loop data is.
What the partner never asks for, and what SourceX decides
A partner records metadata: which tools, how many years, rough volumes of disputes, who owns the QA program and who can authorize a licensing discussion. A partner never asks for tickets, reviews, rubric files or reviewer names.
What stays unknown until SourceX reviews it: whether client contracts allow ticket content to be licensed, how agent and customer identities would be removed, whether reviewer comments can be used, and whether any buyer wants this record type. If the company has cases where the expected outcome is clear, the guide to quality assurance examples with an expected outcome shows a related pattern. Other record families are listed under data types.
The company also needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the QA lead can confirm case material, both scores and written rulings for most disputes, register as a partner, check the company against who qualifies and read the terms, then introduce the executive with the gap stated plainly.