Quality scoring disagreements history: what to inventory before a QA vendor change

Short answer

Before a QA vendor change, inventory the records behind scoring disagreements: calibration sessions, each reviewer's score for the same interaction, the call snippets they cited, the adjudication notes that settled the split, and the form versions in force at the time. Note which platform holds each, how many years are covered, who owns them and when the old vendor's access ends. Keep it at metadata level, then introduce the quality director, who can bring in the executive with authority to license.

Visual briefFull-size diagram
Two panels for an old and a new QA vendor: scoring forms and open evaluations move across, while calibration sessions, adjudication notes and past form versions stay behind in the old system, and recordings stay in the contact center suite under its own retention.
Illustrative. Which records move depends on the company's own migration plan and vendor contract.

What should be inventoried before a QA vendor change affects quality scoring disagreements history?

Record classes, coverage, owners and the real end of access, for both the QA platform being replaced and the contact center suite that holds the recordings. A vendor change usually moves forms and future evaluations. Past calibration sessions and adjudication comments are the history most likely to stay behind in the old vendor's system.

Record classUsual homeWhy it may not moveAsk the quality director
Completed evaluations with item scoresQA platformOnly recent or open evaluations are importedHow many years of closed evaluations can still be reported on?
Calibration sessions, one score per reviewerQA platformOften stored as a separate session type the new tool does not mapWhen did calibration start, and how often did it run?
Adjudication and dispute notesQA platform comments, emailFree text rarely survives a field mappingIs the final ruling tied to the session, or kept elsewhere?
Scoring form versionsQA platform settingsOnly the current form is rebuiltWere old forms versioned or overwritten?
Call recordings and transcriptsContact center suiteSeparate system with its own retentionWhat retention setting applies, and has it changed?
Coaching records linked to scoresQA platform, LMS, team leads' filesTreated as HR, not QAWho owns coaching notes and how long are they kept?

A partner collects answers, not files: no recordings, score exports or logins. The QA scorecard data page describes what a scorecard record contains; this page is about making sure the history is counted before the old vendor's door closes.

Why does calibration history fall out of a QA vendor change?

Because calibration is a comparison, and migrations move single records. A calibration session holds several evaluations of the same interaction plus the one treated as correct. Genesys, for example, documents calibration sessions that keep scoring consistent across evaluators and let quality administrators compare scoring variations; one evaluation can be marked as the scoring index. If only individual evaluations are exported, the grouping and the index can be lost, and with them the record of which reviewer was overruled.

Three details are worth asking about:

  • Agreement figures. Some teams tracked inter-rater agreement for calibration. McHugh's review of the kappa statistic explains why chance-corrected agreement was introduced: percent agreement alone ignores the agreement two reviewers would reach by chance. Ask whether the team kept agreement reports, and which measure they used. The figure itself is a useful metadata answer.
  • Form drift. A disagreement in 2022 was settled against the form in force in 2022. If old form versions are overwritten when the new vendor's forms are built, past rulings lose their rulebook.
  • Snippets that point to deleted audio. An evaluation that cites a call segment keeps the score even after the recording is purged. Note which years still have both.

One regulatory point can affect what still exists. If the center makes outbound telemarketing calls, 16 CFR 310.5 requires sellers and telemarketers to keep specified records, including call detail records, for five years. That is a retention duty for those records, not permission to license them, and it does not cover every QA artifact.

An illustrative inventory sheet

Illustrative only; the company and dates are fictional. A 350-person insurance services contact center is replacing its QA platform at the end of its vendor term. The quality director fills in the sheet from the migration plan, without opening any records.

Record classSystemYears heldOwnerMoves to new vendor?Reachable until
Completed evaluationsLegacy QA platform2018 to nowQuality directorLast 12 months onlyOld QA contract end
Calibration sessionsLegacy QA platform2020 to nowQA calibration leadNoOld QA contract end
Adjudication notesPlatform comments and email2020 to nowQA calibration leadNoOld QA contract end; mailbox policy for email
Scoring form versionsLegacy QA platformCurrent form plus three archivedQuality directorCurrent form rebuiltOld QA contract end
Call recordingsContact center suiteRolling retention windowIT and complianceNot affectedSuite retention setting
Coaching recordsLMS and team lead foldersVariesOperations and HRNoHR retention policy

Reading the sheet: the classes at risk are calibration sessions, adjudication notes and archived forms, all tied to one contract date. The test to run before go-live is whether calibration sessions can be exported with reviewer, scoring index and comments intact. That answer comes from the company's own test, not a vendor's general claim. Contract terms on getting data out are covered in SaaS data export rights, and the data scope worksheet gives the company a format for its own counts.

Which dates set the real access deadline?

Use the company's documents, never an assumed date. Usually four matter: new vendor go-live, the end of the old QA contract, any read-only or export window after it, and the recording retention setting in the contact center suite. If none is written down, there is no deadline yet, and the next step is for the company to set one, not for a partner to invent urgency.

Do not recommend keeping data the company has no authority or reason to keep. Recordings capture customers' voices and personal details; the guide to recordings at platform retirement covers the notice, consent and redaction questions the company decides with its counsel.

Ask the quality director about coverage and rights

The quality director knows coverage; rights sit higher. Recording ownership can depend on client contracts and consent notices (see who owns call recordings), so counsel decides what could ever be licensed and what would need removing or de-identifying first. The person who can authorize a licensing discussion is the CEO, often with the COO or CFO sponsoring the conversation internally.

The introduced company needs 50+ full-time employees at peak (contractors excluded), a documented operating history, rights to license the records, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 90% of the data is in English. Unknown until SourceX reviews it: buyer interest, the rights review result and any terms. Partners sign the Channel Partner Agreement before submitting introductions; attribution goes to the first valid introduction within the 90-day window, credited when the company submits a verified application. The reward calculation is on the rewards page, and other sectors are on the industries hub.

Go deeper on quality scoring disagreements

Next step

Refer a company. Earn up to $100,000. Default reward: 25% of Actual Supplier Payout, capped at $100,000 cumulatively per referred company. Eligibility and payment conditions apply, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the QA vendor change would strand years of calibration sessions with their rulings, register as a partner, check the company against who qualifies, and read the program terms before you send the quality director your brief.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardYour 25% of the actual supplier payout. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Are calibration sessions more useful than ordinary QA evaluations?

They carry different information. Ordinary evaluations show one reviewer's judgment; calibration shows two or more judgments of the same interaction and, where they split, the reasoning used to settle it. Whether buyers want either is decided in SourceX's review.

Should the company extend its old QA contract to keep the history?

That is the company's decision, made by its quality, finance and legal leads against its retention policy. A partner can point out the access date and what sits behind it, but should not recommend keeping data the company has no reason or authority to keep.

What if the contact center is an outsourcer handling calls for clients?

Then recordings and many scores may belong to or be controlled by the client under the services agreement, and the outsourcer cannot include them without client consent. Its own QA methods, forms and calibration practice are a separate question for the rights review.

Sources

  1. refer.sourcex.si/terms
  2. help.mypurecloud.com
  3. pmc.ncbi.nlm.nih.gov/articles/PMC3900052
  4. ecfr.gov/current/title-16/chapter-I/subchapter-C/part…

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 90% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment