QA-scored support interactions
Support conversations that a company’s quality team graded against a rubric, with a score for each criterion and the reviewer’s comments. Because each interaction carries a human judgment of quality, labs use them for reward models, preference data and evaluation rubrics.
What a record contains
One graded interaction: the conversation, the rubric version, a score for each criterion, the reviewer’s comments and any coaching notes.
| Field | What it holds |
|---|---|
interaction_id | Links to the ticket, chat or call |
rubric_version | Which rubric applied |
scores{} | Score per criterion (for example empathy, accuracy, policy) |
reviewer_comment | Free-text feedback |
calibration | Whether several reviewers scored the same interaction |
{
"interaction_id": "T-58213",
"rubric_version": "2024-Q1",
"scores": {"accuracy": 4, "policy_compliance": 5, "tone": 3, "resolution": 5},
"reviewer_comment": "Correct refund, but opened with a template apology.",
"calibration": false
}How AI labs use it
- Reward models
- Rubric scores are graded, human-labeled quality signals.
- Preference pairs
- Higher- and lower-scored answers to similar issues form natural comparisons.
- Calibrating automated graders
- Check an LLM judge against the company’s own reviewers.
Typical preparation requirements
Agreed with the supplier before any work begins. Typical requirements include:
- Agent and reviewer identities pseudonymized
- The underlying conversations de-identified like any support data
- Rubric documents included so scores can be interpreted
Every dataset has a documented owner and confirmed licensing rights. See data governance on sourcex.si.
What makes a strong package
- A stable or clearly versioned rubric
- Calibration sessions on shared interactions
- Written comments, not just numbers
Compared with public datasets
Public sets such as ABCD and Customer Support on Twitter are useful references, but limited as enterprise training data. The customer support category page compares them with licensed data.
Who typically holds it
- In-house contact centers
- BPOs, with their clients’ permission
- SaaS support teams with QA programs
- Insurers and financial services firms
Know a company like this?
Introduce the company to SourceX. If its data deal closes, you can earn up to $100,000 in referral fees, paid after the buyer accepts the data and SourceX receives payment.
Refer a companyQuestions
What if the rubric changed over time?
That’s common. Packages can include each rubric version and which one applied to each score.
Are agents’ performance records included?
Scores are tied to pseudonymous IDs, not named employees, and HR records are out of scope.
Need this data for a model?
Describe what you need: domain, volume, history, format and licensing terms. SourceX looks for companies that hold it and can license it.