This page is part of the appeal-to-course-correction records guide. It reads one Illustrative case twice, first from the LMS alone and then with its surroundings, and lists where each missing piece is kept. The record chain page covers the keys that join those systems.
Before: the rubric change as an LMS export shows it
Illustrative. A fictional 310-person workplace safety training company runs a site supervisor certificate on Canvas. Its practical assessment is scored against a rubric whose criterion B covers the learner's hazard log. An export of the course from the LMS shows this for criterion B:
| Field in the LMS export | Value |
|---|---|
| Course | Site supervisor certificate, autumn template |
| Rubric | Practical assessment rubric |
| Criterion B, current wording | Requires each hazard entry to state a control and an owner |
| Last edited | Mid-August |
| Edited by | Program designer |
| Courses linked to the template | Four autumn delivery groups |
Taken alone, this is a tidy record of a routine edit. A reader would reasonably conclude that the program designer tightened a criterion while preparing the autumn intake, and that all four delivery groups now use it. Both conclusions turn out to be incomplete.
After: the same change with its ticket and appeal around it
Now add what the support desk and the assessment platform hold for the same summer, still Illustrative.
| When | Source | What it adds |
|---|---|---|
| Early June | Support desk | A spring-intake learner disputes a fail on criterion B; the ticket is tagged and routed to assessment |
| Late June | Assessment platform | The review panel overturns the fail: the old wording never mentioned owners, yet markers expected them |
| July | Assessment platform | Panel recommendation: state the expectation in the criterion itself |
| Mid-August | LMS | Criterion B reworded in the autumn template and pushed to linked courses |
| Mid-August | LMS sync history | One delivery group, where a trainer had adjusted the rubric locally, kept the spring text |
| Never | Assessment platform | No note on whether other spring fails on criterion B were looked at again |
The edit is no longer routine. It answers an overturned fail, it was made for the next intake rather than the one that raised the problem, and one delivery group never received it. The LMS export was accurate; it simply could not say any of that.
What goes missing, and where it actually lives
| Question a reader would ask | LMS export alone | Where the answer lives | Who usually owns it |
|---|---|---|---|
| What does criterion B say now? | Yes | LMS | Program design lead |
| What did it say in spring? | Only if rubric history is kept | LMS rubric history | Program design lead |
| Why was it reworded? | No | Panel recommendation in the assessment platform | Assessment manager |
| Who raised the problem, and when? | No | Ticket history in the support desk | Learner services lead |
| Which intake was affected? | No | Appeal record and its intake label | Assessment manager |
| Which delivery groups got the new text? | Partly | Sync history and lock settings | LMS administrator |
| Were other spring fails revisited? | No | A re-marking log, if one exists | Assessment manager |
Three vendor behaviors explain the table. Zendesk's ticket audits keep a read-only history of each update, so the June tagging and routing usually survive in the support desk. Instructure's documentation says a Canvas blueprint sync overwrites locked items in associated courses but leaves unlocked items to instructors, and later syncs are manual, which is why "pushed to linked courses" and "received by every group" are different facts. Moodle's documentation describes flagging existing assessments "Needs review" after a significant grading form change; where a platform records that, the last row has an answer.
Why the missing context matters to an AI developer
A rubric on its own is a list of criteria. A rubric with the appeal that exposed its weakness, the panel's reasoning and the revised wording is a worked record of how people decide edge cases in assessment. That is the material the rubric-based evaluation guide describes as useful for evaluating AI systems, and it is the kind of decision history that what data is missing from AI training identifies as scarce in public sources.
Strip the context away and the decision disappears, leaving only its result. That is why a company that has kept all three systems for overlapping years may hold something more useful than one with a clean LMS archive and nothing else. It is also why curriculum ownership matters: the training materials ownership guide explains when course and rubric history belong to the company.
Asking how much context survived, without seeing any of it
Walk the client through the "where it lives" column and ask, for each row, whether that system still holds the answer and since which year. Write down the answers, not the records. Useful follow-ups: is rubric history kept in the LMS, are rubrics locked in blueprints, and is a re-review of earlier grades ever logged? The completeness checklist turns those answers into a pass, partial or fail score.
The people who answer are the learner services lead, the assessment manager and the program design lead. The person who can authorize a licensing discussion is the CEO, usually the owner in an owner-run provider, with a COO or chief academic officer as a likely sponsor. The company needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.
Unknown until SourceX reviews it under the company's agreement: how learner identities would be removed, whether student-record rules apply, what a buyer would want and what the company would be paid. Partners never handle learner work, tickets or grades.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
If the client still holds the reason, the cohort and the reach of its rubric changes in systems it owns, register as a partner, check who qualifies, and read the terms before you make the introduction.