Appeal-to-course-correction missing context: one rubric fix read from an LMS export alone, then with its ticket and appeal

Short answer

A single-system export loses the reason and the reach. An LMS export can show the current rubric and sometimes when it was edited. It cannot show that a learner's appeal caused the edit, which intake raised the problem, which delivery groups actually received the new wording, or whether learners graded under the old wording were reviewed. Those facts live in the support desk, the assessment platform and the LMS sync history, and they only make sense read together.

Visual briefFull-size diagram
A rubric change record with four blank fields (why, intake, reach and re-marking) that fill in as pieces from the support desk, the assessment platform and the LMS sync history slide into place, leaving one field still unknown.
Illustrative. The LMS shows what changed; the reason, the intake and the reach live in other systems.

This page is part of the appeal-to-course-correction records guide. It reads one Illustrative case twice, first from the LMS alone and then with its surroundings, and lists where each missing piece is kept. The record chain page covers the keys that join those systems.

Before: the rubric change as an LMS export shows it

Illustrative. A fictional 310-person workplace safety training company runs a site supervisor certificate on Canvas. Its practical assessment is scored against a rubric whose criterion B covers the learner's hazard log. An export of the course from the LMS shows this for criterion B:

Field in the LMS exportValue
CourseSite supervisor certificate, autumn template
RubricPractical assessment rubric
Criterion B, current wordingRequires each hazard entry to state a control and an owner
Last editedMid-August
Edited byProgram designer
Courses linked to the templateFour autumn delivery groups

Taken alone, this is a tidy record of a routine edit. A reader would reasonably conclude that the program designer tightened a criterion while preparing the autumn intake, and that all four delivery groups now use it. Both conclusions turn out to be incomplete.

After: the same change with its ticket and appeal around it

Now add what the support desk and the assessment platform hold for the same summer, still Illustrative.

WhenSourceWhat it adds
Early JuneSupport deskA spring-intake learner disputes a fail on criterion B; the ticket is tagged and routed to assessment
Late JuneAssessment platformThe review panel overturns the fail: the old wording never mentioned owners, yet markers expected them
JulyAssessment platformPanel recommendation: state the expectation in the criterion itself
Mid-AugustLMSCriterion B reworded in the autumn template and pushed to linked courses
Mid-AugustLMS sync historyOne delivery group, where a trainer had adjusted the rubric locally, kept the spring text
NeverAssessment platformNo note on whether other spring fails on criterion B were looked at again

The edit is no longer routine. It answers an overturned fail, it was made for the next intake rather than the one that raised the problem, and one delivery group never received it. The LMS export was accurate; it simply could not say any of that.

What goes missing, and where it actually lives

Question a reader would askLMS export aloneWhere the answer livesWho usually owns it
What does criterion B say now?YesLMSProgram design lead
What did it say in spring?Only if rubric history is keptLMS rubric historyProgram design lead
Why was it reworded?NoPanel recommendation in the assessment platformAssessment manager
Who raised the problem, and when?NoTicket history in the support deskLearner services lead
Which intake was affected?NoAppeal record and its intake labelAssessment manager
Which delivery groups got the new text?PartlySync history and lock settingsLMS administrator
Were other spring fails revisited?NoA re-marking log, if one existsAssessment manager

Three vendor behaviors explain the table. Zendesk's ticket audits keep a read-only history of each update, so the June tagging and routing usually survive in the support desk. Instructure's documentation says a Canvas blueprint sync overwrites locked items in associated courses but leaves unlocked items to instructors, and later syncs are manual, which is why "pushed to linked courses" and "received by every group" are different facts. Moodle's documentation describes flagging existing assessments "Needs review" after a significant grading form change; where a platform records that, the last row has an answer.

Why the missing context matters to an AI developer

A rubric on its own is a list of criteria. A rubric with the appeal that exposed its weakness, the panel's reasoning and the revised wording is a worked record of how people decide edge cases in assessment. That is the material the rubric-based evaluation guide describes as useful for evaluating AI systems, and it is the kind of decision history that what data is missing from AI training identifies as scarce in public sources.

Strip the context away and the decision disappears, leaving only its result. That is why a company that has kept all three systems for overlapping years may hold something more useful than one with a clean LMS archive and nothing else. It is also why curriculum ownership matters: the training materials ownership guide explains when course and rubric history belong to the company.

Asking how much context survived, without seeing any of it

Walk the client through the "where it lives" column and ask, for each row, whether that system still holds the answer and since which year. Write down the answers, not the records. Useful follow-ups: is rubric history kept in the LMS, are rubrics locked in blueprints, and is a re-review of earlier grades ever logged? The completeness checklist turns those answers into a pass, partial or fail score.

The people who answer are the learner services lead, the assessment manager and the program design lead. The person who can authorize a licensing discussion is the CEO, usually the owner in an owner-run provider, with a COO or chief academic officer as a likely sponsor. The company needs 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license, and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

Unknown until SourceX reviews it under the company's agreement: how learner identities would be removed, whether student-record rules apply, what a buyer would want and what the company would be paid. Partners never handle learner work, tickets or grades.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the client still holds the reason, the cohort and the reach of its rubric changes in systems it owns, register as a partner, check who qualifies, and read the terms before you make the introduction.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

If the LMS keeps rubric history, is the context still missing?

Partly. Rubric history shows the old and new wording and the edit date. It still does not show that an appeal caused the edit, which cohort raised it, or whether earlier grades were reviewed. Those facts sit in the support desk and the assessment platform.

Is a support desk export any better on its own?

It fails in the opposite direction. It holds the learner's complaint and its routing, but not the panel's reasoning or what changed in the course. Each system answers a different part of the question.

Do I need to see the exports to judge what is missing?

No. Ask the client which of the questions in the table each system can answer, and since when. Their answers describe the gap without anyone opening a record.

Sources

  1. community.instructure.com/en/kb/articles/628548-blueprint-sync-functio…
  2. developer.zendesk.com/api-reference/ticketing/tickets/ticket_audits
  3. docs.moodle.org/36/en/Advanced_grading_methods
  4. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment