Incident-to-fix completeness scorecard: ten metadata checks for a broken evidence trail

Short answer

An incident-to-fix evidence trail breaks in five places: coverage (some incidents or years are missing), linkage (the issue key never reached the fix), timestamps (the status update predates the last merge), outcomes (nobody recorded what finally worked) and gaps nobody has named. The scorecard below turns each of those into checks a platform lead can answer from memory or configuration, with a pass, partial or fail guide, so a partner can describe the trail without opening a record.

Visual briefFull-size diagram
A five-group checklist for coverage, linkage, timestamps, outcomes and gaps ticks item by item while a vertical meter rises to 7 of 10 checks passed.
Illustrative score for a fictional company: 7 pass, 2 partial, 1 fail.

This scorecard belongs to the incident-to-fix guide. The record chain page explains every key in the sequence; this page grades whether those keys were actually used.

How to run the scorecard

Run it as one 30-minute conversation with the person who owns the on-call process, usually a platform engineering manager or engineering operations lead. Ask each question as written. Record pass, partial, fail or "unknown," plus the name of whoever answered.

Three rules keep it at metadata level:

  1. Every check is a question about structure. "Do emergency fixes carry the ticket key?" is a structure question. "Can I see one?" is not, and the partner never asks it.
  2. Estimates come from the company. If a share or count is needed, the company states it as a rough range. The partner does not count anything.
  3. Unknown is a valid answer. Write down who could find out, and move on.

Coverage checks

CheckQuestion to askPassPartialFail
C1. Incident thread coverageWhere were incidents handled, and can those channels be exported on the current Slack plan?Public channels, or an approved export covers private onesSome years in public channels, later years private and not yet approvedIncidents ran in private channels or DMs with no export route
C2. Issue coverageDid every incident above a set severity get its own issue?Yes, by written policyYes for severity 1, informal belowNo consistent issue for incidents

Slack's export rules drive C1. On Free and Pro plans, owners and admins can export public channels, while private channels and DMs are exportable only in limited circumstances after an application. Business+ and Enterprise owners can apply for a broader self-serve export. The plan and the channel type matter more here than the age of the workspace.

Linkage checks

CheckQuestion to askPassPartialFail
L1. Key in the threadIs the issue key posted in the incident channel as a habit?Standard step in the runbookUsually, not alwaysRarely or never
L2. Key on hotfix branchesRoughly what share of hotfix PRs carry the issue key?Most, by the team's estimateAbout halfFew, or nobody knows
L3. Non-default branch fixesDo incident fixes merge to release or hotfix branches?No, fixes go to the default branch firstSometimes, then reach the default branch laterOften, and stay on release branches

L2 is the single most informative question. Atlassian's documentation says the work item key in a branch name, commit message or PR title links the work to the Jira item in connected tools such as GitHub, so an emergency fix without the key is invisible from the ticket. L3 matters because GitHub's auto-close syntax links a PR to a GitHub issue only when it targets the default branch; a fix merged elsewhere closes nothing through it.

Timestamp and outcome checks

CheckQuestion to askPassPartialFail
T1. Status update versus last mergeIs the resolved message ever posted before the final linked merge?Rarely, and the issue is reopened when it isSometimes, with no reopen stepOften, and nobody tracks it
T2. Timestamps surviveDo both the message time and the merge time survive in the systems of record?Yes, in bothOne survives, the other is uncertainUnknown for both
O1. Review recorded on fixesDo branch protection rules require approvals on release and hotfix branches?Yes, same rules as the default branchDefault branch onlyNo review rules
O2. Final outcome recordedDoes the issue or postmortem say which change fixed the incident?Postmortem cites the key and the final changePostmortem exists, does not cite the keyNo postmortem or resolution note

On O1, GitHub's protected-branch settings can require a set number of approving reviews before a merge and can dismiss approvals made stale by new commits. Whether those rules covered release branches is a configuration question for the platform owner. If they did not, a reviewed patch on main and an unreviewed hotfix can sit inside the same incident. The root cause analysis records page covers postmortems as a record type of their own.

Gap check and an Illustrative scored result

The tenth check asks whether anyone has already named the gaps.

CheckQuestion to askPassPartialFail
G1. Known gaps namedCan the team name the periods, teams or branches where the chain is known to break?Yes, specific and written downRoughly, from memoryNo one has looked

Illustrative: a fictional 340-person B2B software company scored as follows.

GroupChecksResult
CoverageC1, C2C1 partial (private channels since 2022, export application not made); C2 pass
LinkageL1, L2, L3L1 pass; L2 partial (about half of hotfixes carry the key); L3 pass
TimestampsT1, T2Both pass
OutcomesO1, O2O1 fail (release branches unprotected); O2 pass
GapsG1Pass: hotfixes from one mobile team named as the main break
Total107 pass, 2 partial, 1 fail

That is a sound description to hand over: strong linkage on the main path, a named weak spot on hotfixes, and no review evidence on release branches. It says nothing about contents, and it overstates nothing. The missing-context page shows what a failed L2 looks like in a real sequence of events.

Who signs off before an introduction

The platform lead answers the scorecard; the CTO or another executive with authority over company data decides whether to discuss a license; counsel checks customer contracts. The company needs 50+ full-time employees at peak (contractors excluded), a documented operating history, rights to license the records and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

What stays unknown until SourceX reviews it: how much of the trail survives export, whether customer data limits use, and whether there is demand. Partners sign the Channel Partner Agreement before submitting an introduction. For the baseline screen, use the incident and on-call introduction checklist; ticket-first companies can also read the ServiceNow incident page.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the scorecard shows a mostly linked trail with its gaps named, register as a partner, check the company against who qualifies, and read the program terms before you send the introduction.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Does a company need to pass every check?

No. The scorecard is a description, not a gate. A history with a known, well-described gap is easier to discuss than one where nobody has asked the questions, and partial answers are normal.

Who should run the checks?

The company's own platform or engineering operations lead, in a short conversation with the partner. The partner records the answers and the person who gave them, never the records themselves.

What score is good enough to introduce the company?

There is no threshold. SourceX decides what to review after the company applies. The score helps the partner describe the history accurately and avoid overstating it.

Sources

  1. docs.github.com/en/issues/tracking-your-work-with-issues/usi…
  2. docs.github.com/repositories/configuring-branches-and-merges…
  3. slack.com/help/articles/201658943-Export-your-workspac…
  4. support.atlassian.com/jira-software-cloud/docs/reference-issues-in…
  5. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment