Failed automation to policy repair completeness scorecard: eight checks from failed run to revised instruction

Short answer

A failed-automation-to-policy-repair trail breaks at four kinds of gap: missing identifiers between runs and tickets, timestamps that cannot be put in order, article edits with no version note, and outcomes nobody recorded. One case breaks it most often: a workaround applied in the helpdesk again and again that never reaches the documented process. The eight checks below let an automation consultant score the trail pass, partial or fail at metadata level, so the introduction describes what really exists.

Visual briefFull-size diagram
Eight scorecard rows for a failed-automation trail tick pass, partial or fail while a horizontal meter fills to 10 of 16, with the undocumented-workaround row marked fail.
Illustrative scoring for a fictional distributor. The total describes the trail; it does not forecast acceptance or value.

This scorecard is part of the failed automation to policy repair guide. For the keys behind checks 2 and 5, see the record chain guide.

The scorecard on one page

Score each check pass (2), partial (1) or fail (0), for a maximum of 16. Mark "unknown" as 0 and note who could answer. Record the year each practice began, because most trails improve over time.

#CheckDimensionWhat it tells you
1Run history depthCoverageHow far back failures can be proved
2Run-to-ticket keyLinkageWhether a repair can be tied to its failure
3Comparable timestampsTimestampsWhether failure, repair and rule change can be ordered
4Repeat groupingLinkageWhether one cause can be seen across many tickets
5Article version notesLinkageWhether a rule change can be tied to a repair
6Draft-to-published pathGapsWhether repairs stall before becoming policy
7Rule-tested evidenceOutcomesWhether anyone can show the new instruction worked
8Undocumented workaroundsGapsHow many real rules exist only in habit

How to check each item without opening a record

Ask each question as written and mark the answer you hear.

1. Run history depth. "How long does your plan keep detailed run history, and have you exported it?" Pass: detailed history exported or retained for years. Partial: summaries kept, detail expired. Fail: neither, or unknown. Workato's documentation is a useful example of why this matters: retention depends on plan, and once it expires, action inputs and outputs are deleted while job summaries remain.

2. Run-to-ticket key. "What share of failure tickets were opened by an alert that writes the run ID?" Pass: most, since a known year. Partial: some, or the ID is pasted by hand. Fail: tickets carry no run reference.

3. Comparable timestamps. "Which time zone does each system export in, and do tickets record when work started?" Pass: known zones and start times. Partial: known zones, close times only. Fail: unknown.

4. Repeat grouping. "When the same failure repeats, how are tickets grouped?" Pass: tags, linked tickets or a problem record. Partial: grouped by an agent's memory. Fail: not grouped, or merged in a way that drops run IDs.

5. Article version notes. "Do article revisions usually carry a note or a ticket number?" Pass: usually. Partial: sometimes. Fail: edits made in place with no note. Zendesk help center revisions show the differences between versions, and light agents cannot see revisions, so ask the knowledge owner rather than a frontline agent.

6. Draft-to-published path. "How many article drafts created from tickets are waiting for review, and since when?" Pass: drafts reviewed on a schedule. Partial: a backlog exists but is known. Fail: nobody knows. In ServiceNow, closing an incident with the Knowledge check box selected creates a draft that does not appear in the knowledge base until reviewed and published.

7. Rule-tested evidence. "Do agents cite the article they followed when they close a ticket?" Pass: routinely. Partial: in some queues. Fail: never.

8. Undocumented workarounds. See the test below.

The workaround test: recurring fixes with no matching rule

Check 8 is the one to run before anyone calls the archive complete. Ask the support operations manager to name, from memory, three recurring manual fixes their team applies, then ask whether each has a matching article, macro or SOP.

  • Pass: all three have a written instruction that names the failure.
  • Partial: one or two do, or the instruction exists only as a macro with no reference.
  • Fail: none does. For those cases the chain ends at the helpdesk; the company performed the repair but never wrote down the rule.

A fail here does not make the company a poor introduction. It tells SourceX where the evidence stops, which is what an honest scoping conversation needs.

A filled-in scorecard (Illustrative)

A fictional 310-person industrial parts distributor, scored in one call with its integration lead and support operations manager. All values are invented.

#CheckAnswer heardMarkPoints
1Run history depthSummaries only beyond the plan window; no exportsPartial1
2Run-to-ticket keyAlerts write the run ID since 2021Pass2
3Comparable timestampsPlatform exports UTC, helpdesk local time; both knownPass2
4Repeat groupingAgents remember; no tagsPartial1
5Article version notesSometimes cite a ticketPartial1
6Draft-to-published pathSmall known backlogPartial1
7Rule-tested evidenceOrder desk cites articles on closePass2
8Undocumented workaroundsTwo of three fixes have no articleFail0
AllTotalEight checks scoredSum10 of 16

How a partner would summarize it: "Failure tickets linked to runs since 2021; article changes sometimes cite tickets; at least two recurring fixes were never documented." That sentence goes into the introduction, not the number. For portfolio-wide automation programs, see AI agents in back-office automation; for a sector where repair workflows are common, see collision repair private equity.

Owners behind the checks and the person who signs

The integration or IT lead owns checks 1 to 3; the support operations manager owns 4, 7 and 8; the knowledge owner owns 5 and 6. The person who can authorize a licensing discussion is the CEO, usually advised by the COO, CFO, CIO or general counsel, since tickets and run payloads can hold customer and supplier details.

Confirm the company baseline: 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. The company fit checker covers the same basics.

What stays unknown until SourceX reviews it: whether the company's rights cover each system, what must be removed, which years a buyer wants and what the company would be paid. Partners never ask for run payloads, ticket text or article content. If an ERP or platform change is coming, the ERP replacement handoff checklist helps time the conversation before run history is lost. To see the cost of a gap in a single case, read the sibling before-and-after guide.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

When the scorecard lets you describe your client's trail in one honest sentence, register as a partner, check the company on who qualifies and read the program terms before you introduce the CEO.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Which person should answer this scorecard?

The integration or IT lead for the run checks, the support operations manager for ticket checks and the knowledge owner for article checks. A 30-minute call with all three usually covers it; nobody needs to open a ticket.

A company fails the workaround check. Is it still worth introducing?

Often yes. Repairs that never became a written rule are common, and saying so plainly helps SourceX scope the review. The scorecard is for describing the trail honestly, not for screening companies out.

Does the scorecard depend on which automation platform the company uses?

No. The checks ask about practices, such as whether alerts create tickets and how long run detail is kept. Platform names go into your notes as plain labels, and SourceX does not claim a connector to any of them.

Sources

  1. refer.sourcex.si/terms
  2. docs.workato.com/security/data-protection/data-retention
  3. docs.servicenow.com/bundle/orlando-it-service-management/page/pr…
  4. support.zendesk.com/hc/en-us/articles/4408829321498-Viewing-arti…

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment