# Failed automation trail completeness scorecard | SourceX

Canonical: https://refer.sourcex.si/resources/failed-automation-to-policy-repair-completeness-checklist
Last updated: 2026-11-04

A failed-automation-to-policy-repair trail breaks at four kinds of gap: missing identifiers between runs and tickets, timestamps that cannot be put in order, article edits with no version note, and outcomes nobody recorded. One case breaks it most often: a workaround applied in the helpdesk again and again that never reaches the documented process. The eight checks below let an automation consultant score the trail pass, partial or fail at metadata level, so the introduction describes what really exists.

This scorecard is part of the [failed automation to policy repair guide](/guides/chain-failed-automation-to-policy-repair). For the keys behind checks 2 and 5, see [the record chain guide](/guides/failed-automation-to-policy-repair-record-linkage).

## The scorecard on one page

Score each check pass (2), partial (1) or fail (0), for a maximum of 16. Mark "unknown" as 0 and note who could answer. Record the year each practice began, because most trails improve over time.

| # | Check | Dimension | What it tells you |
|---|---|---|---|
| 1 | Run history depth | Coverage | How far back failures can be proved |
| 2 | Run-to-ticket key | Linkage | Whether a repair can be tied to its failure |
| 3 | Comparable timestamps | Timestamps | Whether failure, repair and rule change can be ordered |
| 4 | Repeat grouping | Linkage | Whether one cause can be seen across many tickets |
| 5 | Article version notes | Linkage | Whether a rule change can be tied to a repair |
| 6 | Draft-to-published path | Gaps | Whether repairs stall before becoming policy |
| 7 | Rule-tested evidence | Outcomes | Whether anyone can show the new instruction worked |
| 8 | Undocumented workarounds | Gaps | How many real rules exist only in habit |

## How to check each item without opening a record

Ask each question as written and mark the answer you hear.

**1. Run history depth.** "How long does your plan keep detailed run history, and have you exported it?" Pass: detailed history exported or retained for years. Partial: summaries kept, detail expired. Fail: neither, or unknown. Workato's documentation is a useful example of why this matters: retention depends on plan, and once it expires, action inputs and outputs are deleted while job summaries remain.

**2. Run-to-ticket key.** "What share of failure tickets were opened by an alert that writes the run ID?" Pass: most, since a known year. Partial: some, or the ID is pasted by hand. Fail: tickets carry no run reference.

**3. Comparable timestamps.** "Which time zone does each system export in, and do tickets record when work started?" Pass: known zones and start times. Partial: known zones, close times only. Fail: unknown.

**4. Repeat grouping.** "When the same failure repeats, how are tickets grouped?" Pass: tags, linked tickets or a problem record. Partial: grouped by an agent's memory. Fail: not grouped, or merged in a way that drops run IDs.

**5. Article version notes.** "Do article revisions usually carry a note or a ticket number?" Pass: usually. Partial: sometimes. Fail: edits made in place with no note. Zendesk help center revisions show the differences between versions, and light agents cannot see revisions, so ask the knowledge owner rather than a frontline agent.

**6. Draft-to-published path.** "How many article drafts created from tickets are waiting for review, and since when?" Pass: drafts reviewed on a schedule. Partial: a backlog exists but is known. Fail: nobody knows. In ServiceNow, closing an incident with the Knowledge check box selected creates a draft that does not appear in the knowledge base until reviewed and published.

**7. Rule-tested evidence.** "Do agents cite the article they followed when they close a ticket?" Pass: routinely. Partial: in some queues. Fail: never.

**8. Undocumented workarounds.** See the test below.

## The workaround test: recurring fixes with no matching rule

Check 8 is the one to run before anyone calls the archive complete. Ask the support operations manager to name, from memory, three recurring manual fixes their team applies, then ask whether each has a matching article, macro or SOP.

- **Pass:** all three have a written instruction that names the failure.
- **Partial:** one or two do, or the instruction exists only as a macro with no reference.
- **Fail:** none does. For those cases the chain ends at the helpdesk; the company performed the repair but never wrote down the rule.

A fail here does not make the company a poor introduction. It tells SourceX where the evidence stops, which is what an honest scoping conversation needs.

## A filled-in scorecard (Illustrative)

A fictional 310-person industrial parts distributor, scored in one call with its integration lead and support operations manager. All values are invented.

| # | Check | Answer heard | Mark | Points |
|---|---|---|---|---|
| 1 | Run history depth | Summaries only beyond the plan window; no exports | Partial | 1 |
| 2 | Run-to-ticket key | Alerts write the run ID since 2021 | Pass | 2 |
| 3 | Comparable timestamps | Platform exports UTC, helpdesk local time; both known | Pass | 2 |
| 4 | Repeat grouping | Agents remember; no tags | Partial | 1 |
| 5 | Article version notes | Sometimes cite a ticket | Partial | 1 |
| 6 | Draft-to-published path | Small known backlog | Partial | 1 |
| 7 | Rule-tested evidence | Order desk cites articles on close | Pass | 2 |
| 8 | Undocumented workarounds | Two of three fixes have no article | Fail | 0 |
| All | Total | Eight checks scored | Sum | 10 of 16 |

How a partner would summarize it: "Failure tickets linked to runs since 2021; article changes sometimes cite tickets; at least two recurring fixes were never documented." That sentence goes into the introduction, not the number. For portfolio-wide automation programs, see [AI agents in back-office automation](/guides/ai-agents-back-office-automation-portfolio-companies); for a sector where repair workflows are common, see [collision repair private equity](/industries/collision-repair-private-equity).

## Owners behind the checks and the person who signs

The integration or IT lead owns checks 1 to 3; the support operations manager owns 4, 7 and 8; the knowledge owner owns 5 and 6. The person who can authorize a licensing discussion is the CEO, usually advised by the COO, CFO, CIO or general counsel, since tickets and run payloads can hold customer and supplier details.

Confirm the company baseline: 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. The [company fit checker](/tools/company-fit-checker) covers the same basics.

What stays unknown until SourceX reviews it: whether the company's rights cover each system, what must be removed, which years a buyer wants and what the company would be paid. Partners never ask for run payloads, ticket text or article content. If an ERP or platform change is coming, the [ERP replacement handoff checklist](/resources/a-data-opportunity-handoff-checklist-for-an-erp-replacement) helps time the conversation before run history is lost. To see the cost of a gap in a single case, read the sibling [before-and-after guide](/guides/failed-automation-to-policy-repair-missing-context).

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

When the scorecard lets you describe your client's trail in one honest sentence, [register as a partner](/register), check the company on [who qualifies](/who-qualifies) and read the [program terms](/terms) before you introduce the CEO.

## Sources

- https://refer.sourcex.si/terms
- https://docs.workato.com/security/data-protection/data-retention/
- https://docs.servicenow.com/bundle/orlando-it-service-management/page/product/incident-management/task/create-knowledge-incident.html
- https://support.zendesk.com/hc/en-us/articles/4408829321498-Viewing-article-revisions-and-restoring-a-previous-version
