# Tool-call provenance evidence gap analysis | SourceX

Canonical: https://refer.sourcex.si/questions/tool-call-provenance-evidence-gaps
Last updated: 2026-11-04

Four gaps cause most of the trouble: the actor is not recorded, the rule version is lost, the input is overwritten, or the outcome is never linked. Two hidden gaps follow close behind: shared credentials that make the actor field meaningless, and retention limits that cut history short. Each one removes a different piece of evidence a reviewer expects. The honest introduction names which pieces the company says it has and which it does not know.

This gap analysis is part of the guide to [tool-call provenance in a business workflow](/guides/concept-tool-call-provenance), which walks through one help desk status change from billing input to customer outcome. Here the question is the reverse: when a client's records fall short of that example, where do they usually fall short, and how much does it matter?

## The evidence a review expects for each automated change

A reviewer judging an automated action wants six things recorded together. The list maps closely onto the audit-content control in NIST SP 800-53 (AU-3), which asks audit records to establish the type of event, when and where it occurred, its source, its outcome, and the identity of the individuals or subjects involved. Provenance for AI use adds one item security auditing does not stress: the version of the logic that ran.

| # | Expected evidence | What it answers |
|---|---|---|
| 1 | The change itself, with old and new value | What happened to the record |
| 2 | Timestamp | When, and in what order relative to other events |
| 3 | Named actor: which rule, integration or person | Who or what was responsible |
| 4 | Rule or integration version | Which logic ran at that moment |
| 5 | Input value as it stood at the time, and who wrote it | What the action relied on |
| 6 | Linked outcome: reopen, refund, dispute, next invoice | Whether the action turned out right |

## Expected versus typical: the gap matrix

Companies vary, but the shape repeats. Items 1 and 2 are nearly always present, because every audit trail records a change and a time. The others thin out from there.

| Expected evidence | What companies typically have | The usual gap | Metadata question to ask |
|---|---|---|---|
| Change with old and new value | Present in most help desk and ERP audit trails | Some systems store only the new value | "Does the history keep the before and after of each field?" |
| Timestamp | Present almost everywhere | Clocks differ across systems, so cross-system order is fuzzy | "Do your systems log in one time zone?" |
| Named actor | Often present for people; mixed for automation | Every automated change shows as "system" or one service account | "Can you tell a rule's change from a person's in the history?" |
| Rule version | Least often present | Only today's rule definition exists; past edits are gone | "Are changes to workflow rules versioned or logged, and since when?" |
| Input at the time | Present if the field's own history is kept | The field shows today's value, not the value the rule read | "If an input field changed later, can you still see its earlier value?" |
| Linked outcome | Present within one system, rare across systems | Refunds and disputes sit in billing or a payment portal with no shared key | "Can a reopen or billing dispute be tied back to the ticket number?" |

Platform detail matters at the version row. Zendesk's documentation, for example, says a trigger-made change records the trigger's ID and title, and adds a revision ID on Enterprise plans. Two companies on the same help desk can therefore sit on opposite sides of the version gap depending on their plan, which is why the question is "what does your history show?" rather than "which product do you use?"

## Why each gap matters for a review

**Actor missing.** Without a named actor, an approved workflow and a manual override look identical. That distinction is the whole point of tool-call provenance.

**Version missing.** A trigger edited twice in a quarter ran three sets of logic. If only the latest survives, any change older than the last edit is attributed to logic that did not exist yet.

**Input overwritten.** The rule read a credit note ID at 14:02; finance corrected the field at 16:40. The ticket now shows the corrected value, and the action looks like it relied on information it never saw.

**Outcome unlinked.** The change is fully traced but nobody can say whether it was right. For training, a traced action with no result teaches sequence without judgment; for evaluation, it cannot be scored.

A company with one gap can still be worth introducing, because the gap narrows what can be licensed rather than erasing it. A company with all four has status histories, not provenance.

## Retention and shared credentials: the two hidden gaps

These rarely come up unprompted, and both make a record look more complete than it is.

**Retention.** Audit trails are kept for a period that varies by platform, plan and company policy, and older records may be archived rather than deleted. Zendesk, for instance, usually archives tickets 120 days after they close; archived tickets drop out of views but stay reachable through search and the API. Ask how many years of history exist and whether older years are archived or purged, rather than assuming all of it survives.

**Shared credentials.** If three integrations and two staff members write through the same login, the actor field is technically filled in and practically useless. NIST's enhancement AU-3(1) names this directly as additional audit content: the individual identities of group account users. Ask whether each integration has its own API user and since when.

## Describing the gap honestly in an introduction

Write down what the company told you, attributed to the company, and mark what nobody knows yet. An illustrative note:

> **Records:** help desk audit history, about six years; billing system with credit notes.
> **Present, per the support lead:** field changes with before and after; trigger names on automated changes.
> **Unknown:** whether trigger revisions were captured before the 2023 plan upgrade.
> **Gap, per the company:** two integrations share one API user until 2024; refund disputes live in the payment portal with no ticket key.
> **Who decides:** COO, with the support operations lead as the systems contact.

This tells SourceX where to look without overstating anything. Do not estimate volumes you have not been given, and do not describe a gap as fixable; whether and how history can be reconstructed is part of the review. Before writing the note, the [tool-call provenance scorecard](/questions/does-tool-call-provenance-data-qualify) helps decide whether an introduction makes sense at all, and the [review packet](/resources/tool-call-provenance-review-packet) shows what the company would assemble afterwards.

The company baseline still applies: 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English. For agent-connected systems, see [MCP and data provenance](/resources/mcp/mcp-data-provenance); for dataset-level origin and rights, see [what data provenance is](/proof/what-is-data-provenance); for the wider family of records, see [tool-use training data](/data-types/tool-use-training-data). Governance expectations for training data in the EU are covered in [AI Act Article 10](/guides/ai-act-article-10-data-governance).

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the company's history names the actor on automated changes and at least one other gap is known rather than guessed, [register as a partner](/register), check [who qualifies](/who-qualifies), and read the [terms](/terms) before you send the introduction note.

## Sources

- https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf
- https://developer.zendesk.com/documentation/ticketing/reference-guides/via-object-reference/
- https://developer.zendesk.com/documentation/ticketing/reference-guides/ticket-audit-events-reference/
- https://support.zendesk.com/hc/en-us/articles/4408887617050-About-ticket-archiving
- https://refer.sourcex.si/terms
