When a passing test misses a customer dependency: a worked network change acceptance exception

Short answer

The records that explain an exception are the ones that show the gap between "the test passed" and "the customer was fine", and how the operator closed it: the change request with its impact analysis, the test results the approver relied on, the engineer's maintenance notes, the incident ticket, the configuration backups compared afterwards, the emergency change and the review that updated the test template. An approval alone proves only that someone signed.

Visual briefFull-size diagram
A horizontal case timeline: change approved, tests pass at 02:20, accepted at 02:40, then a branch drops at 07:15 when a customer's replication fails, followed by the config diff at 07:50, the emergency fix at 08:05 and the template update in the review, each with its record.
Illustrative case. The branch after acceptance is where the reasoning lives.

This worked case is part of the network change acceptance guide. The system map shows which tools hold each record and how they join; this page follows one hard case in time order, so a network automation integrator can see what useful exception history looks like and spot it at a client.

The setup: a router swap with 14 customer circuits (Illustrative)

The case is fictional. A 210-person regional fiber and managed network provider is replacing an aggregation router during a hardware refresh. It is not a real incident, a quote or a completed deal.

The provider runs a conventional change process. The change request lists the circuits on the router, the impact analysis estimates customer exposure, and a test plan is generated from the network inventory: for each listed circuit, confirm the interface is up, optical levels are in range, routing sessions are established and the customer's edge device answers. The new router is configured from the migration template. On paper, every customer dependency is covered.

The exception this case teaches is a passing test that misses a customer dependency. The inventory listed 14 circuits. It did not list a static route one customer depended on for replication to its backup site, because that route had been added by hand years earlier and never documented.

Hour by hour: who decided what, and which record kept it

TimeDecision or eventWhoRecord and system
Two weeks beforeChange approved; impact analysis lists 14 customer circuitsChange approval boardChange request, change system
01:00Window opens; config loaded from the migration templateNetwork engineerMaintenance notes, change system
02:20Interfaces up, optics in range, routing sessions established, edge devices reachableNetwork engineerTest results, test system
02:40Change accepted against those four resultsShift leadAcceptance approval, change system
07:15Customer reports that replication to its backup site is failingCustomer, service desk agentIncident ticket, service desk
07:50Old and new config backups compared; a static route for the backup subnet existed only on the old routerNetwork engineerConfig backups, maintenance notes
08:05Route restored under an emergency change; replication confirmed with the customerEngineer, on-call managerEmergency change, test results
Following weekReview adds a config-diff check for customer static routes to the test template; inventory updatedNetwork operations headPost-implementation review

Each row names a person and a record. That pairing is what turns an event log into evidence: anyone reviewing the case later can see not only what happened but who made each call and what they were looking at when they made it.

Why this sequence teaches more than a clean change

Read the rows together and the reasoning becomes visible. The test plan was built from the inventory. The inventory was incomplete. So a correct test of the wrong scope passed, and the shift lead accepted the change in good faith on four true results.

A clean change that passes and stays passed shows the normal pattern. This one shows something rarer: a judgment that was reasonable at 02:40 and wrong by 07:15, a diagnosis that found the cause in a record nobody had consulted, and a process correction that prevents the same miss next time. NIST SP 800-171 Rev. 3 asks organizations to verify that security requirements still hold after a change is implemented, and NIST SP 800-128 describes post-implementation review as the step that catches unintended effects, including after emergency changes. The 07:50 row and the following week's review are exactly that verification at work.

For AI training and evaluation, sequences like this are valuable because the outcome is known. A model can be shown the 02:40 state and asked whether to accept, then scored against what actually happened. The exception handling and AI agents guide explains why documented exceptions with verified outcomes matter more than volumes of routine records. Whether a buyer wants this particular record type is decided in SourceX's review, not assumed.

Below the reporting line: why only internal records exist

The scale of the miss decides where it was recorded. Under 47 CFR Part 4, covered providers file outage reports for events lasting at least 30 minutes that potentially affect at least 900,000 user-minutes, among other criteria. One customer's broken replication route is nowhere near that line. No regulator ever heard about it; the provider's ticket, notes and review are the only trace.

That is typical. Most exceptions in change acceptance are small, customer-specific and invisible outside the operator. Even where a change does cause a reportable outage, Part 4 filings are presumed confidential, so the internal trail remains the record worth describing. Contractor involvement adds a wrinkle: if a refresh contractor or MSP wrote the maintenance notes, the services agreement may affect who owns them, a question the MSP ticket data page covers from the provider side.

Spotting similar cases at a client

A network automation integrator rarely needs to see a single record to know whether a client has cases like this. Listen for these signals in ordinary project conversations:

  1. "The test passed but..." Any story about a change that passed acceptance and still caused a customer call.
  2. Emergency changes linked to planned ones. Ask whether emergency changes reference the planned change that triggered them.
  3. Config backups kept by version. Pre- and post-change backups retained per device, not only the latest copy.
  4. Test templates with a change history. If templates were updated after incidents, the reasons are often written down.
  5. Post-implementation reviews that name causes. A review that says "inventory missing static route" is worth far more than one that says "resolved".

Then ask the network operations head four metadata questions: which systems hold changes, tests, notes and backups; since which year; roughly how many changes a year have a linked incident; and who would approve a data licensing discussion. The decision tree on who can license network change acceptance records maps that last answer.

The introduced company needs 50+ full-time employees at peak (contractors excluded), a documented operating history, rights to license the records and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 90% of the data is in English. Partners sign the Channel Partner Agreement before submitting introductions and never collect change records, configs or tickets. Customer circuit details and topology are sensitive; what can be licensed, after de-identification, stays unknown until SourceX's rights review. More on the sector is on the telecommunications page, and building a referral network as a consultant covers turning project conversations into introductions.

Next step

Refer a company. Earn up to $100,000. Default reward: 25% of Actual Supplier Payout, capped at $100,000 cumulatively per referred company. Eligibility and payment conditions apply, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If a client's change history includes accepted changes that later needed an emergency fix and a template update, register as a partner, check the operator on who qualifies and read the program terms before introducing the network operations head.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardYour 25% of the actual supplier payout. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Is a change that caused a customer problem embarrassing for the operator to discuss?

It can feel that way, but a well-documented miss followed by a fix and a better test template shows a process that learns. Whether to discuss it at all is the operator's decision, and customer details are removed or excluded in any review.

What if the operator never wrote a post-implementation review?

The case is weaker but not empty. The incident ticket, the config backups and the emergency change still show the miss and the fix; what is lost is the stated reason the test template changed.

Sources

  1. nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-128…
  2. nvlpubs.nist.gov/nistpubs/SpecialPublications/800-171r3/NIST.…
  3. ecfr.gov/current/title-47/chapter-I/subchapter-A/part-4
  4. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 90% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment