Feature flag rollback records across systems: a system map of the keys that join one incident

Short answer

To reconstruct a feature flag incident rollback, four keys have to survive across five systems: the incident ID (alert to incident to postmortem), the flag key (incident to flag change), the environment name (flag change to the exact rule edited) and a time window (flag change to the recovery check). The incident side is usually well joined. The weak link is between the incident and the flag platform, where the join often rests on timestamps alone. Ask the engineering director which keys exist, then introduce a decision-maker.

Visual briefFull-size diagram
A map of five systems, alerting, incident platform, team chat, flag platform and postmortem wiki, with the incident ID, flag key, environment and time window traveling between them to join one fictional rollback case.
Illustrative: the linking keys that join one fictional rollback case across five systems.

This system map is part of the feature flag incident rollback guide, which explains which records show how a rollback exception was handled. This page does a narrower job: it shows where each record lives, which identifier carries the case from one system to the next, and what a fully joined case looks like.

The system map: where each rollback record lives

A rollback leaves records in at least five places, each owned by a different team. The table is the map; every row needs a linking key to the row before it.

SystemRecord it holdsLinking keyUsual owner
Alerting or monitoringAlert, metric threshold, alert timeAlert ID, copied into the incidentPlatform team
Incident platformIncident log: statuses, actions, notificationsIncident IDIncident management or on-call lead
Team chatIncident channel messages, decisions, approvalsIncident ID in the channel name, or time windowIT or collaboration admin
Flag platformChange history: flag key, environment, rule, member, diffFlag key plus environment plus timestampRelease engineering
Postmortem wiki or document storeCause, decision, follow-up tasksIncident ID or link in the documentEngineering director

Why is the flag platform row usually rich but poorly linked? Take LaunchDarkly as one example: its change log stores, for every edit in every environment, who made it, when, and a before-and-after diff, and its REST API can list those audit log entries filtered by time window, resource and member. Nothing in that record asks for an incident reference. The flag change knows exactly what happened, but not why.

The incident platform is the mirror image. PagerDuty, for instance, keeps a timestamped log of every status change, action and notification on an incident, which can be listed through its API, so the why is captured while the exact flag change often is not. Partners whose clients run PagerDuty can read more on PagerDuty incident exports, and ServiceNow shops can compare ServiceNow incident data.

One fully joined case (Illustrative)

Illustrative. A fictional 410-person logistics software company ships a new route optimizer behind a flag set to 50% of accounts in production. All names, IDs and times are invented.

StepSystemRecordKey that carries the case forward
1MonitoringAlert at 14:02: route calculation latency over thresholdAlert ID copied into incident INC-2291
2Incident platformINC-2291 opened 14:03, acknowledged 14:06Incident ID
3Team chatChannel inc-2291: "latency only on accounts with route-optimizer-v3 on"Incident ID in channel name, flag key in a message
4Flag platform14:19: route-optimizer-v3 in production, rollout 50% to 10%, by the on-call engineerFlag key, environment, timestamp inside the incident window
5Incident platform14:34 note: "p95 latency back under threshold for the 10% group"Time window and metric name
6Postmortem wikiReview links INC-2291, records the partial rollback and a fix before going back to 50%Incident ID in the document

The case is fully joined because every step carries a key the next step can match. Remove the flag key from step 3 and the only link to step 4 is the timestamp, which still works if both systems record precise times and use the same environment names. Remove the note at step 5 and the recovery is unproven, even though it happened.

Count what the joined case holds: a symptom, a scoped decision (10%, not zero), the engineer who acted, a measured result and a written explanation that links back by ID. Every one of those depends on a key surviving the jump between two tools owned by two different teams.

It helps to grade each join rather than call the case simply linked or broken:

  • Hard join: the same identifier appears on both sides, such as INC-2291 in the channel name and in the review document. Machine-matchable, nothing inferred.
  • Soft join: the match rests on a time window plus a shared name, such as route-optimizer-v3 edited at 14:19 inside an incident open from 14:03 to 14:40. Usually right, but two overlapping incidents or two flags edited in the same minute make it ambiguous.
  • Orphan: a flag edit or an incident with no plausible partner. Orphans are not useless, but they cannot show a decision and its result together.

A history where most rollback cases have hard joins on the incident side and soft joins to the flag platform is common, and still reviewable. A history full of orphans is much harder to use.

Where each join breaks, and how to spot it

Each key fails in a predictable way. The engineering director can usually say which failures apply without opening a single record.

  • Incident ID not carried into chat. Channels named by date or by service instead of by incident. The fix in the map is a time window, which is weaker.
  • Flag key named in prose only. Engineers write "turned off the new optimizer" rather than the key. A reviewer has to infer which flag, which is error-prone when several flags were live.
  • Environment renamed or retired. The flag change says "prod-eu-2", the incident says "Europe". If the environment no longer exists, the rule detail may be hard to read.
  • Recovery check left on a dashboard. The on-call engineer watched a graph recover and never wrote it down. Monitoring data often expires; the incident record does not say what was seen.
  • Postmortem with no link back. The review sits in a wiki with a title but no incident ID, so it can only be matched by date and topic.

The same pattern of strong records and weak joins shows up in infrastructure-as-code histories, where commits are precise but the reason for a change often lives elsewhere.

A twenty-incident join count

Ask the engineering team to pick twenty incidents, at random, from at least a year ago that were resolved by a flag change. Ask for three numbers, not the incidents:

  1. Incidents where the flag change is linked by incident ID or flag key.
  2. Incidents matched only by time window and environment.
  3. Incidents where no matching flag change can be found.

Then ask which link failed most often, and whether the answer differs by year. SourceX publishes no benchmark for these counts. They are an honest starting picture for SourceX's review, and they tell the decision-maker how much of the history is reconstructable before anyone discusses licensing.

What the partner records at metadata level

Partners never ask for or handle raw incident records. The map above becomes a short set of facts:

  • Records that exist: flag change history, incident timelines, incident channel messages, written recovery checks and postmortems, with first and last year for each.
  • Who owns them: the company, subject to its customer agreements and its platform vendors' terms. Admin access to any one system is not authority to license it; the licensing authority decision tree covers who can say yes.
  • Who can authorize: the CEO (in an owner-run company, usually the owner), with the CTO's support.
  • Company criteria: 50+ full-time employees at peak (contractors excluded), documented operating history, rights to license and an authorized contact. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 90% of the data is in English.
  • Unknown until SourceX reviews it: buyer interest in this record type, how customer names in incident messages are treated, the rights review result and any offer.

The incident and on-call introduction checklist covers incident records generally. When the facts are ready, the rollback referral brief template turns them into one page.

Next step

Refer a company. Earn up to $100,000. Default reward: 25% of Actual Supplier Payout, capped at $100,000 cumulatively per referred company. Eligibility and payment conditions apply, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

If the twenty-incident count shows most rollbacks still join from alert to postmortem, register as a partner, confirm the company against who qualifies, and read the program terms before you introduce the decision-maker.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardYour 25% of the actual supplier payout. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

What if the company never wrote the incident ID on a flag change?

That is normal. The case can usually still be joined by the flag key, the environment and a time window around the incident. The join is weaker, so the engineering director should report how many cases match only by time.

Do we need the monitoring tool's raw metrics to complete the map?

No. The map needs to know whether a recovery check was written down somewhere, such as in the incident timeline or the postmortem. Raw metrics often expire long before the incident record does.

Can a partner build this map from an export?

No. The partner asks the engineering team to describe systems, keys and counts. SourceX looks at records only under its own agreement with the company.

Sources

  1. launchdarkly.com/docs/home/releases/change-history
  2. launchdarkly.com/docs/api/audit-log/list-audit-log-entries
  3. support.pagerduty.com/main/docs/incidents
  4. refer.sourcex.si/terms

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 90% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment