# Referral experiment retrospective: unexpected harm | SourceX

Canonical: https://refer.sourcex.si/guides/the-referral-experiment-retrospective-that-captures-unexpected-harm
Last updated: 2026-11-04

A referral experiment retrospective catches unexpected harm only if it looks past the measures in the plan. Review outcomes and side effects together: ask every group the test touched, including clients who were never asked and your own colleagues, what went wrong; log each report whether or not it fits a measure; then record who bore it, whether the plan foresaw it, how serious and reversible it was, and whether one version, both versions or the test itself caused it. Every harm gets one named change.

What follows is a review design, with no results behind it; the one worked review is Illustrative. The general case, a write-up others can check line by line, is covered in [how to publish a pilot retrospective investors can trust](/guides/how-to-publish-a-pilot-retrospective-investors-can-trust). This page takes the narrower question that outline leaves thin: how to find harm nobody set out to count, and what to do about it.

## Why the plan's own measures miss the harm that matters

A plan counts what the team expected to happen. Harm tends to land where no measure points:

- **Outside the unit.** The unit is usually the company asked; the people hurt may be a relationship owner whose goodwill was spent, a client never asked, or the colleague who sent the message.
- **After the reading date.** A CEO who agreed in week two may learn in week ten that it misunderstood what it agreed to.
- **Inside a success.** An application from a CEO who believed the company already qualified counts as a win on the planned measure and as a harm on this review.
- **In the team's own behavior.** Counting introductions puts pressure on the people making asks, and pressure finds shortcuts.

Clinical trial reporting names both routes. CONSORT's harms item asks authors to state "How harms were defined and assessed (eg, systematically, non-systematically)". Its explanation separates harms solicited through active or targeted surveillance from harms that arrive unsolicited or as spontaneous declarations, and warns that the unsolicited kind is especially exposed to under-reporting and selective non-reporting (our paraphrase). A referral test produces both kinds, and a retrospective that tabulates only what the plan asked about is a results report, not a harm review.

"Unexpected" needs a definition too. ICH's E2A guideline, written for drug safety, treats a reaction as unexpected when its nature or severity does not match the reference information for the product (our paraphrase). For a referral test the reference is its dated plan: a harm is unexpected when the plan did not name it, or named it at a milder severity.

## Everyone the test touched

List the groups before collecting anything, so the review cannot quietly shrink to the cohort.

| Group | How the test can harm them | How the review hears from them | What the review never asks |
|---|---|---|---|
| CEOs asked, in either version | A misread fit, a sense of pressure, a worry about their payout | One open question at the close, unless they asked for no further contact | Anything about records, systems or contracts |
| Clients screened but not asked | Hearing of the test second-hand and feeling left out | Only when they raise it; log what they said | Nothing; nobody approaches them |
| Relationship owners | Spent goodwill, a strained client relationship | A short interview with each one | A rating of the CEO |
| Colleagues who sent messages | Pressure from the count, questions they could not answer, departures from the script | A private written debrief | Who to blame, in a group setting |
| SourceX | Questions or applications your wording caused | A note about your own introductions only | Anything about other partners or applicants |

## The outcome-and-side-effect review template

Keep two ledgers side by side: the outcome ledger repeats the plan's measures per version, with denominators, and the side-effect log takes one row per report, whatever its source.

| Field | What to record | Allowed values |
|---|---|---|
| Report | Number and date received | 1, 2, 3 and onward |
| Channel | How it reached you | Asked in the review; volunteered; found in the dated record |
| Who bore it | The person or group harmed | CEO asked; client not asked; relationship owner; sender; SourceX; other |
| What happened | One factual sentence, in the reporter's words if they agree | Free text; no company identity in any shared copy |
| Foreseen | Compared with the dated plan | Named; named but milder; not named |
| Severity | The fixed scale below | 1, 2 or 3 |
| Reversible | Whether it can be put right | Yes, by a named step; partly; no |
| Linked to | Where it came from | One version; both versions; running the test; unclear |
| Change | One action, with an owner and a date | Free text |

Fix the severity scale before the review opens, so nobody grades a harm to fit a conclusion:

- **1, friction.** Annoyance or a question that took effort, settled the same week.
- **2, acted on or strained.** A misunderstanding someone acted on, or a relationship that needed repair, put right with effort.
- **3, lasting.** Trust that did not come back, a decision made on a false belief that cannot be undone, or information that went somewhere it should not have.

The review is itself a small design, and it reads like one:

| Element | Setting |
|---|---|
| Hypothesis | Asking every group the test touched one open question finds harms the plan's measures missed |
| Unit | One report |
| Measure | Reports counted by group, by foreseen status and by severity |
| Stop rules | A reply that contains company records or confidential details is not kept or forwarded, and the sender is told; a request for no further contact ends that line of inquiry at once |
| Decision rule | Any severity 3 harm: the version or process behind it does not run again until changed and re-approved. Any harm not named in the plan: it becomes a named stop rule or measure in the next plan. A version is adopted on its outcomes only if no harm of severity 2 or 3 is linked to that version alone |

## Collecting what nobody planned to count

Run the review in two passes. The first opens on the reading date. The second, about 30 days later, goes to relationship owners and senders, and to CEOs only if they invited it, because some harms surface late: a company that learns in SourceX's review that it does not fit, or a client who mentions the note at a quarterly meeting.

Use open questions with no score attached:

- To a CEO: "Was there anything about how this was raised with you that you would have changed?"
- To a relationship owner: "Has anything in this client relationship shifted since the note went out?"
- To a sender: "Did you say anything the script did not, or get asked something you could not answer?"

Ask about events, not culprits: a sender who expects blame reports nothing. The blameless framing in [the software ecosystem's failure review](/guides/what-a-software-ecosystem-can-learn-from-a-failed-referral-pilot) applies unchanged.

## Variant, test or neither: sorting a harm before deciding

The "linked to" field decides what kind of change a harm calls for.

- **One version only.** A candidate effect of that version's wording. With eight companies a side, one report is a reason to change the wording, not proof the version caused it.
- **Both versions.** It belongs to the program explanation or the process. Switching versions will not fix it.
- **Running the test.** It exists because a test was running: the count, the random draw, two colleagues asking the same CEO.
- **Unclear.** It stays unclear. Assigning it anyway turns a guess into a finding.

Every harm count in a small test is descriptive: it says what happened to whom, not how often it would happen again.

## One retrospective, worked through (Illustrative)

Illustrative. An invented consultancy, invented clients and invented counts; no SourceX outcome is shown or implied.

A 40-person operations consultancy tested two client notes with 16 client CEOs, eight per version, drawn before sending. Version A opened with how the program works. Version B opened with the published fit questions for the CEO to answer privately. The planned measure was CEO-agreed introductions; the planned harm measures were complaints and requests for no further contact.

On the outcome ledger, 3 of 8 CEOs agreed under A and 4 of 8 under B. Complaints: none under either. Requests for no further contact: one under A, none under B. The test looked clean and leaned slightly toward B. The side-effect log disagreed.

| Report | Who bore it | What happened | Foreseen | Severity | Linked to | Change |
|---|---|---|---|---|---|---|
| 1 | CEO asked, version B | Answered the fit questions, concluded the company qualified, told colleagues so, and the application was later declined in SourceX's review | Not named | 2 | One version | Put "SourceX decides after its review" beside the questions, and add a closing question on what the CEO now believes |
| 2 | CEO asked, version A, and the relationship owner | Asked the account lead whether the consultancy's reward would come out of the company's payment | Not named | 1 | Both versions | Add the terms sentence on separate payment to both versions |
| 3 | Client not asked | Heard about the note from a peer and asked why it had been left out | Not named | 1 | Running the test | A plain answer on how clients were chosen, built from the published fit facts |
| 4 | Client asked, version A | A consultant offered to pull the client's ticket export to save time; the CEO declined, nothing was sent, and it was logged a week late | Named, but not the reporting lag | 2 | Running the test | Same-day reporting of any offer involving records, and a rebrief for everyone sending |
| 5 | CEO asked in both versions | Two colleagues contacted the same CEO, one with each version | Named only as a validity risk | 1 | Running the test | One speaker per client, names checked against the list before each ask |

B's one-introduction edge is well within what chance produces with eight a side, and report 1 ties a severity 2 harm to B alone, so under the decision rule B is not adopted as written; the next test keeps B's idea with the added line. Three of the five harms came from running the test, not from either note, so changes 3 to 5 apply whichever version runs next.

Report 2 has a published answer. Section 4 of the terms reads: "The referral reward is paid separately by SourceX and is never deducted from the supplier's agreed compensation." SourceX has also confirmed that a company receives the same amount whether or not a partner introduced it. The reward comes from SourceX's own share. For testing wording that states every condition, see [eligibility messaging that hides no restrictions](/guides/how-to-test-eligibility-messaging-without-hiding-restrictions).

## The program facts the review checks wording against

Report 1 shows why every test message must state fit facts correctly. Clients are screened from conversations rather than files: whether full-time staff ever numbered 50 or more at the company's high point (contractors do not count), whether it holds several years of operating records it produced on its own, whether it may license that material, and whether its CEO is open to a discussion. Companies in the U.S., Canada and the EU qualify, as do companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when at least 70% of the data is in English.

SourceX applies each program's own criteria and checks relevance in its review, so whether a client's records match a current program, whether contracts restrict them and what a license would cover stay open until then. Material that belongs to the client's customers is usually not a fit. Nobody at the firm collects records or system details for SourceX, and the CEO decides for the company. Partners sign the Channel Partner Agreement before submitting introductions; only one partner can be credited per company, decided from dated records, with overlapping cases settled in SourceX's duplicate review.

## Where a side-effect review misleads

- **Harm travels upward reluctantly.** People seldom report harm to whoever caused it, so route reports to the reviewer.
- **Asking is itself a contact.** One question, once, and never to anyone who asked for no further contact.
- **Counts of one are not differences.** Severity and reversibility carry more weight than tallies.
- **"Linked to" is a judgment.** Two reviewers coding separately, then reconciling, reduce the guesswork.
- **It judges your process only.** It says nothing about whether any company fits; SourceX decides that. For a firm-wide look at six months of referrals, use [the midpoint program review](/guides/the-partner-program-retrospective-after-six-months); for client-trust tripwires set before a pilot starts, see [the MSP pilot plan](/guides/the-msp-referral-pilot-that-protects-client-trust).

## Next step

> **Refer a company. No cap on rewards.** Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

Before your next test, put the side-effect log and severity scale in its dated plan, [register as a partner](/register) under the entity that will sign, check each client against [who qualifies](/who-qualifies) and the current [program terms](/terms), and turn last time's harms into stop rules with [the early-stop rules](/guides/when-to-stop-a-referral-experiment-early). More designs are in the [partner guides](/guides).

## Sources

- https://refer.sourcex.si/terms
- https://refer.sourcex.si/who-qualifies
- https://www.consort-spirit.org/item15-harms
- https://database.ich.org/sites/default/files/E2A_Guideline.pdf
