This page is for a partner comparing two versions of its own outreach, such as two opening angles. It contains arithmetic and a decision rule, not results; no partner or SourceX results are published. Every worked number below is labeled Illustrative.
Five replies are the wrong count, twice
First, replies are not outcomes. A reply can say yes, no, not now, or "what is this?". Counting them together rewards whichever message provokes more questions. The outcome a partner can actually verify is later and rarer: a verified application credited to the introduction. The terms put it there too: under section 2, the earliest valid introduction of a company SourceX did not already know, made inside the 90-day window, earns credit only "when that company submits a verified application", and "an earlier link click alone does not decide credit".
Second, five of anything is too few. Even if all five were verified applications, the split between two messages would still be mostly noise, as the next section shows with nothing more than counting.
The arithmetic of five when nothing is going on
Assume three things: each message went to the same number of CEOs, one CEO's response does not depend on another's, and the two messages are equally good. Then each of the five responses is as likely to come from message A as from message B, like a fair coin. Five coins can land in 2 × 2 × 2 × 2 × 2 = 32 equally likely sequences. Counting the sequences that give each split:
| Split, A to B | Sequences out of 32 | How often by chance alone |
|---|---|---|
| 5-0 | 1 | 3.1% |
| 4-1 | 5 | 15.6% |
| 3-2 | 10 | 31.3% |
| 2-3 | 10 | 31.3% |
| 1-4 | 5 | 15.6% |
| 0-5 | 1 | 3.1% |
Add the four outer rows: 1 + 5 + 5 + 1 = 12 of 32, or 37.5%. More than one comparison in three between identical messages will hand you a split of 4-1 or wider, ready to be announced as a winner. Even a clean sweep, 5-0 or 0-5, turns up 2 times in 32.
The counts come from the binomial distribution. NIST's statistics handbook gives the chance of x outcomes out of n as the number of ways to choose x from n, times p to the power x, times (1 - p) to the power n - x (paraphrase). With n = 5 and p = 0.5, every term becomes the number of sequences divided by 32, which is the table above. The handbook's glossary describes the trials as independent; that is the second assumption, and the one most likely to fail in a network where CEOs talk to each other.
Percentages make the same split look bigger. Illustrative: 4 of 30 CEOs answer message A and 1 of 30 answer message B. Written as 13% against 3%, it reads as "four times better". It is still the 4-1 row.
The small-sample interpretation worksheet
Fill in lines 1 to 5 before the first send and lines 6 to 9 at the cutoff. Keep the old sheet when you start a new test.
| Line | What you write | Filled in |
|---|---|---|
| 1. The comparison | Message A and message B, each text fixed and dated | Before |
| 2. Allocation | How CEOs were assigned, and sends per message, which must be equal for the table below to apply | Before |
| 3. The outcome | Verified applications credited to your introduction, not closed as duplicates, with stage words copied from your dashboard | Before |
| 4. When you look | The total number of outcomes you will wait for, and a calendar end date | Before |
| 5. The bar | The smallest winning split for that total, taken from the threshold table | Before |
| 6. Counts | Outcomes for message A and message B at the cutoff, plus sends for each | At cutoff |
| 7. Vetoes | Requests to stop, complaints, misreadings and any records sent to you, for each message | At cutoff |
| 8. Verdict | One of: winner named; no winner yet; this test cannot answer at this scale | At cutoff |
| 9. What would change it | The count, date or event that would reopen the verdict | At cutoff |
Line 3 needs the dashboard. SourceX's how it works page says it shows each referral's stage and its reward status. Line 7 needs only your inbox: you never ask for or handle a company's records, and the same page says a partner does not need to export, upload or describe confidential records.
How lopsided must the split be? A threshold table
The bar on this page is one in twenty: a split counts as a difference only if two equally good messages would produce a split at least that lopsided, in either direction, 5% of the time or less. One in twenty is a convention, not a law; choose your own bar before the test, write it on line 5 and never move it afterwards.
| Total verified applications, both messages | Smallest split that clears the bar | Chance of a split that wide or wider, either direction |
|---|---|---|
| 5 | None: even 5-0 falls short | 6.3% |
| 6 | 6-0 | 3.1% |
| 8 | 8-0 | 0.8% |
| 10 | 9-1 | 2.1% |
| 12 | 10-2 | 3.9% |
| 15 | 12-3 | 3.5% |
| 20 | 15-5 | 4.1% |
| 30 | 21-9 | 4.3% |
| 40 | 27-13 | 3.9% |
Method note. Each row is the exact binomial calculation from the previous section with p = 0.5, summed over the winning split and every wider one, then doubled because either message could have been the leader. The table needs equal sends per message and independent responses. If one message went to more CEOs than the other, it does not apply; get help from someone who can run an exact comparison for unequal groups, or rerun with equal groups. Clearing the bar says chance alone is an unlikely explanation for the split. It says nothing about how large the difference is, whether it will last, or whether it carries to another partner's network.
The decision rule, written before the first send
- Look at the split once. Wait until the total of verified applications across both messages reaches the number on line 4, or the end date arrives. Totals can be watched; the split is read only then. A rule that stops when the split looks good gives chance many tries.
- Winner. If the leader's count meets or beats the split on line 5, and the leader has no more requests to stop, complaints or misreadings than the other message, adopt the leader for the same kind of CEO.
- No winner yet. If the split falls short, keep your current message. Say "no winner yet", not "the messages are equal": the test could not tell them apart, which is a different claim.
- Cannot answer at this scale. If the end date arrives before the total on line 4, report the counts without percentages and choose the message on other grounds, such as truthfulness and relevance, saying so. A longer window, or pooling with other partners who used the same frozen texts and draw rule, are the honest routes to a bigger total.
- Veto. Any records sent to you stop the test until the wording is fixed. More complaints or misreadings in the leader block its adoption, whatever the split.
A rule like this cannot speak to acceptance or payment. SourceX decides fit in its review under the criteria of the specific program, and section 5 of the terms says a lead, meeting or signed agreement alone does not earn a payable reward. Changes on SourceX's side matter too: section 11 says SourceX may publish new versions of the terms, and a change in the middle of a test is a reason to close it and start again.
Four replies to one, worked through (Illustrative)
Illustrative: invented partner, invented counts. A partner sends angle A to 30 CEOs and angle B to 30, alternating within each sending day. By day ten, four CEOs have replied to A and one to B, and a colleague announces that A has won.
| Line | Message A | Message B |
|---|---|---|
| Sends | 30 | 30 |
| Replies | 4 | 1 |
| What the replies said | 2 declined, 1 asked a question, 1 said yes | 1 said yes |
| Verified applications at the cutoff | 1 | 1 |
| Requests to stop | 1 | 0 |
The worksheet's verdict is straightforward. The outcome count is 1-1 with a total of 2, far below any row of the table; the only lopsided count, replies, was never the outcome, and message A's lead in replies came with its only request to stop. The verdict is "this test cannot answer at this scale". Line 9 reads: pool with two partners in the same network who agree to use both frozen texts unchanged, and read the split when the pooled total reaches 20, at which point it would take 15-5 to name a winner.
Where the arithmetic can mislead
- Responses are not independent. CEOs in one peer group compare notes, and one enthusiastic CEO can bring a second. Clustered responses make chance splits wider than the table assumes.
- Unequal groups. If one version went to more CEOs, more outcomes for that version are expected even when the texts perform identically, and the table no longer applies.
- Many cuts. Split the results by sector, size and relationship type and one cut will clear the bar by chance. Cuts are descriptive; only the comparison on line 1 can produce a verdict.
- Drift over a long window. Season, news and changes to SourceX's pages move responses. Interleave the sends and restart after a terms change.
- Small effects need large totals. A real but modest difference will usually fail to clear the bar at 20 outcomes. "No winner yet" is the expected verdict for most partners most of the time, and it is a correct one.
Two related pages cover neighboring problems: reporting a small pilot without exaggerated percentages, which sets out counts over named denominators, and the ten-introduction experiment, which tests a thesis rather than two messages. The checklist-versus-control design applies the same reading discipline to a different treatment.
Next step
Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.
Fill in lines 1 to 5 of the worksheet before your next comparison goes out: register as a partner and sign the Channel Partner Agreement first, keep the outreach consistent with who qualifies, and note the program terms version on the sheet; more designs are in the guides hub.