When better screening lowers applications but improves outcomes: a quantity-versus-quality cohort analysis that keeps CEOs who agreed to hear as the fixed denominator, so a shorter pile of applications counts as progress only when qualified outcomes hold or rise

Short answer

Only when qualified outcomes per CEO reached hold or rise. Ask CEOs to check their company against SourceX's published who-qualifies page before applying, and applications will probably fall while the share that gets through review climbs, almost automatically, because the denominator shrank. That climb proves nothing. Fix the denominator at CEOs who agreed to hear about SourceX, count verified applications still standing at a cutoff, and watch for fitting companies that talked themselves out. No results from this design are published yet.

Visual briefFull-size diagram
Two panels of stacked fractions for one Illustrative cohort of 40 CEOs. Left, per application: 3 over 15 for the standard message becomes 3 over 6 after the self-check, labeled looks twice as good and struck through. Right, per CEO who agreed: 3 over 40 stays 3 over 40, labeled no change, under the headline read the bottom number first.
Illustrative arithmetic that shows how to read the measure. No partner or SourceX results for this design are published.

Screening here means one thing: the company checks itself against who qualifies before it applies. You, the partner, never collect, review or score company records, and you never see the CEO's answers. Fit is decided in SourceX's own review, which also checks relevance to its current data needs.

Fewer applications is the expected effect, not the finding

A self-check step is supposed to cut applications. A CEO who reads that client-owned material, scraped data and personal consumer data are usually not a fit, and recognizes the company's best records in that list, should stop before applying. That saves the CEO an application and saves SourceX a review. So when applications drop after you add the step, you have learned only that the step was read.

The finding you want is different: did the companies that still applied do better, without losing companies that would have fit? Two outcomes look identical at the application stage and mean opposite things. In one, the self-check removed companies SourceX would have closed anyway. In the other, it scared off a company that met every criterion but read one line too strictly. The who-qualifies page itself warns against strict readings: "Some programs set higher thresholds, and smaller specialists can still be considered."

Why the bottom of the fraction must stay fixed

Epidemiologists met this trap long ago. CDC's training lesson on measures of risk says the denominator for a risk measure is the population at the start of the observation period, limited to the population at risk: the people who could end up in the numerator (paraphrase). For a partner, the population at risk of becoming a qualified outcome is every CEO who agreed to hear about SourceX, whether or not that CEO later applied.

Applications cannot be that denominator, because the self-check changes them. Dividing outcomes by applications rewards any step that discourages applying, good or bad.

Illustrative arithmetic, not results. Invented counts for one cohort of 40 CEOs, to show how the same outcomes read under each denominator.

Cohort (Illustrative)ApplicationsQualified at cutoffPer applicationPer 10 CEOs who agreedWhat it means
Standard message1533 of 150.75The baseline
Self-check, reading X633 of 60.75Same outcomes, nine fewer applications
Self-check, reading Y622 of 60.5Per application looks better; a fit was lost
Self-check, reading Z855 of 81.25A real improvement

Reading Y is the one a per-application report hides: 2 of 6 beats 3 of 15, yet one fewer company is still standing.

Standard, self-check, standard: three cohorts in sequence

A parallel test that splits CEOs between two messages at the same time, like the company fit checklist experiment, is the stronger design when you introduce enough companies at once. Many partners do not: they introduce a few companies a month and want to know whether a change they are already considering helps. For them, a planned sequence works better.

ElementSpecification
DecisionKeep or drop a self-check step in your own outreach
HypothesisThe self-check lowers applications per 10 CEOs and holds or raises qualified outcomes per 10 CEOs
UnitA company whose CEO agreed in writing to hear about SourceX before any link was sent
Cohort 1Your standard message for every CEO who agrees during the first window
Cohort 2The self-check message for every CEO who agrees during the second window
Cohort 3The standard message again, for the third window
AssignmentBy the date of the CEO's yes, with window dates written down before the first message
ExclusionsCompanies you already know were in contact with SourceX, removed before assignment and never afterwards
ComparisonCohort 2 against cohorts 1 and 3 together

Why the return to the standard message? Outcomes drift for reasons that have nothing to do with your wording: SourceX's programs move, your network warms up, a season changes who answers email. If cohort 2 beats cohort 1 but cohort 3 matches cohort 2 with the old message, the calendar did it.

Both messages are truthful. Each says that you may receive a reward from SourceX if the company completes a qualifying deal, that SourceX does not endorse you or promise anyone a result, that nobody gets preferential access to the company's data, and that the CEO alone decides. The self-check message adds one paragraph: read the who-qualifies page, decide for yourself whether to apply, keep your answers to yourself, and if a line is unclear, apply and let SourceX's review settle it or try the company fit checker, which SourceX describes as "a preliminary, non-binding screening." It never says or hints that a company qualifies, and never tells anyone to stay away.

The quantity-versus-quality cohort analysis

One row per cohort, every count read at the same age: 120 days after that cohort's last entry, so the 90-day attribution window and some review time fit inside it. Copy SourceX's stage and status words exactly as your dashboard shows them; how it works lists the reward statuses as Estimated, Eligible, Approved and Paid.

ColumnDefinitionSource
CEOs agreedCompanies assigned to the cohortYour dated written records
ApplicationsCohort companies that applied, seen as referrals on your dashboardDashboard
Qualified outcomes (primary)Applications SourceX verified and credited to your introduction that are not closed at the read dateDashboard
Later-stage outcomesQualified outcomes showing a reward figure at a second read, 180 days after the last entryDashboard; the terms call any figure before supplier payment an estimate based on proposed supplier compensation
Closed in reviewApplications SourceX closed by the read dateDashboard
DuplicatesCompanies SourceX treated as already known to itDashboard
Per 10 CEOsApplications, qualified and later-stage outcomes, each divided by CEOs agreed and multiplied by 10Calculated

A click, a visit and an opened email appear nowhere in the table. The terms say an earlier link click alone does not decide credit, and a lead, meeting or signed agreement alone earns no payable reward, so neither belongs in a quality measure. The later-stage column is a progress signal, not a forecast of anything being paid.

What self-screening can cost, counted from the first message

MeasureWhat it detectsHow you record it
Self-declinesCEOs who decided not to applyTheir reply, logged as declined with no reasons kept unless offered
Size-line stopsPossible lost fits, because the page says smaller specialists can still be consideredAn optional follow-up, "which part of the page made you stop, if any?", logged as the section name only
Gatekeeping complaintsA CEO who felt pre-judged or turned awayUnprompted replies
MisreadingsA CEO who took the self-check as SourceX's verdict, positive or negativeUnprompted replies
Answers or records sent to youProtocol failures; the target is zeroYour inbox
Days from yes to applicationWhether the extra reading slows companies downDates you already hold

Record a decline without the private reasons behind it; the seven-code decline taxonomy shows how. Never contact a company that declined to argue it back in.

Cohort floors, read dates and stop rules

Floor. Each cohort needs at least 15 CEOs who agreed before any per-10 figure is published. If a window ends short, extend that window; never merge cohorts. Below the floor, report counts only.

Duration. Windows of equal length, set in advance; eight weeks each suits a partner who introduces a few companies a month. With reads at 120 and 180 days, the full design runs about a year.

Stop rules.

  • A company sends you its answers or any record: stop using the self-check text until it is rewritten, and keep nothing that arrived.
  • A CEO reads the self-check as SourceX's rejection or approval: pause cohort 2 and fix the wording before the next message.
  • SourceX changes who-qualifies or the terms during a window: close the cohort on that date, note both versions and restart the sequence.
  • More than a third of cohort 2's self-declines name the size section: pause and rewrite that paragraph, because the page itself leaves room for smaller specialists.
  • Never stop because applications fell. That is the expected effect, and stopping on it would end the test just before it says anything.

Four quadrants and the decision each one supports

Before cohort 1 starts, write down what "about the same" means for qualified outcomes per 10 CEOs, for example within 0.25 either way, and state uncertainty as a range rather than a single figure.

Applications per 10 CEOsQualified outcomes per 10 CEOsDecision
FellRose, above your margin, with no rise in complaintsKeep the self-check: screening improved outcomes
FellAbout the sameScreening saved effort without visible loss; keep it only if size-line stops and complaints stay low, and say this is your judgment
FellFellThe self-check is costing fits: drop or rewrite it
Did not fallRoseSomething other than screening changed, perhaps clarity; investigate before crediting the self-check

Any protocol failure above zero comes first: fix the wording, then read the quadrants. Cuts by sector or relationship type stay descriptive. If your counts are small, the form walkthrough is a cheaper first test of where companies stumble.

Where a sequential cohort design misleads

  • Unequal cohorts. The CEOs who agree in spring may differ from those who agree in autumn. Cohort 3 catches drift, not every difference; describe each cohort's mix of relationship types in one line.
  • Self-report you cannot check. A CEO who rules the company out may be wrong. The size-line follow-up only flags the likeliest error.
  • Review is wider than the page. Relevance to current data needs and the rights review are SourceX's; a self-check that predicts applications well still predicts nothing about acceptance or payment.
  • Long clocks. A year is a long time for programs to stay still, which is why every cohort carries the version dates it ran under.
  • Duplicates. A company already known to SourceX is credited to nobody, in any cohort, so check before assignment.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

Before cohort 1 opens, register as a partner and sign the Channel Partner Agreement, draft the self-check paragraph only from the current who qualifies page, and log the program terms version each cohort runs under. Related test designs sit in the guides hub.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Is a higher share of applications getting through SourceX's review a good sign?

Not on its own. When fewer companies apply, the share that gets through rises almost automatically, because only the denominator moved. Compare qualified outcomes per CEO who agreed to hear about SourceX instead.

Who does the screening in this design?

The company. Its CEO reads the published who-qualifies page and decides whether to apply. The partner never sees answers, collects records or scores anyone, and SourceX's own review decides fit.

Can I run the three cohorts faster by putting every CEO into the self-check group?

Then nothing can be compared. The two standard-message cohorts on either side are what show whether a change follows the message or just the calendar.

Sources

  1. refer.sourcex.si/who-qualifies
  2. refer.sourcex.si/terms
  3. refer.sourcex.si/how-it-works
  4. archive.cdc.gov/www_cdc_gov/csels/dsepd/ss1978/lesson3/secti…

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment