How to measure whether a tool helps partners find better fits: a baseline-then-split screening study that compares partners with and without a criteria tool on qualified outcomes per company considered, and audits every company the tool led them to set aside

Short answer

Measure the tool by what happens to the companies partners consider, not by how often they open it. Run every partner without the tool for a baseline period, then give it to a randomly chosen half for a second period. Compare how qualified outcomes per ten companies considered change in each half, and audit every company the tool led a partner to set aside. The tool only restates SourceX's published criteria; partners never upload, scan or score records, and SourceX's review decides fit. No results have been published.

Visual briefFull-size diagram
A two-by-two grid of Illustrative rates per 10 companies considered: the tool half moves from 1.0 in the baseline to 1.6 in the tool period, a change of +0.6, and the usual half from 1.0 to 1.2, a change of +0.2; a side panel reads difference in differences, 0.6 minus 0.2 equals 0.4, credited to the tool, and a row beneath lists the tool's only outputs, ask, ask first and not now, with score struck through.
Illustrative arithmetic with invented rates; SourceX publishes no conversion rates and this study's results are not published.

The tool here is the partner's own: a short question sheet, a spreadsheet tab or SourceX's company fit checker, used before a partner decides whether to raise SourceX with a CEO. A different question, whether a company that checks itself applies more selectively, is tested in the company fit checklist experiment.

What a fit tool may do, and what it may never touch

A fit tool turns the who-qualifies page into questions a partner answers from two places only: public information and what the CEO has said. Full-time headcount at the company's peak, contractors left out. Years of records the company created itself. Rights to license them. Where the company is, read against the published location sentence. Whether the CEO knows about the idea and wants the conversation.

The tool mayThe tool may never
Restate each published criterion as a question, in SourceX's wordsAccept a file, a pasted export or a screenshot
Return one of three outputs: ask, ask first or not nowReturn a score, a percentage or the word "qualifies"
Turn every unknown into a question for the CEOTreat an unknown as a fail
Link to who-qualifies and the termsAdd a criterion SourceX does not publish, such as a guess about demand
Change the day SourceX changes its pageKeep an old version running

"Ask" means every known answer passes. "Ask first" means at least one answer is unknown, and the unknown becomes a question for whoever holds the CEO relationship. "Not now" means a published criterion fails on a fact the company stated or that is public, and the log records which one and what would change it. Who-qualifies puts the rule for the middle case in one line: "Unknown is not a fail; it is a question for the relationship holder."

SourceX describes its own checker as "a preliminary, non-binding screening," and its page adds: "Nothing is saved unless you choose to, and no contact details are needed." Whatever a tool says, SourceX's review decides fit, including whether a company's records match a current program. Partners never collect, scan or score company records, as the no-upload screening form also shows.

The hypothesis and the count that decides

Hypothesis. Partners who run the tool before asking a CEO produce more qualified outcomes per ten companies considered than they would have without it, and set aside no company on an unknown or on a guess about demand.

Primary measure: qualified outcomes per ten companies considered.

  • A company considered is any company a partner thinks of raising with its CEO, logged with the date at the moment it comes to mind and before any tool is run. Both halves follow the same logging rule.
  • A qualified outcome is a considered company whose application SourceX verified and credited to that partner's introduction, still open on the read date. Copy SourceX's stage words exactly as the partner dashboard shows them; how it works lists the reward statuses as Estimated, Eligible, Approved and Paid.

Why companies considered, not CEOs asked? A tool that simply makes partners ask fewer CEOs would look brilliant per ask while losing fits. The considered log is written before the tool can influence it, so it stays fixed. The screening cohort analysis explains the same trap from the company's side. Why not clicks or tool sessions? The terms say an earlier link click alone does not decide credit, and that a lead, meeting or signed agreement alone earns no payable reward.

Baseline first, then a split by lot

ElementSpecification
UnitA partner, because the tool changes how a person screens
Period 1, baseline90 days in which every partner works as usual and keeps the considered log
SplitPartners are paired by their period 1 volume, and one in each pair is drawn by lot to get the tool
Period 290 days in which the tool half runs the tool before every ask and the usual half carries on
Read datesEach period is read 120 days after it ends, so the 90-day attribution window and some review time fit inside
ExclusionsCompanies SourceX treated as already known, removed from both halves in both periods by the same rule
EstimateTool half's change from period 1 to period 2, minus the usual half's change

The baseline is there because a partner team is small. With a dozen partners, the two halves can differ before anyone touches the tool, and subtracting each half's own baseline removes differences that stay constant. This is the difference-in-differences design. The World Bank's Impact Evaluation in Practice rests it on an equal trends assumption between the groups, and its DIME Wiki suggests comparing the groups' trends across several earlier periods and repeating the estimate with other comparisons (paraphrase). If your logs reach back another quarter, compare the halves on it before reading period 2.

A simple before-and-after on the whole team cannot do this job. Who-qualifies says requirements follow SourceX's current data needs and differ by program, so outcomes drift for reasons no tool caused. The usual half absorbs that drift.

The tool-assisted screening study log

One row per company considered. The study lead sets up the columns before period 1 and locks the half and period fields at the split.

FieldFilled byRule
Company and legal namePartnerOpened when the company first comes to mind
Date consideredPartnerWritten before any tool run
Partner, half, periodStudy leadNever changed after the split
Tool outputTool half onlyAsk, ask first or not now, copied as shown
Partner's decisionPartnerAsk, hold or set aside
Reason for setting asidePartnerThe published criterion and where the fact came from, public or the CEO's words; "unknown" is never a reason
CEO asked and answerPartnerDates, and yes, later or no
Introduction submittedPartnerDate
Stage at read dateStudy leadCopied word for word from the dashboard
Already known to SourceXStudy leadAs the dashboard shows it

Columns that never exist: attachments, figures read from company documents, exports, screenshots, logins, and contact details for anyone at the company but the CEO.

The set-aside audit: fits the tool may have cost

Every company the tool half set aside in period 2, plus an equal random sample of the usual half's set-asides, goes to a second partner who is not told which half a row came from. The reviewer reads only the logged reason, against SourceX's published wording.

ClassExampleHow it counts
Published criterion, stated factThe CEO said full-time headcount never reached 50Justified, under section 1 of the terms
Unknown treated as a fail"Not sure what language the records are in"Wrongful set-aside
Guess about demand"SourceX probably has no use for freight records"Wrongful; relevance is decided in SourceX's review
Relationship reasonThe person who knows the CEO declined to askNot a fit question; left out of the audit

The target is zero wrongful set-asides. A tool that raises qualified outcomes by teaching partners to drop every company with an open question has bought its gains with lost fits, and the comparison alone would never show it.

Side effects counted alongside

  • The tool quoted as a verdict. Check one in five messages the tool half sends to CEOs for lines such as "the tool says you qualify." Target: none.
  • Company material in the tool or the log. Target: none.
  • CEO complaints, including a CEO who felt pre-judged.
  • Asks per ten companies considered, read beside outcomes; fewer asks is fine only if outcomes hold.
  • Minutes per company, so a team can weigh the tool's cost.

A partner who delegates screening to a colleague keeps the judgment calls; delegating screening without delegating judgment shows where those lines fall.

Floors, duration and stop rules

Floors. At least six partners in each half and 40 companies considered per half in each period before any per-ten rate is published. Below that, report counts and audit classes only.

Duration. Two 90-day periods, each read 120 days after it ends: the final read comes about ten months after the start.

Stop rules.

  • A file, export or record is pasted into the tool or the log: stop the tool half, delete it, fix the tool and restart period 2.
  • A partner tells a CEO the tool says the company qualifies or is approved: pause, correct it with that CEO and brief the partner again.
  • Two of the first ten audited set-asides from the tool half are wrongful: stop and rewrite how the tool handles unknowns.
  • SourceX changes who-qualifies or the terms on a criterion the tool restates: update the tool that day, and report outcomes before and after the change separately.
  • A CEO complains of pressure: pause that partner's asks and review the message.

Illustrative arithmetic, not results. Invented rates to show the calculation; SourceX publishes no conversion rates.

Half (Illustrative)Period 1, per 10 consideredPeriod 2, per 10 consideredChange
Tool half1.01.6+0.6
Usual half1.01.2+0.2
Tool half minus usual half0.00.4+0.4

The last row is the difference in differences. The usual half's 0.2 is drift no tool caused, so only the extra 0.4 is credited to the tool, and only if the audit found no wrongful set-asides. With 40 companies considered per half, 0.4 per ten is under two companies, well within what chance moves. A gap that size would justify continuing the study, not rolling the tool out.

Reading the comparison and the audit together

ComparisonAuditReadingAction
Tool half gains moreNo wrongful set-asidesThe tool helpsGive it to everyone and keep auditing a sample
Tool half gains moreWrongful set-asides foundGains bought with lost fitsFix the unknown handling and run period 2 again
No differenceNo wrongful set-asidesHarmless, not helpfulKeep it only if it saves time, and claim nothing about fit
Tool half does worseAny resultIt discourages good asksWithdraw it
Below the floorsAny resultNo comparisonReport counts and audit classes only

Where a partner-level comparison misleads

  • A few partners carry the result. One productive partner can swing a half. Pairing by baseline volume helps but does not cure it.
  • Spillover. Partners talk, and the usual half can absorb the tool's questions. Ask them not to share it, and note any leak.
  • Logging drift. The tool may prompt its users to log more companies, inflating their denominator. Compare each partner's considered counts across periods.
  • Being watched. The tool half knows it is being studied and may screen more carefully for that reason alone.
  • The tool is not the review. Even a perfect restatement of published criteria cannot see demand, rights or record quality; those stay with SourceX.

Next step

Refer a company. No cap on rewards. Default reward on a company's first qualifying transaction: the greater of $50,000 plus 5% and 15% of Actual Supplier Payout, with no cap; further qualifying transactions first agreed within 12 months of the first Closing pay 15%. Eligibility and payment conditions apply, including the USD 100,000 Minimum Transaction, and a signed Schedule A or addendum can differ. A lead, meeting or signature alone does not trigger payment.

When the tool returns "ask" and the CEO wants the conversation, register as a partner, compare the tool's questions with the live who-qualifies page and the current program terms, then submit.

  1. Step 1Share your linkSend your personal link to a company you know.
  2. Step 2Company appliesThe company applies itself at /apply.
  3. Step 3Company signs and is paidThe company signs, delivers its data and is paid by SourceX under its agreement.
  4. Step 4You get your rewardthe greater of $50,000 plus 5% and 15% of the actual supplier payout, with no cap. A reward becomes payable after SourceX has paid the referred company under its signed agreement and the payout conditions in the program terms are met, including finance approval and your tax and payment details.

Common questions

Can a fit tool give each company a score?

No. A score suggests a precision nobody has and invites partners to rank companies on guesses. Three outputs, ask, ask first and not now, keep every unknown as a question and leave fit to SourceX's review.

Can partners use SourceX's company fit checker as the tool under test?

Yes. SourceX describes it as a preliminary, non-binding screen, and the same design applies: the half that runs the checker before each ask is compared with the half that does not, on qualified outcomes per company considered.

What if our partners already know the criteria by heart?

Then the tool may add nothing, and the study will show that. A tool that restates a published page helps only where people misremember the page, which is worth knowing before anyone builds more of them.

Sources

  1. refer.sourcex.si/who-qualifies
  2. refer.sourcex.si/tools/company-fit-checker
  3. refer.sourcex.si/terms
  4. refer.sourcex.si/how-it-works
  5. dimewiki.worldbank.org/Difference-in-Differences
  6. documents1.worldbank.org/curated/en/698441474029568469/pdf/108270-PUB…

Free resources

Know a company with valuable proprietary data?

Become a referral partner, get your link and introduce the company’s CEO (in an owner-run company, usually the owner). Referral partners can join from most countries. We can't work with partners in countries or regions under U.S. sanctions or on U.S. restricted-party lists. The companies you introduce should be based in the U.S., Canada and the EU, or in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia with at least 70% of their data in English.

Refer a company →

I own a business

Explore licensing your company's data directly to SourceX for AI training and evaluation. Start a short assessment; no uploads needed.

Start an assessment