The mistake that looked sensible
For our first six months, confidence in Semradr meant one thing: how far a number had moved. A search term up 200% got a higher score than one up 20%. It felt right. Big moves are easy to see, easy to explain and easy to get excited about.
It was backwards.
Why a big move on one source proves little
A number can jump for reasons that have nothing to do with demand: one viral post, one promotion, one broken listing feed, one outlet repeating another. When a single source moves 200% and nothing else does, the most likely explanation is that something happened to that source.
A smaller move is harder to dismiss when it shows up everywhere. If search interest, marketplace listings, social posts and prices all lean the same way, each by a modest amount, they are unlikely to be explained by one accident.
A signal that moves 200% on one source is weaker than one that moves 20% across five.
What confidence measures now
Confidence now reflects how many independent sources agree on direction. Independent means they measure different things in different ways: what people search for, what sellers list, what people post and what things cost. Two sources that copy each other count as one.
- Agreement across independent sources raises confidence.
- A large move on a single source does not.
- Disagreement between sources is shown, not averaged away.
Magnitude still matters. It decides which signals you see first and how urgent they are. It just no longer decides how much we trust them.
What it cost us
When we recalculated history with the new method, some scores went down, in a few cases noticeably. We marked the change as both a methodology change and a breaking change in the changelog, because a silent change to how a number is calculated is indistinguishable from a lie.
How to read this in the product
Every opportunity shows its confidence next to the evidence behind it. Measured evidence and generated interpretation are labelled separately, and the counter-signals that weaken the case are shown too. See how an evidence screen works.