How an AI impact score for market news is actually built
What a 0-10 news impact score measures, the rubric behind it, why novelty beats severity, and the specific ways an automated score is wrong.
THE SHORT ANSWER
An impact score estimates how much a rational trader's positioning would change on reading one specific headline. The two variables that decide it are novelty — is there a new, direction-relevant fact — and confirmation status. Severity of the underlying subject is deliberately not the scale.
Automated news scoring has a reputation problem, and it is deserved. Most implementations run a general-purpose sentiment model over a headline and publish the result as though it described market relevance. It does not. "Earthquake kills hundreds" is maximally negative and frequently moves nothing; "Committee vote was 7-2 rather than 8-1" is emotionally flat and repriced the front end of a curve.
This piece describes how the score in this product is constructed, because a score whose rubric is secret is a number you have to take on faith, and a number you take on faith is decoration.
The scale measures positioning change, not severity
The question the model is asked is narrow on purpose: how much would a rational trader's positioning or view on price direction change on reading this specific headline right now? Not how serious the subject is. Not how much coverage it will attract. Two headlines about the same underlying situation can and should score very differently.
| Band | What belongs there |
|---|---|
| 9-10 | A development that would plausibly move markets broadly and materially right now: a new interstate war, a genuine change in an existing conflict's trajectory, a surprise central-bank decision once announced, a major exploit or venue collapse, a sovereign default |
| 7-8 | Scheduled FOMC / CPI / NFP outcomes once released, a major corporate action or filing, significant sanctions, an ETF approval or denial, a genuinely new escalation within a known situation |
| 4-6 | Notable sector news, a large prediction-market odds swing, an official statement that is opinion rather than decision, routine continuing coverage that adds minor detail |
| 0-3 | Noise: opinion pieces, price recaps, promos, routine social chatter, vague unconfirmed rumour |
Novelty is the first filter
The rubric asks whether the headline adds a new, direction-relevant fact beyond what is already known and priced, or whether it is additional colour, an operational detail, or a restatement. An ongoing situation continuing along its expected path is not high impact merely because the topic is grave. A genuine new escalation or reversal within that same situation can still reach the top band.
This single rule removes most of the noise that makes naive scoring useless. Severe topics generate enormous headline volume, the overwhelming majority of it restating a known state of the world. Scoring by subject produces a feed that screams continuously and therefore carries no information. Scoring by what is new in this specific headline produces a feed that is usually quiet.
Confirmed fact versus speculation
The second filter separates things that have actually happened — released, decided, officially confirmed, or stated by someone with direct authority over the outcome — from everything else. Rumours, one-sided claims, analyst forecasts, odds updates and hedged language all mean nothing has been decided yet, and they are capped several points below what the same story earns once confirmed.
- Hedged verbs — weighing, mulling, considering, may, might, could, expected to, divided on, sources say — are treated as "not yet an event".
- A leader's subjective characterisation of a situation is opinion, not a fact about it.
- A prediction-market odds move is information about expectations, not the outcome.
- A headline with no named action, no number and no confirmed actor cannot exceed the noise band regardless of how dramatic its wording is.
Duplicate suppression
A single development produces a dozen near-identical headlines within minutes across outlets, plus running updates on the same story. Scoring each one at full impact would turn one event into a wall of alerts and make the feed a proxy for press-release volume. The batch is scored together: the most informative statement of the development takes the deserved impact, and the rest are scored as follow-ups unless they add a materially new fact.
What else the pass produces
Impact is one field of several, and the others are what make the score usable. Each event also gets a sentiment value in the range -1.0 to +1.0 representing the expected market reaction for the most affected assets, a single category, the list of watchlist symbols the event materially concerns, a region, and coordinates for where the event occurs — not where it was reported from.
The asset mapping is the part that turns a news feed into something tradeable, and it is deliberately conservative: only symbols the event materially concerns, and an empty list when none apply. A headline tagged with eight assets is usually a headline tagged with nothing. The same structured fields are readable in the console, available through the events endpoint, and the delay applied to each view is set out on the pricing page.
The ways this is wrong
An honest description of a model includes its failure modes, and these are the recurring ones.
- It cannot know current positioning. The same news lands differently on a crowded book than on a clean one, and the score does not see the book.
- Novelty is judged against the model's context, not against the market's memory. A fact that is new to the feed may have been priced days earlier.
- Category and asset mapping degrade on genuinely unusual events — the ones that matter most.
- Sentiment is an expected reaction, not a measured one. It is a prior, and priors are frequently wrong.
- A score is produced under time pressure from a headline, sometimes before the full text exists.
The correct use follows from the limits. A score is a triage instrument: it decides what deserves a human look in the next thirty seconds, and it does not decide what to do about it. Everything after the score — the transmission map, the confirmation, the invalidation — is the process in the breaking-news checklist.
The question of whether the scale is calibrated — whether headlines the engine calls an 8 are followed by larger moves than ones it calls a 5 — is an empirical one, and it is answered by measurement rather than assertion. Measuring news impact on price sets out the method.
Frequently asked questions
- What does an AI news impact score measure?
- How much a rational trader's positioning would change on reading that specific headline right now. It is a market-impact scale rather than a severity scale, which is why a grave but well-known situation continuing along its expected path scores low while a small, genuinely unexpected fact can score high.
- Is news sentiment analysis useful for trading?
- General-purpose sentiment on its own is a poor proxy for market relevance, because emotional tone and price impact are different things. Sentiment becomes useful when paired with a novelty judgement, a confirmation status and an explicit mapping to the assets actually exposed.
- Why do most headlines score low?
- Because most headlines restate something already known. A scoring rule that rewards novelty rather than subject matter produces a feed that is quiet most of the time, which is the intended behaviour — a feed that flags everything flags nothing.
- Can an impact score be wrong?
- Routinely. It cannot see current positioning, it judges novelty against its own context rather than the market's memory, its asset mapping degrades on unusual events, and its sentiment field is a prior rather than a measurement. It is a triage tool for deciding what to look at, not a conclusion.
Related reading
- News intelligenceA trader's checklist for breaking market newsHow to separate a headline that changes market structure from one that only creates noise: source, delta from expectations, transmission channel and confirmation.
- News intelligenceHow much does news actually move price? A method for measuring itEvent-study mechanics for traders: choosing horizons, defining a base close, handling events you cannot measure, and why a result without a baseline says nothing.
- Trading processHow to check whether a trading signal record can be falsifiedMost published signal records cannot be proved wrong. The seven questions that separate a scoreable record from marketing, and what an honest denominator looks like.
Or browse the full library on the research index, and see the same data live in the console.