Change log: what changed in our numbers
AIamond publishes how often news sources were right and how that was measured. When we change how a number is measured, or find a mistake in the data behind it, it is recorded here with the date, what was wrong and what it does to the published figures. Nothing is removed from this list.
- DataCorrection running
Currency pairs were checked against the wrong market
Until 1 November 2026 a text rule filed currency pairs such as EUR/JPY, EUR/GBP and NZD/JPY under “Japanese stocks”, “Australian stocks” and “UK markets”, so their price checks ran against the Nikkei, the ASX and the FTSE instead of the currency pair. Of the 5,533 stored reports under those three labels, 1,617 are about a currency pair (1,174 “Japanese stocks”, 206 “UK markets”, 237 “Australian stocks”). New reports are filed correctly since 1 November 2026. The older ones are being moved to their currency pair and re-evaluated in the background; the old label is kept with each moved report. The rule only moves clear cases. We estimate that 15 to 20 % of the currency reports under these labels, mostly yen stories without a pair code in the title, are not recognised and stay where they are, so the figures for Japan, Australia and the UK remain less reliable than the others. On the old method, taking the moved reports out changes the share of correct calls of these three markets by 1 to 3 points. Currency pairs that were never evaluated before, such as AUD/USD and USD/CAD, are evaluated now, so the forex figures gain rows. The figures after the re-evaluation will be added here.
- Measurement methodDone
Checks now start at the first price a reader could trade
Until 1 November 2026 the check “did the market move the way the article said?” started at the close of the day the report was written. Nobody can trade at a price that is already behind them. Now the check starts at the first tradable price after the article was published: the session open if the article appears before the open or after the close, the next hourly price during the session. The difference is large. The same 23,469 fresh reports about stock indices (analysed within two hours of publication) show an edge over chance at 1, 3 and 7 days of +7.2 / +2.6 / +2.0 points when measured from the previous close, +3.5 / +0.4 / +0.7 from the session open and +0.8 / −0.5 / 0.0 (± 0.3) from the first tradable price. Against the previous close, news looks right mostly because it describes moves that had already happened. Older reports are re-computed in the background; until then the source pages mix old and new rows.
- DataDone
New reports carry a word-for-word quote from the article
To make “the source says …” checkable, the analysis now has to copy one passage from the article text word for word, and our code checks that it really occurs in the text. A report keeps its direction only if the quote is found and the text contains a real outlook, not a recap; otherwise the report keeps its market but has no direction. In a test on 200 articles the model's quote was found in the text in 139 of 142 cases; the 3 invented ones would have been dropped. Reports written before 1 November 2026 have no quote. For them, an independent re-reading of 800 random reports found that only 59 % of those carrying a direction were truly forward-looking.
- DataDone
Late reports now carry the time of the article
Articles that were analysed days after they appeared were given “now” as the time of their report, so feeds, buzz and the checks treated old news as new. In October 53 % of the reports were written more than six hours after publication. New reports now carry the publication time of the article; the time of the analysis is stored separately.