AI stock prediction accuracy: what a fair test shows
Accuracy numbers mean little without a baseline. We tested a simple learned model (it re-trains every day on the last 3 years) on predicting next-day up or down for 10 years. It was right 53.0% of the time on SPY - but SPY simply rose on 55.0% of days, so guessing 'up' every day did better. The same held on QQQ, the Nifty 50 and Bitcoin. A real edge must beat the right baseline, after costs, on data the model never saw.
Next-day direction, out of sample, 14 Oct 2016 - Oct 2026 (Yahoo daily closes)
| Market | Days tested | Learned model right | 'Always up' right | 'Same as yesterday' right | Luck band (2 SE) |
|---|---|---|---|---|---|
| S&P 500 ETF (SPY) | 2,511 | 53.0% | 55.0% | 50.0% | ±2.0 pts |
| Nasdaq 100 ETF (QQQ) | 2,511 | 56.0% | 56.4% | 50.2% | ±2.0 pts |
| Nifty 50 | 2,466 | 52.6% | 53.8% | 53.9% | ±2.0 pts |
| Bitcoin | 3,653 | 51.7% | 52.2% | 48.1% | ±1.7 pts |
A fair test has three parts
- Out of sample: the model only sees data before the day it predicts. Ours re-trained daily on the previous 756 trading days.
- The right baseline: markets that rise over time are 'up' on more than half of days, so 53% accuracy can be worse than guessing 'up'. Compare with that, not with 50%.
- Costs and luck: with about 2,500 days, results within roughly ±2 points of the baseline can be luck. And being right often is not the same as making money: one big wrong day can cancel many small right ones.
What about big AI models and '90% accuracy' claims?
Claims of 80-99% accuracy usually come from testing on the same data the model learned from, predicting something easier than direction, or tiny samples. Academic studies of large language models reading news report small, short-lived effects. Our backtest checklist shows how to check a claim, and our test of AI-written trading rules found none beat buy-and-hold over 10 years.
Our own luck-check example
When we tested a popular day-trading setup across 34 stocks it made 17,183 trades and won 26% of them - profitable in 1 of the 34 stocks. Many trades does not mean an edge; checking the result against chance does.
Questions
- How accurate is AI at predicting stocks?
- In our 10-year test, a simple learned model predicted next-day direction 52-56% of the time - no better than always guessing 'up'. Accuracy without a baseline is not evidence.
- Can AI predict the stock market?
- It can find patterns, but published, independently checked evidence of reliable profits after costs is thin. Most high accuracy claims fail a fair out-of-sample test.
- What accuracy would be meaningful?
- Clearly above the 'always up' rate for that market, out of sample, over thousands of predictions, and still profitable after costs.
- Is 60% accuracy good for stock prediction?
- Only if it beats the market's own up-day rate and the wins are not smaller than the losses. Check both before trusting it.
Sources
- Lopez-Lira and Tang: Can ChatGPT forecast stock price movements? (SSRN)
- Yahoo Finance: SPY historical prices
Check a prediction track record
Upload the trades or calls and Truth Test checks them against luck - free.