We rerun each model in up to nine ways meant to break it, because one good test on past prices proves little. So far only the model rated Robust has been scored on all nine.
Six start dates
Does the result depend on the day it started?
We run the model from six start dates, average the results and show the worst start too. The start date alone can move one test by about 2 points a year.
Six start dates
Start dates, sketched. The adopted model's report card: +22.0% a year on average, worst start +19.2%.
Worse prices (0.3% each way)
Does it survive worse prices?
Every trade gets a price 0.3% worse than the open (the usual is 0.1%), on top of the trading charges.
Worse prices (0.3% each way)
Sketch of a price bar, gaps enlarged. Report card figure: +21.7%.
Without its 5 best winners
Is the result carried by a few lucky stocks?
We remove the five stocks that made the most money and measure again. One tested version earned over 40% of ten years’ profit from a single stock.
Without its 5 best winners
Sketch of bars. Report card figure: +18.9%.
From 2023 onwards
How did it do from 2023 onwards?
We measure 2023 onwards on its own. Those years were already visible when the settings were chosen.
From 2023 onwards
The model's real growth, log scale (equal percentage moves look equal). Report card figure: +24.7%.
2023 onwards, without 5 best winners
Do both hold at once?
We measure 2023 onwards with the five best stocks removed. It is the hardest of the nine.
2023 onwards, without 5 best winners
Figures from the report card: 2023 onwards +24.7%; without its 5 best winners +12.5%.
Random-pick portfolios
Could stocks picked at random have done as well?
We keep the entry rules, stake size and costs, but pick the stocks at random from the same list. We repeat this many times.
Biggest winner's share of profit
How much of the profit came from one stock?
We measure how much of the total profit came from the best single stock. A high share means one stock did most of the work.
Biggest winner's share of profit
Figure from the report card: 3%. A small share means the result does not hang on one stock.
Worst fall
How deep was the worst fall?
We take the biggest fall from a peak to a later low. We show it averaged over the start dates, and at the worst start.
Worst fall
The model's real fall from peak, weekly, from the release. Report card figure: −35% average, −35% at the worst start.
Without the 100 biggest stocks
Does it need the biggest stocks?
Each month the 100 biggest stocks by traded value leave the list, and the model runs on what remains.
How do you read the report card?
The Robust model's report card shows all nine checks, each with a figure and a pass mark. Other models show the seven they ran, without marks: all but random picks and the run without the biggest stocks. The figures are on each model sheet. This chapter explains the checks.
The diagrams use the report card of Elm and its released data.