Forecast accuracy is not a trading claim
Forecast error, ranking quality and portfolio return are three targets with three different baselines.
Short research notes on assumptions, baselines and what a result can support. They are working notes, not peer-reviewed papers; dates give the month of the underlying analysis.
Forecast error, ranking quality and portfolio return are three targets with three different baselines.
A placebo test condemned a mediation result, until a positive control showed it condemned everything.
Replaying real index membership removed about a third of an equal-weight universe’s return, yet left the comparisons intact.
A bounded KV cache sped up Kronos inference 11.7× with bit-identical output, inside a boundary it has to respect.
Write down exposure, mediator and outcome at each wave before fitting anything.
Check what was knowable at each decision, then compare against a simple baseline.
A lower prediction error does not automatically become a better signal, and a signal does not automatically become a profitable portfolio. Each step changes the target, the baseline and the relevant friction.
My Kronos evaluation follows that chain. In a five-path CSI300 run of the public Kronos-small checkpoint, the ranking signal was not statistically significant (Newey–West p = 0.295), and the portfolio result moved from +6.01% to +1.93% a year when ten forecast paths were averaged instead of five. A single headline number would have hidden both facts.
↑ All notesIn a longitudinal mediation analysis, a temporal-placebo negative control first suggested that an apparent mediation effect was an artefact. A positive control, the same test applied to the exposure–outcome effect we had most reason to believe, condemned that too. The control could not tell a real effect from a spurious one in this setting, so its verdict was withdrawn and the question moved to other tools.
The lesson is general: before a falsification test is allowed to reject a result, show that it can pass one.
↑ All notesBacktesting today's S&P 500 members from 2015, an equal-weight universe returned 16.14% a year. Holding each stock only while it was actually in the index returned 11.06%. About a third of the apparent return came from knowing, in 2015, which companies would still be there.
The bias was shared by strategy and benchmark, so active comparisons held up while absolute levels did not. That is why the Helix pages quote no absolute backtest returns.
↑ All notesKronos generates forecasts one bar at a time, and the original loop recomputed every layer over the whole context at each step. I added a pre-allocated rolling key–value cache. On CPU, with a 400-bar lookback and 100 greedy steps, it ran 11.7 times faster with bit-identical output, and the upstream regression tests still pass.
The cache is exact only while the lookback plus the forecast length fits inside the model's context window. Beyond that it falls back to the original path rather than returning a faster, subtly different answer.
↑ All notesFor longitudinal data, write down the exposure, mediator and outcome at each wave before fitting a model. Prior outcomes and exposure-affected covariates can change which direct or indirect effects are identifiable.
I keep intervention-regime estimands separate from within-person associations. Both can be informative, but they answer different questions.
↑ All notesThe first question is not whether a curve rises. It is whether every input was available when the decision was made. I look for point-in-time membership, realistic execution prices, costs and a test window left untouched by model selection.
A random walk or simple momentum baseline is often more useful than another complex model. If the candidate fails that comparison, the failure is information about the research process.
↑ All notes