01 Home02 Research03 Projects04 Notes05 About06 Lab
Home/Lab

Experiments, run in your browser.

Four simulations behind questions in my research: when a backtest is luck, whether a forecast knows its own uncertainty, what to adjust for, and why panel data gives two answers. Each states its model, sets the simulation beside the theory, and ends with the Python that reproduces it.

EXP-01

Quantitative researchSimulation, computed in your browserRelated: Before trusting a backtest · Helix

When is a backtest luck?

QuestionTry enough strategies and one of them will look brilliant. How good does the best of N backtests have to be before it is evidence rather than selection?

Model

Each strategy earns Gaussian daily returns with zero mean and 1% volatility over T trading days. One strategy can be given a real edge. The winner's Sharpe ratio is compared with the best you should expect from N tries of pure noise:

SR^i=rˉisi,i=1,…,N\widehat{SR}_i = \frac{\bar r_i}{s_i}, \qquad i = 1,\dots,N
SR0=E[max⁡iSR^i]≈V [(1−γ) Φ−1 ⁣(1−1N)+γ Φ−1 ⁣(1−1Ne)]\begin{aligned} SR_0 &= \mathbb{E}\Big[\max_i \widehat{SR}_i\Big] \\ &\approx \sqrt{V}\,\Big[(1-\gamma)\,\Phi^{-1}\!\Big(1-\frac{1}{N}\Big) + \gamma\,\Phi^{-1}\!\Big(1-\frac{1}{Ne}\Big)\Big] \end{aligned}

Here V is the variance of the N Sharpe estimates and γ ≈ 0.577 is the Euler–Mascheroni constant (Bailey & López de Prado, 2014). The deflated Sharpe ratio is the probabilistic Sharpe ratio of the winner measured against SR₀ instead of zero, allowing for the skewness γ̂₃ and kurtosis γ̂₄ of its returns:

DSR=Φ ⁣((SR^∗−SR0)T−11−γ^3 SR^∗+γ^4−14 SR^∗2)\mathrm{DSR} = \Phi\!\left(\frac{(\widehat{SR}^{*}-SR_0)\sqrt{T-1}}{\sqrt{1-\hat\gamma_3\,\widehat{SR}^{*}+\frac{\hat\gamma_4-1}{4}\,\widehat{SR}^{*2}}}\right)

Helix applies this test as a gate; its combined book scored a deflated Sharpe ratio of 0.21 over four trials and failed.

TryAt N = 1 the two tests agree. Raise N: the winner’s Sharpe ratio rises with it while its DSR does not. Then press Repeat to count how often each test would have called pure noise significant, and plant a real edge of 1.0 to see how often noise still beats it.

Left: up to 60 of the N equity curves, the winner in clay. Right: every strategy's annualised Sharpe ratio, the dashed density it would have with no skill, the expected maximum SR₀ and the winner.
200
3 years
none
Readout
SimulatedTheory or threshold
Winner's Sharpe ratio, annualised—SR₀ = —
Average winner over 2,000 repeatsrun Repeatthe formula above
PSR against zero—significant if > 0.95
DSR against SR₀—significant if > 0.95
Noise called significant, PSR · DSRrun Repeatshare of 2,000 repeats
The real edge won the selectionno real edgeshare of 2,000 repeats

Reproduce in Pythonoutput/lab/exp01_overfitting.py33 lines · numpy, scipy
"""EXP-01 · Backtest overfitting: the best of N strategies with no edge, and its deflated Sharpe ratio."""
import numpy as np
from scipy import stats

rng = np.random.default_rng(7)
N, years, vol = 200, 3, 0.01            # strategies tried, sample length, daily volatility
T = 252 * years
R = rng.normal(0.0, vol, size=(N, T))   # daily returns; every true Sharpe ratio is zero

sr = R.mean(1) / R.std(1, ddof=1)       # per-day Sharpe ratio of each strategy
best = int(sr.argmax())

# Expected maximum of N Sharpe estimates under H0 (Bailey & López de Prado, 2014)
g = np.euler_gamma
V = sr.var(ddof=1)
sr0 = np.sqrt(V) * ((1 - g) * stats.norm.ppf(1 - 1 / N) + g * stats.norm.ppf(1 - 1 / (N * np.e)))

def psr(sr_hat, sr_star, r):
    """Probabilistic Sharpe ratio: P(true SR > sr_star), allowing for skew and fat tails."""
    g3, g4 = stats.skew(r), stats.kurtosis(r, fisher=False)
    z = (sr_hat - sr_star) * np.sqrt(len(r) - 1) / np.sqrt(1 - g3 * sr_hat + (g4 - 1) / 4 * sr_hat**2)
    return stats.norm.cdf(z)

ann = np.sqrt(252)
print(f"best annualised SR   {sr[best] * ann:5.2f}")
print(f"E[max SR] under H0   {sr0 * ann:5.2f}")
print(f"PSR vs 0 (naive)     {psr(sr[best], 0, R[best]):5.3f}")
print(f"DSR (deflated)       {psr(sr[best], sr0, R[best]):5.3f}")

# Monte Carlo check of E[max]: under H0 each SR estimate is close to N(0, 1/T)
mx = rng.normal(0, 1 / np.sqrt(T), size=(5000, N)).max(1)
theory = (1 / np.sqrt(T)) * ((1 - g) * stats.norm.ppf(1 - 1 / N) + g * stats.norm.ppf(1 - 1 / (N * np.e)))
print(f"E[max SR]: simulated {mx.mean() * ann:.3f} vs formula {theory * ann:.3f}")

The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

EXP-02

Financial MLSimulation, computed in your browserRelated: Kronos evaluation · Forecasting note

Does the forecast know its own uncertainty?

QuestionA model that forecasts by sampling paths makes a claim about its spread as well as its centre. Do the outcomes land where the paths said they would?

Model

There are 2,000 cases, yt = μt + εt with μt ~ N(0, 0.25²) and εt ~ N(0, 1). The forecaster knows μt and draws K paths xt(k) ~ N(μt + b, s²). It is calibrated when s = 1 and b = 0. Three checks follow. The first is the outcome's rank among its own paths, uniform when the forecaster is calibrated:

rt=#{k:xt(k)<yt}  ∼  Unif{0,…,K}r_t = \#\{k : x_t^{(k)} < y_t\} \;\sim\; \mathrm{Unif}\{0,\dots,K\}

The second is the continuous ranked probability score, a proper score that rewards sharpness only when it comes with calibration; with K paths it is estimated from the two expectations on the right:

CRPS(F,y)=∫−∞∞(F(z)−1{y≤z})2 dz=E∣X−y∣−12 E∣X−X′∣\begin{aligned} \mathrm{CRPS}(F, y) &= \int_{-\infty}^{\infty}\big(F(z) - \mathbb{1}\{y \le z\}\big)^2\,dz \\ &= \mathbb{E}|X - y| - \tfrac12\,\mathbb{E}|X - X'| \end{aligned}

The third is how well the average of K paths ranks the cases. Averaging leaves Monte Carlo noise of variance s²/K:

corr⁡(xˉt,μt)=σμσμ2+s2/K\operatorname{corr}\big(\bar x_t, \mu_t\big) = \frac{\sigma_\mu}{\sqrt{\sigma_\mu^2 + s^2/K}}

Kronos forecasts this way. In the CSI 300 evaluation, Kronos-small returned +1.93% a year over the index with 10 averaged paths and +6.01% with 5, so the number of paths is part of the result.

TryPress Too narrow and the rank histogram turns into a U: outcomes keep falling outside every path. Shifted tilts it. Then move K between 5 and 10, the two Kronos settings, and watch the ranking correlation.

Left: where each outcome ranks among its K paths; bars outside the ±2 SE band of the uniform are in clay. Right: the first 40 cases, each with its 80% interval, its paths as ticks and the outcome as a dot.
0.70
0.00
10
Readout
CheckSimulatedTheory
Rank histogram, χ² and p——
80% interval coverage——
CRPS of the K paths, lower is better——
Rank correlation of the mean path with μ——

Reproduce in Pythonoutput/lab/exp02_calibration.py34 lines · numpy, scipy
"""EXP-02 · Forecast calibration from K sample paths: rank histogram, interval coverage, CRPS."""
import numpy as np
from scipy import stats

rng = np.random.default_rng(11)
T, K = 2000, 10                 # forecast cases, sample paths per forecast
sig_mu = 0.25                   # spread of the predictable part across cases
spread, bias = 0.7, 0.0         # forecaster's scale and shift; (1, 0) is calibrated

mu = rng.normal(0, sig_mu, T)                       # what is knowable
y = mu + rng.normal(0, 1, T)                        # what happens
X = mu[:, None] + bias + spread * rng.normal(0, 1, (T, K))   # K sample paths per case

# 1. Rank histogram: where y falls among its K samples; uniform on 0..K if calibrated
rank = (X < y[:, None]).sum(1)
counts = np.bincount(rank, minlength=K + 1)
chi2 = ((counts - T / (K + 1)) ** 2 / (T / (K + 1))).sum()
print(f"rank histogram chi2 = {chi2:.1f}, p = {stats.chi2.sf(chi2, K):.3g}")

# 2. Central 80% interval of the forecaster's distribution, and its true coverage
z = stats.norm.ppf(0.9)
lo, hi = mu + bias - z * spread, mu + bias + z * spread
cover = ((y >= lo) & (y <= hi)).mean()
theory = stats.norm.cdf(z * spread - bias) - stats.norm.cdf(-z * spread - bias)
print(f"80% interval covers {cover:.3f} (theory {theory:.3f})")

# 3. CRPS from the ensemble: mean|X - y| - mean|X - X'| / 2   (lower is better)
crps = np.abs(X - y[:, None]).mean(1) - np.abs(X[:, :, None] - X[:, None, :]).mean((1, 2)) / 2
print(f"CRPS {crps.mean():.3f} vs perfect forecast {1 / np.sqrt(np.pi):.3f}")

# 4. Ranking by the mean of K paths: Monte Carlo noise dilutes the signal
rho = stats.spearmanr(X.mean(1), mu).statistic
r = sig_mu / np.sqrt(sig_mu**2 + spread**2 / K)
print(f"Spearman(mean of K, mu) = {rho:.3f} (theory {6 / np.pi * np.arcsin(r / 2):.3f})")

The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

EXP-03

Causal inferenceSimulation, computed in your browserRelated: A mediation question needs a timeline · Negative controls

What should you adjust for?

QuestionAdding a control variable can remove bias or create it. Which variables belong in the regression depends on the causal graph, and the data alone cannot say which graph is true.

Model

Five variables follow a linear structural equation model with standard normal errors:

Vj=∑i∈pa(j)βij Vi+εj,εj∼N(0,1)V_j = \sum_{i \in \mathrm{pa}(j)} \beta_{ij}\,V_i + \varepsilon_j, \qquad \varepsilon_j \sim \mathcal{N}(0, 1)

with βZX = βZY = 0.8, βXM = 0.7, βMY = 0.6, βXY = 0.4 and βXC = βYC = 0.7, on n = 2,000 draws. The target is the total effect of X on Y, the sum over causal paths of the products of their coefficients:

τ=∂ E[Y∣do(X=x)]∂x=∑causal paths p ∏(i→j)∈pβij\tau = \frac{\partial\,\mathbb{E}[Y \mid do(X = x)]}{\partial x} = \sum_{\text{causal paths } p}\ \prod_{(i \to j) \in p} \beta_{ij}

A set S identifies it when S holds no descendant of a node on a causal path and blocks every non-causal path (Shpitser, VanderWeele & Robins, 2010); for such a set the adjustment formula holds, and in a linear model the coefficient on X in Y ~ X + S is τ:

E[Y∣do(x)]=ES[ E[Y∣x,S] ]\mathbb{E}[Y \mid do(x)] = \mathbb{E}_S\big[\,\mathbb{E}[Y \mid x, S]\,\big]

TryAdjust for C and an effect appears to shrink; it is the collider opening X → C ← Y. Then try M alone: it lands near the truth, because confounding and over-control happen to cancel. The criterion still calls it biased, and switching off Z → Y shows why.

Causal graph of Z, X, M, Y and C. Use the buttons below to switch arrows and choose what to adjust for.ZconfounderXtreatmentMmediatorYoutcomeCcollider
Click an arrow to remove or restore it and a variable to adjust for it, or use the buttons. Open causal paths are drawn in teal, open biasing paths in clay; an adjusted variable is ringed.
Arrows
Adjust for
Paths from X to Y
    Readout
    Value
    True total effect τ—
    OLS estimate ± 95% CI—
    Bias—

    Every adjustment set, same draws
    Adjust forEstimate ± 95% CIBiasCriterion
    Reproduce in Pythonoutput/lab/exp03_dag.py28 lines · numpy, scipy
    """EXP-03 · What to adjust for: OLS of Y on X under every adjustment set, in a linear SEM."""
    import itertools
    import numpy as np
    
    rng = np.random.default_rng(3)
    n = 2000
    coef = {('Z', 'X'): 0.8, ('Z', 'Y'): 0.8, ('X', 'M'): 0.7, ('M', 'Y'): 0.6,
            ('X', 'Y'): 0.4, ('X', 'C'): 0.7, ('Y', 'C'): 0.7}
    
    v = {}
    for node in ['Z', 'X', 'M', 'Y', 'C']:            # topological order
        v[node] = rng.normal(size=n) + sum(b * v[a] for (a, c), b in coef.items() if c == node)
    
    truth = coef[('X', 'Y')] + coef[('X', 'M')] * coef[('M', 'Y')]   # sum over causal paths
    
    def ols_x(adjust):
        """Coefficient on X, with a classical 95% interval, from Y ~ 1 + X + adjust."""
        A = np.column_stack([np.ones(n), v['X']] + [v[s] for s in adjust])
        beta, *_ = np.linalg.lstsq(A, v['Y'], rcond=None)
        resid = v['Y'] - A @ beta
        se = np.sqrt(resid @ resid / (n - A.shape[1]) * np.linalg.inv(A.T @ A)[1, 1])
        return beta[1], 1.96 * se
    
    print(f"true total effect of X on Y = {truth:.2f}")
    for k in range(4):
        for S in itertools.combinations(['Z', 'M', 'C'], k):
            b, h = ols_x(S)
            print(f"adjust {{{', '.join(S)}}}".ljust(18), f"{b:+.3f} ± {h:.3f}")
    

    The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

    EXP-04

    Causal inferenceSimulation, computed in your browserRelated: CFPS mediation research

    Between people, or within a person?

    QuestionPeople in deprivation are more often depressed. That association across people does not say what happens to one person when their deprivation changes. A panel can answer both questions, but only with different estimators.

    Model

    Each of 1,000 people is followed for 6 waves. Their observed deprivation D and depression Y are stable personal levels κ plus movements around them:

    Dit=κiD+dit,Yit=κiY+yit,corr⁡(κD,κY)=ρD_{it} = \kappa^{D}_i + d_{it}, \qquad Y_{it} = \kappa^{Y}_i + y_{it}, \qquad \operatorname{corr}(\kappa^{D}, \kappa^{Y}) = \rho

    The movements follow a cross-lagged process with stability a = 0.4, where c is the within-person effect of last wave's deprivation on this wave's depression:

    yit=a yi,t−1+c di,t−1+eity_{it} = a\,y_{i,t-1} + c\,d_{i,t-1} + e_{it}

    Pooled regressions mix the two levels. First differences remove κ, and levels two waves back are valid instruments for the lagged differences (Anderson & Hsiao, 1981). The equation below is estimated with Yi,t−2 and Di,t−2 as instruments:

    ΔYit=a ΔYi,t−1+c ΔDi,t−1+Δeit\Delta Y_{it} = a\,\Delta Y_{i,t-1} + c\,\Delta D_{i,t-1} + \Delta e_{it}

    The CFPS project separates the levels with a random-intercept cross-lagged panel model; this simpler estimator stands in for it because it fits in a browser. Intervals are clustered by person.

    TryKeep c at 0 and ρ at 0.6: the between-person slope is clearly positive and the cross-lagged regression still reports an effect, while the differenced estimator finds none. Set ρ to 0 and c to 0.2 and the cross-lagged regression halves a real effect instead.

    Left: one dot per person, their average D and Y over six waves. Right: three estimators with 95% intervals against the true within-person effect c.
    +0.60
    0.00
    1.0
    Readout · true c = 0.00
    EstimatorAnswersEstimate ± 95% CIBias against c
    Between-person slopeDo people with more D have more Y?——
    Cross-lagged OLSBoth questions at once——
    Anderson–Hsiao IVDoes a person's Y follow their own D?——

    Reproduce in Pythonoutput/lab/exp04_panel.py34 lines · numpy, scipy
    """EXP-04 · Two estimands in panel data: between-person association vs within-person effect."""
    import numpy as np
    
    rng = np.random.default_rng(5)
    N, W, burn = 1000, 6, 30
    rho, c_dy, a = 0.6, 0.0, 0.4    # trait correlation, true within-person lag effect D -> Y, stability
    
    # Stable traits: people differ in their typical deprivation and depression
    k = rng.multivariate_normal([0, 0], [[1, rho], [rho, 1]], size=N)
    d = np.zeros(N); y = np.zeros(N); D = []; Y = []
    for t in range(burn + W):                            # within-person dynamics around each trait
        d, y = a * d + rng.normal(size=N), a * y + c_dy * d + rng.normal(size=N)
        if t >= burn:
            D.append(k[:, 0] + d); Y.append(k[:, 1] + y)
    D, Y = np.array(D).T, np.array(Y).T                  # N x W observed panels
    
    # 1. Between-person: regress person means on person means
    between = np.polyfit(D.mean(1), Y.mean(1), 1)[0]
    
    # 2. Cross-lagged panel regression, pooled over waves: Y_t ~ 1 + Y_{t-1} + D_{t-1}
    A = np.column_stack([np.ones(N * (W - 1)), Y[:, :-1].ravel(), D[:, :-1].ravel()])
    clpm = np.linalg.lstsq(A, Y[:, 1:].ravel(), rcond=None)[0][2]
    
    # 3. Anderson-Hsiao: first differences remove traits; instrument with levels at t-2
    dY, dD = np.diff(Y, axis=1), np.diff(D, axis=1)
    yv = dY[:, 1:].ravel()                                          # dY_t, t = 2..W-1
    Xm = np.column_stack([dY[:, :-1].ravel(), dD[:, :-1].ravel()])  # dY_{t-1}, dD_{t-1}
    Zm = np.column_stack([Y[:, :-2].ravel(), D[:, :-2].ravel()])    # Y_{t-2}, D_{t-2}
    ah = np.linalg.solve(Zm.T @ Xm, Zm.T @ yv)[1]
    
    print(f"true within-person effect  {c_dy:+.3f}")
    print(f"between-person slope       {between:+.3f}")
    print(f"cross-lagged OLS           {clpm:+.3f}")
    print(f"Anderson-Hsiao IV          {ah:+.3f}")
    

    The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

    Illustrations

    Physics, chaos and sampling, drawn live.

    Four ideas worth watching: a magnet at its critical point, the limit chaos puts on any forecast, how a diffusion model draws a sample, and a trial told in sixty seconds. Each runs the real model and sets it beside the exact result.

    Illustration 01 · statistical physics

    A magnet at its critical point

    The Ising model: 25,600 spins, each up or down, on a torus. Neighbours that agree lower the energy, and temperature decides how much disorder the system tolerates. A Metropolis sampler draws configurations from the Boltzmann distribution:

    H(s)=−J∑⟨ij⟩sisj,P(s)∝e−H(s)/TH(s) = -J\sum_{\langle ij \rangle} s_i s_j, \qquad P(s) \propto e^{-H(s)/T}

    Below Onsager's critical temperature, one colour takes over; above it, the lattice boils. At Tc islands of every size appear at once, which is what a phase transition looks like from inside. The inset draws the exact magnetisation, and the dot is this lattice's running average:

    ∣m∣(T)=(1−sinh⁡−42JT)1/8(T<Tc)Tc=2Jln⁡(1+2)≈2.269\begin{gathered} |m|(T) = \Big(1 - \sinh^{-4}\tfrac{2J}{T}\Big)^{1/8} \quad (T < T_c) \\ T_c = \frac{2J}{\ln(1+\sqrt2)} \approx 2.269 \end{gathered}

    Drag across the lattice to paint spins and watch the temperature erase or keep your marks.

    2.27
    Readout
    SimulatedExact
    Temperature—Tc = 2.269
    Magnetisation |m|——
    Energy per spin——
    Sweeps of the lattice—near Tc the chain mixes slowly
    Reproduce in Pythonoutput/lab/ill_ising.py30 lines · numpy, scipy
    """Ising model: checkerboard Metropolis on an L x L torus, against Onsager's exact results."""
    import numpy as np
    from scipy.special import ellipk
    
    rng = np.random.default_rng(1)
    L, T = 64, 2.0
    Tc = 2 / np.log(1 + np.sqrt(2))
    s = rng.choice([-1, 1], size=(L, L))
    checker = (np.indices((L, L)).sum(0) % 2).astype(bool)
    
    def neighbours(s):
        return np.roll(s, 1, 0) + np.roll(s, -1, 0) + np.roll(s, 1, 1) + np.roll(s, -1, 1)
    
    ms, es = [], []
    for sweep in range(3000):
        for mask in (checker, ~checker):                 # update one colour of the board at a time
            dE = 2 * s * neighbours(s)
            flip = mask & (rng.random((L, L)) < np.exp(-dE / T))
            s[flip] *= -1
        if sweep >= 1000:
            ms.append(abs(s.mean()))
            es.append(-(s * (np.roll(s, 1, 0) + np.roll(s, 1, 1))).mean())
    
    b2 = 2 / T
    k = 2 * np.sinh(b2) / np.cosh(b2) ** 2
    m_exact = (1 - np.sinh(b2) ** -4) ** (1 / 8) if T < Tc else 0.0
    e_exact = -1 / np.tanh(b2) * (1 + 2 / np.pi * (2 * np.tanh(b2) ** 2 - 1) * ellipk(k ** 2))   # scipy takes m = k^2
    print(f"T = {T}, Tc = {Tc:.4f}")
    print(f"|m|: simulated {np.mean(ms):.3f}, Onsager {m_exact:.3f}")
    print(f"energy per spin: simulated {np.mean(es):.3f}, exact {e_exact:.3f}")
    

    The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

    Illustration 02 · chaos and forecasting

    How far ahead can anyone see?

    Lorenz's 1963 model of convection, three equations that never settle and never repeat:

    x˙=σ(y−x)y˙=x(ρ−z)−yz˙=xy−βz\begin{aligned} \dot x &= \sigma(y - x) \\ \dot y &= x(\rho - z) - y \\ \dot z &= xy - \beta z \end{aligned}

    with σ = 10, β = 8/3 and ρ = 28. Six hundred forecasts start within ε of the truth and are integrated exactly as it is. The error grows exponentially at the leading Lyapunov exponent until the ensemble covers the whole butterfly:

    ∥δ(t)∥≈ε eλ1t,λ1≈0.906⇒  thorizon≈1λ1ln⁡Δε\begin{gathered} \lVert \delta(t) \rVert \approx \varepsilon\, e^{\lambda_1 t}, \qquad \lambda_1 \approx 0.906 \\ \Rightarrow\; t_{\text{horizon}} \approx \frac{1}{\lambda_1}\ln\frac{\Delta}{\varepsilon} \end{gathered}

    That is why better measurement buys so little: each tenfold cut in ε adds only ln 10 / 0.906 ≈ 2.5 time units. The same arithmetic sits behind weather forecasts, and it is why a probabilistic forecast reports a spread instead of a single path.

    1e−5
    1×
    Readout
    SimulatedTheory
    Initial error ε——
    Growth rate of the spread——
    Time until the spread reaches 1——
    Spread now—saturates near the attractor's size
    Reproduce in Pythonoutput/lab/ill_lorenz.py30 lines · numpy, scipy
    """Lorenz ensemble: how fast does an initial error of size eps grow?"""
    import numpy as np
    
    sigma, beta, rho = 10.0, 8 / 3, 28.0
    def f(s):
        x, y, z = s[..., 0], s[..., 1], s[..., 2]
        return np.stack([sigma * (y - x), x * (rho - z) - y, x * y - beta * z], -1)
    
    def rk4(s, h):
        k1 = f(s); k2 = f(s + h / 2 * k1); k3 = f(s + h / 2 * k2); k4 = f(s + h * k3)
        return s + h / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
    
    rng = np.random.default_rng(7)
    truth = np.array([1.0, 1.0, 20.0])
    for _ in range(4000):
        truth = rk4(truth, 0.01)                         # start on the attractor
    
    eps, M, dt = 1e-5, 600, 0.004
    X = truth + eps * rng.normal(size=(M, 3))
    t, ts, logs = 0.0, [], []
    while t < 24:
        truth, X, t = rk4(truth, dt), rk4(X, dt), t + dt
        ts.append(t); logs.append(np.log(np.sqrt(((X - truth) ** 2).sum(1).mean())))
    ts, logs = np.array(ts), np.array(logs)
    
    ok = (logs > np.log(20 * eps)) & (logs < np.log(0.5))   # exponential phase, before saturation
    rate = np.polyfit(ts[ok], logs[ok], 1)[0]
    horizon = ts[np.argmax(logs > 0)]
    print(f"fitted growth rate {rate:.3f}  (leading Lyapunov exponent 0.906)")
    print(f"spread reaches 1 at t = {horizon:.1f}  (ln(1/eps)/lambda = {np.log(1 / eps) / 0.9056:.1f})")
    

    The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

    Illustration 03 · generative models

    Drawing samples by following the score

    Score-based diffusion models generate images by Langevin dynamics: each sample takes a small step up the gradient of the log density and receives a little noise. Here the density is a known mixture of Gaussians, so the score is exact:

    xk+1=xk+α ∇xlog⁡pσ(xk)+2α  ξkξk∼N(0,I)\begin{gathered} x_{k+1} = x_k + \alpha\, \nabla_x \log p_\sigma(x_k) + \sqrt{2\alpha}\; \xi_k \\ \xi_k \sim \mathcal N(0, I) \end{gathered}

    Plain Langevin finds every mode, but particles cannot cross the empty space between distant modes, so each mode keeps whatever share it caught at the start. Annealed Langevin first follows the score of a blurred target, whose components overlap, and lowers the blur σ from 4 to 0:

    pσ=p∗N(0,σ2I)=∑kwk N(μk,(s2+σ2)I)\begin{aligned} p_\sigma &= p * \mathcal N(0, \sigma^2 I) \\ &= \textstyle\sum_k w_k\, \mathcal N\big(\mu_k, (s^2 + \sigma^2) I\big) \end{aligned}

    Run both on the 80 : 20 pair and compare the bars with the clay marks for the true weights.

    0.08
    Readout
    SimulatedTarget
    Total variation, ½Σ|share − weight|—0 when the shares match
    Share in mode 1 · its weight——
    Noise level σ—4 → 0 when annealed
    Langevin steps——
    Reproduce in Pythonoutput/lab/ill_langevin.py25 lines · numpy, scipy
    """Plain vs annealed Langevin dynamics on an 80:20 mixture of two Gaussians."""
    import numpy as np
    
    w = np.array([0.8, 0.2]); mu = np.array([[-3.0, 0.0], [3.0, 0.0]]); s = 0.6
    
    def score(x, sig):
        """grad log of the mixture blurred by N(0, sig^2 I): each component widens to s^2 + sig^2."""
        v = s**2 + sig**2
        logr = np.log(w) - ((x[:, None, :] - mu) ** 2).sum(-1) / (2 * v)
        r = np.exp(logr - logr.max(1, keepdims=True)); r /= r.sum(1, keepdims=True)
        return (r[:, :, None] * (mu - x[:, None, :])).sum(1) / v
    
    def run(annealed, c=0.08, N=1500, levels=12, per=45, tail=300, seed=0):
        rng = np.random.default_rng(seed)
        x = rng.uniform([-6.5, -3.6], [6.5, 3.6], size=(N, 2))
        sigmas = np.geomspace(4, 0.02, levels) if annealed else []
        for sig in [*np.repeat(sigmas, per), *np.zeros(tail)]:
            a = c * (s**2 + sig**2)
            x = x + a * score(x, sig) + np.sqrt(2 * a) * rng.normal(size=x.shape)
        share = np.bincount(((x - mu[:, None]) ** 2).sum(-1).argmin(0), minlength=2) / N
        return share, 0.5 * np.abs(share - w).sum()
    
    for annealed in (False, True):
        share, tv = run(annealed)
        print(f"{'annealed' if annealed else 'plain   '}  shares {share.round(3)}  vs weights {w}  TV = {tv:.3f}")
    

    The same model as the page, with its own random draws, so the numbers agree in distribution rather than digit for digit.

    Illustration 04 · causal inference in a minute

    Did the pill work?

    A simulated trial of 400 people, told as one story: an anecdote, a crowd that recovers anyway, coin-flip randomization, a permutation test and an effect with its interval. Every number is computed in your browser.

    01 · The question

    She took the pill and got better.

    Did the pill work? One story can’t tell you.

    Read the script
    1. The questionShe took the pill and got better. Did the pill work? One story can’t tell you.
    2. ChanceMany people get better without any pill. In a crowd of 400, luck alone writes stories like hers.
    3. RandomizationSo flip a coin for each person: pill or placebo. Chance now builds two groups that are alike in every way, even in ways no one measured.
    4. The comparisonThen count who recovers in each group. Both groups are treated the same in every other way, so the pill is the only systematic difference.
    5. Could it be luck?Could chance alone open a gap this wide? Shuffle the labels 1,000 times, as if the pill did nothing, and count how often it happens.
    6. The size of the effectNot just yes or no: how much, and how sure. About 15 more recoveries per 100 people, with a 95% confidence interval.
    7. EvidenceOne story couldn’t tell us. Four hundred coin flips could. More people, a narrower cone: uncertainty, drawn to scale.