mr.piron.Back to the research desk
AI & MACHINE LEARNING

Validation is a design decision, not a final checkbox.

Reproducible experiments, leakage checks, and clear baselines for AI-assisted trading research.

TECHNICAL RESEARCHAdvanced3 primary sources248 words
MR. PIRON / RESEARCH DESK2 min read
0102123ARTICLE, IN READING ORDER →↑ NUMBERED SOURCES0102123ARTICLE, IN READING ORDER →↑ NUMBERED SOURCES0102123ARTICLE, IN READING ORDER →↑ SOURCES
FIG. 0 EVIDENCE MAP · 2 SECTIONS · 2/3 SOURCES CITED · 4 CITATIONS · 1 EQUATION · 1 CODE LISTINGComputed from this article. Each bar is a section, as long as its share of the words. A curve joins a passage to a source it cites; a circle grows with the number of passages that cite it. Diamonds mark equations and squares mark code. 1 listed source is not cited in the text (dashed).
Sources cited, equations and code listings per section
SectionSources citedEquationsCode listings
1. Encode bitemporal availability and label-interval purging in the data schema1, 210
2. Verify zero temporal overlap before fitting preprocessing or models1, 201

01 /Encode bitemporal availability and label-interval purging in the data schema

A quantitative research pipeline must define its falsification protocol before celebrating a backtest curve. Data ingestion, feature normalization, target construction, and portfolio sizing all require explicit temporal boundaries that mirror live decision times.

Chronological splits prevent a model from training on future regimes, but index-based splitting alone is insufficient when forward return labels span multiple bars. Scikit-learn’s TimeSeriesSplit enforces ordered folds with an optional gap [1], and we extend that boundary by purging any training observation whose forward label interval [ti,start,ti,end][t_{i,\mathrm{start}}, t_{i,\mathrm{end}}] overlaps the validation window.

AI-assisted research increases the number of candidate architectures evaluated [2]. That makes immutable experiment manifests, bitemporal point-in-time joins, and strict separation between validation selection and the final holdout essential.

Bitemporal feature admissibility and purged training setEQ. 1.1
Dtrain(tval)={i:  ti,avail≤ti,dec<tval−e    ∧    ti,end<tval−e}\mathcal D_{\mathrm{train}}(t_{\mathrm{val}})=\left\{i:\;t_{i,\mathrm{avail}}\le t_{i,\mathrm{dec}}<t_{\mathrm{val}}-e\;\;\wedge\;\;t_{i,\mathrm{end}}<t_{\mathrm{val}}-e\right\}

Every training record i must be published prior to its decision timestamp t_{i,dec} and its forward return label must terminate before the validation start minus embargo e.

02 /Verify zero temporal overlap before fitting preprocessing or models

Every rolling z-score, covariance matrix, and hyperparameter choice must be fitted strictly inside Dtrain(tval)\mathcal D_{\mathrm{train}}(t_{\mathrm{val}}) and frozen prior to scoring validation observations [1, 2]. Full-sample standardization before splitting leaks future volatility into early position sizes.

A complete deliverable pairs versioned code and bitemporal data schemas with simple baseline counterfactuals, cost stress tests, and forward paper-trading reconciliation. The verification snippet below checks zero label overlap for one synthetic chronological split with a pre-validation gap.

  • Enforce t_available <= t_decision on every fundamental, alternative, and price feature before model ingestion.
  • Purge training observations whose multi-period return labels overlap the validation start timestamp.
  • Fit feature scalers and covariance estimators strictly inside each training fold to prevent lookahead normalization.
python13 LINESILLUSTRATIVE · NOT EXECUTED
# Illustrative only; verifies zero label overlap in a purged chronological split.
import numpy as np

decision_times = np.arange(0, 100)
label_horizon = 5
label_ends = decision_times + label_horizon
val_start, embargo = 60, 3

train_mask = (decision_times < val_start - embargo) & (label_ends < val_start - embargo)
val_mask = decision_times >= val_start

assert np.max(label_ends[train_mask]) < np.min(decision_times[val_mask])
print(f"train_count={int(np.sum(train_mask))}, max_train_label_end={int(np.max(label_ends[train_mask]))}, val_start={val_start}")
∎

Research is complete only when the assumptions, experiment, and limitations can be reconstructed.

This perspective is research commentary. It does not constitute investment advice or evidence of live strategy performance. Experimental methods require project-specific testing. Equations and code illustrate research formulations, not measured live returns.

More perspectivesExplore the engineering atlas