mr.piron.Back to the research desk
AI & MACHINE LEARNING

AI can accelerate research. Evidence still decides.

Inside the shift toward AI-assisted factor discovery — and the validation that makes it useful.

TECHNICAL RESEARCHAdvanced3 primary sources304 words
MR. PIRON / RESEARCH DESK2 min read
0102123ARTICLE, IN READING ORDER →↑ NUMBERED SOURCES0102123ARTICLE, IN READING ORDER →↑ NUMBERED SOURCES0102123ARTICLE, IN READING ORDER →↑ SOURCES
FIG. 0 EVIDENCE MAP · 2 SECTIONS · 3/3 SOURCES CITED · 6 CITATIONS · 2 EQUATIONS · 1 CODE LISTINGComputed from this article. Each bar is a section, as long as its share of the words. A curve joins a passage to a source it cites; a circle grows with the number of passages that cite it. Diamonds mark equations and squares mark code.
Sources cited, equations and code listings per section
SectionSources citedEquationsCode listings
1. Accelerate hypothesis throughput without relaxing the acceptance gate1, 220
2. Enforce point-in-time feature lineage and deflated acceptance1, 2, 301

01 /Accelerate hypothesis throughput without relaxing the acceptance gate

AI is becoming part of the quantitative research workflow: proposing hypotheses, generating candidate factors, writing experiments, and helping analysts compare results. Microsoft’s RD-Agent-Quant research explores coordinated factor and model development across iterative agent loops [1]. The engineering opportunity is faster exploration of the hypothesis space, leaving more researcher attention available for structural critique and falsification.

The core risk is mistaking throughput for evidence. Under an illustrative null with independent, equal-variance Gaussian Information Ratio estimates, their expected maximum grows asymptotically with 2log⁡M\sqrt{2\log M} as the number MM of trials increases. Correlated factors and non-Gaussian estimates require a different calibration. A compelling natural-language rationale attached to a factor formula does not alter its sampling distribution.

A 2026 survey of LLM trading-agent studies examines uneven reporting of temporal splits, transaction costs, survivorship controls, and execution assumptions [2]. Without a frozen holdout ledger and explicit search-breadth accounting, agentic discovery pipelines can amplify overfitting.

Extreme-value inflation of maximum in-sample Information RatioEQ. 1.1
E ⁣[max⁡m∈{1,…,M}IR^m  |  IRm=0]≈σIR^[(1−γ)Φ−1 ⁣(1−1M)+γΦ−1 ⁣(1−1Me)]\mathbb E\!\left[\max_{m\in\{1,\dots,M\}}\widehat{\mathrm{IR}}_m\;\middle|\;\mathrm{IR}_m=0\right]\approx \sigma_{\widehat{\mathrm{IR}}}\left[(1-\gamma)\Phi^{-1}\!\left(1-\frac1M\right)+\gamma\Phi^{-1}\!\left(1-\frac1{Me}\right)\right]

This extreme-value approximation assumes M independent Gaussian estimates with common standard deviation σ_IR; γ ≈ 0.5772 is the Euler–Mascheroni constant. It estimates selection inflation, not a significance cutoff or a correction for correlated trials.

Cost-adjusted cross-sectional rank utilityEQ. 1.2
Unet(fm)=1T∑t=1T(wm,t⊤rt+1−κ∥wm,t−wm,t−1∥1)U_{\mathrm{net}}(f_m)=\frac1T\sum_{t=1}^T\left(w_{m,t}^\top r_{t+1}-\kappa\lVert w_{m,t}-w_{m,t-1}\rVert_1\right)

Every candidate factor f_m must be scored after converting scores to bounded weights w_{m,t} and subtracting proportional one-way turnover cost κ.

02 /Enforce point-in-time feature lineage and deflated acceptance

Our design principle connects AI-assisted discovery to an auditable research ledger: record every generated hypothesis, version the exact point-in-time dataset, enforce strict availability timestamps, and increment the trial counter MM on every backtest execution [1, 3]. Deployment follows out-of-sample survival rather than narrative plausibility.

Before any candidate factor enters a model ensemble, we evaluate its incremental contribution orthogonal to existing baseline factors and penalize high-turnover signals whose gross rank IC disappears under realistic spread and impact assumptions [2, 3]. The synthetic fixture below illustrates how search breadth increases expected null selection inflation.

  • Log every rejected and accepted agent-generated factor so the effective trial count M reflects full search breadth.
  • Require orthogonalized net utility improvement over linear momentum and value baselines before promoting any nonlinear feature.
  • Freeze prompt templates, parser rules, and universe snapshots prior to touching the chronological holdout partition.
python16 LINESILLUSTRATIVE · NOT EXECUTED
# Illustrative only; Gaussian null selection inflation, not a significance test.
import math
from statistics import NormalDist

def expected_max_z(n_trials: int) -> float:
    assert n_trials >= 2
    gamma = 0.5772156649015329
    norm_ppf = NormalDist().inv_cdf
    z1 = norm_ppf(1.0 - 1.0 / n_trials)
    z2 = norm_ppf(1.0 - 1.0 / (n_trials * math.e))
    return (1.0 - gamma) * z1 + gamma * z2

single_hurdle = expected_max_z(2)
agent_sweep_hurdle = expected_max_z(250)
assert agent_sweep_hurdle > single_hurdle * 2.0
print(f"2-trial hurdle={single_hurdle:.3f}, 250-trial hurdle={agent_sweep_hurdle:.3f}")
∎

Research is complete only when the assumptions, experiment, and limitations can be reconstructed.

This perspective is research commentary. It does not constitute investment advice or evidence of live strategy performance. Experimental methods require project-specific testing. Equations and code illustrate research formulations, not measured live returns.

More perspectivesExplore the engineering atlas