Assistants

When research arrives ready for review, the Research Validation Assistant takes it directly. The full review happens inside the Research Environment and is documented like a report, walking the next reader through what was tested and what each test revealed.

The Research Validation Assistant runs the model’s residuals through four diagnostic tests, each designed to catch a different way a quant model can quietly be wrong. An augmented Dickey-Fuller test with no constant and no trend checks whether the residuals are stationary, because non-stationary residuals are usually a sign the model has missed something structural in the data. A Breusch-Godfrey test then looks for serial correlation in those same residuals, since autocorrelated errors mean the model is leaving predictable patterns on the table.

From there, a Breusch-Pagan test screens for heteroscedasticity, flagging models whose error variance shifts with the inputs and whose standard errors therefore cannot be trusted. Finally, a population stability index compares the residuals from the training window to the residuals from the testing window, giving an honest read on whether the model behaves the same way on data it has never seen. Each test produces a clear pass, warning, or fail outcome rather than a wall of numbers.

A validation notebook covering stationarity, serial correlation, heteroscedasticity, and population stability, each with the test result and a plain verdict the next Assistant on the chain can act on. By the time the review is finished, the strategy either earns its place in the backtesting queue or gets sent back with a specific reason it is not yet ready, so no engineering time is spent coding a model that was never going to hold up.

The Research Validation Assistant has access to the following tools:

• jupyter_create_cell
• jupyter_read_cell
• jupyter_update_cell
• jupyter_delete_cell
• jupyter_execute_cell
• jupyter_create_notebook
• jupyter_read_notebook
• jupyter_execute_notebook
• financial_data_blog_posts
• financial_data_news_articles
• financial_data_web_get
• datasets
• user_input
• object_store_get
• object_store_set