Key Concepts

Research Engine

Introduction

The Research Environment is a Jupyter notebook-based environment where you can access our data through the QuantBook class instead of through the QCAlgorithm class in a backtest. The environment supports both Python and C#. If you use Python, you can import code from the code files in your project into the Research Environment to aid development.

Before you run backtests, we recommend testing your hypothesis in the Research Environment. It's easier to perform data analysis and produce plots in the Research Environment than in a backtest. In the Research Environment, you can use the QuantConnect API to import your backtest results for further analysis.

Before backtesting or live trading with machine learning models, you may find it beneficial to train them in the Research Environment, save them in the ObjectStore, and then load them from the ObjectStore into the backtesting and live trading environment

Batch vs Stream Analysis

The backtesting environment is an event-based simulation of the market. Backtests aim to provide an accurate representation of whether a strategy would have performed well in the past, but they are generally slow and aren't the most efficient way to test the foundational ideas behind strategies. You should only use backtests to verify an idea after you have already tested it with statistical analysis.

The Research Environment lets you build a strategy by starting with a central hypothesis about the market. For example, you might hypothesize that an increase in sunshine hours will increase the production of oranges, which will lead to an increase in the supply of oranges and a decrease in the price of Orange Juice Futures. You can attempt to confirm this working hypothesis by analyzing weather data, production of oranges data, and the price of Orange Juice futures. If the hypothesis is confirmed with a degree of statistical significance, you can be confident in the hypothesis and translate it into an algorithm you can backtest.

Jupyter Notebooks

Jupyter notebooks support interactive data science and scientific computing across various programming languages. We carry on that philosophy by providing an environment for you to perform exploratory research and brainstorm new ideas for algorithms. A Jupyter notebook installed in QuantConnect allows you to directly explore the massive amounts of data that is available in the Dataset Market and analyze it with python or C# commands. We call this exploratory notebook environment the Research Environment.

Open Notebooks

To open a notebook, open one of the .ipynb files in your cloud projects or see Running Local Research Environment.

Execute Code

The notebook allows you to run code in a safe and disposable environment. It's composed of independent cells where you can write, edit, and execute code. The notebooks support Python, C#, and Markdown code.

Keyboard Shortcuts

The following table describes some useful keyboard shortcuts:

ShortcutDescription
Shift+EnterRun the selected cell
aInsert a cell above the selected cell
bInsert a cell below the selected cell
xCut the selected cell
vPaste the copied or cut cell
zUndo cell actions

Terminate Research Sessions

If you use the Research Environment in QC Cloud, to terminate a research session, stop the research node in the Resources panel. If you use the local Research Environment, see Managing Kernels and Terminals in the JupyterLab documentation.

Your Research and LEAN

To analyze data in a research notebook, create an instance of the QuantBook class. QuantBook is a wrapper on QCAlgorithm, which means QuantBook allows you to access all the methods available to QCAlgorithm and some additional methods. The following table describes the helper methods of the QuantBook class that aren't available in the QCAlgorithm class:

MethodDescription
GetFundamentalGet fundamental data for some Symbol(s).
GetFutureHistoryGet the expiration, open interest, and price data of the contracts in a Futures chain.
GetOptionHistoryGet the strike, expiration, open interest, option right, and price data of the contracts in an Options chain.
IndicatorGet the values of an indicator for an asset over time.

QuantBook gives you access to the vast amounts of data in the Dataset Market. Similar to backtesting, you can access that data using history calls. You can also create indicators, consolidate data, and access charting features. However, keep in mind that event-driven features available in backtesting, like universe selection and OnData events, are not available in research. After you analyze a dataset in the Research Environment, you can easily transfer the logic to the backtesting environment. For example, consider the following code in the Research Environment:

// Initialize QuantBook
var qb = new QuantBook();

// Subscribe to SPY data with QuantBook
var symbol = qb.AddEquity("SPY").Symbol;

// Make history call with QuantBook
var history = qb.History(symbol, TimeSpan.FromDays(10), Resolution.Daily);
# Initialize QuantBook
qb = QuantBook()

# Subscribe to SPY data with QuantBook
symbol = qb.AddEquity("SPY").Symbol

# Make history call with QuantBook
history = qb.History(symbol, timedelta(days=10), Resolution.Daily)

To use the preceding code in a backtest, replace QuantBook()new QuantBook() with selfthis.

public override void Initialize()
{
    // Set qb to instance of QCAlgorithm
    var qb = this;
    
    // Subscribe to SPY data with QCAlgorithm
    var symbol = qb.AddEquity("SPY").Symbol;
    
    // Make history call with QCAlgorithm
    var history = qb.History(symbol, TimeSpan.FromDays(10), Resolution.Daily);
}
def Initialize(self) -> None:

    # Set qb to instance of QCAlgorithm
    qb = self
    
    # Subscribe to SPY data with QCAlgorithm
    symbol = qb.AddEquity("SPY").Symbol
    
    # Make history call with QCAlgorithm
    history = qb.History(symbol, timedelta(days=10), Resolution.Daily)

Import Project Code

To import code from your code files to your Research Environment session, use Python.

One of the drawbacks of using the Research Environment you may encounter is the need to rewrite code you've already written in a file in the backtesting environment. Instead of rewriting the code, you can import the methods from the backtesting environment into the Research Environment to reduce development time. For example, say you have the following helpers.py file in your project:

def Add(a, b): 
   return a+b

To import the preceding method into your research notebook, use the import statement.

from helpers import Add

# reuse method from helpers.py
Add(3, 4)

If you adjust the file that you import, restart the Research Environment session to import the latest version of the file. To research the Research Environment, stop the research node and then open the notebook again.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: