Algorithm Reference

Historical Data

Introduction

Historical data can be provided in a batch request, or by "warming up" your algorithm. Batch requests feed requested data back as an enumerable you can use to manually prime your algorithm. The warm up feature feeds data to your event handlers from before your start date for a fixed period.

In live mode the history API can return data up to 5 seconds before the present moment.

Historical Data Requests

Batch history API is often easier to understand as you can manually feed data into your components which need it. If Resolution of the data request is not specified the history API uses the resolution of data you've specified with AddSecurity() or your Universe. If the Symbol is not specified then history for all symbols is returned for the period

// Request history for specific symbol, default to AddSecurity resolution.
IEnumerable<TradeBar> bars = History("SPY", TimeSpan.FromDays(7));
// Request history for specific symbol, at specific resolution.
IEnumerable<TradeBar> bars = History("SPY", TimeSpan.FromDays(7), Resolution.Minute);

//Get last 14 bars of SPY, default to AddSecurity resolution.
//Note you can't get "14 ticks" -- getting a specific number of bars only applies to TradeBar data.
IEnumerable<TradeBar> bars = History("SPY", 14);
//Get last 14 bars of SPY, at specific resolution.
IEnumerable<TradeBar> bars = History("SPY", 14, Resolution.Minute);
// Request history for specific symbol, default to AddSecurity resolution.
IEnumerable<QuoteBar> bars = History<QuoteBar>("EURUSD", TimeSpan.FromDays(7));
// Request history for specific symbol, at specific resolution.
IEnumerable<QuoteBar> bars = History<QuoteBar>("EURUSD", TimeSpan.FromDays(7), Resolution.Minute);

//Get last 14 bars of EURUSD, default to AddSecurity resolution.
//Note you can't get "14 ticks" -- getting a specific number of bars only applies to QuoteBar data.
IEnumerable<QuoteBar> bars = History<QuoteBar>("EURUSD", 14);
//Get last 14 bars of EURUSD, at specific resolution.
IEnumerable<QuoteBar> bars = History<QuoteBar>("EURUSD", 14, Resolution.Minute);

// Get history for all tickers we're subscribed to, at their native resolution
IEnumerable<Slice> slices = History(TimeSpan.FromDays(7));
// Get history for all tickers we're subscribed to, at a specific resolution
IEnumerable<Slice> slices = History(TimeSpan.FromDays(7), Resolution.Minute);

// For custom data, we need to define the type. E.g.: Quandl
AddData<Quandl>("SYMBOL")
IEnumerable<Quandl> bars = History<Quandl>("SYMBOL", TimeSpan.FromDays(7));
IEnumerable<Quandl> bars = History<Quandl>("SYMBOL", 14);
# Get history for all tickers we're subscribed to, at their native resolution
slices = self.History(timedelta(7))
# Get history for all tickers we're subscribed to, at a specific resolution
slices = self.History(timedelta(7), Resolution.Minute)

# Equity case:
# Request history for specific symbol, default to AddSecurity resolution.
bars = self.History("SPY", TimeSpan.FromDays(7))
# Request history for specific symbol, at specific resolution.
bars = self.History("SPY", TimeSpan.FromDays(7), Resolution.Minute)

# Get last 14 bars of SPY, default to AddSecurity resolution.
# Note you can't get "14 ticks" -- getting a specific number of bars only applies to TradeBar data.
bars = self.History("SPY", 14)
# Get last 14 bars of SPY, at specific resolution.
bars = self.History("SPY", 14, Resolution.Minute)

# Other assets (QuoteBar) case:
# We need to use map method to select only the specific symbol
bars = map(lambda x: x["EURUSD"], self.History(timedelta(7)))
bars = map(lambda x: x["EURUSD"], self.History(timedelta(7), Resolution.Minute))
bars = map(lambda x: x["EURUSD"], self.History(14))
bars = map(lambda x: x["EURUSD"], self.History(14, Resolution.Minute))

# Get history in pandas.DataFrame format, use list of string
df = self.History(["EURUSD"], timedelta(7))
df = self.History(["EURUSD"], timedelta(7), Resolution.Minute)
df = self.History(["EURUSD"], 14)
df = self.History(["EURUSD"], 14, Resolution.Minute)

The returned data is presented in ascending order, meaning that last element has most recent information. Therefore, if you want to use historical data to warm up an indicator, you should use the list in its original order.

Batch data returned as IEnumerable<Slice> slices, IEnumerable<TradeBar> slices or IEnumerable<QuoteBar> slices has all the normal slice helpers - to assist using the data in your strategy, for example:

In Python batch data can also be returned as pandas.DataFrame. It is a multi-index dataframe where the first index is the symbol where the time column is the EndTime of the bar. It is designed in LEAN to prevent the forward-looking in algorithm trading.

//From slice -> tradebars, or slice -> decimals.
IEnumerable<Slice> slices = History(TimeSpan.FromDays(7), Resolution.Minute);
IEnumerable<TradeBar> bars = slices.Get("SPY"); // Bars
IEnumerable<decimal> decimals = slices.Get("SPY", Field.Close); // Close from slice
// Many math libraries need double arrays, decimals -> double[]
double[] doubleArray = decimals.ToDoubleArray();
// Get the most recent bar
List<TradeBar> list = bars.ToList();
TradeBar mostRecentBar = list.Last();
# Request history for a single asset
eurusd_slices = self.History(["EURUSD"], 5) # multi-index dataframe
# Good practice to check whether the dataframe is empty
if not eurusd_slices.empty:
    eurusd_bars = eurusd_slices.loc["EURUSD"] # single index dataframe
    eurusd_close = eurusd_bars["close"] # use lowercase
    # Get the most recent bar
    most_recent_bar = eurusd_close.tail(1)

# Request history for a list of symbols
symbols_slices = self.History(["AAPL", "SPY", "VXX"], 4) # multi-index dataframe
if not symbols_slices.empty:
    # single index dataframe for SPY
    df_spy = slices.loc["SPY"]
    # A series of the SPY closing price
    spy_close = df_spy['close']
    # single index dataframe with the closing price of all symbols
    all_symbols_close = slices["close"].unstack(level=0)
    # Get the most recent bar of each symbol
    most_recent_bars = all_symbols_close.tail(1)
Data shape results from a History request on Forex:
Data shape results from a History Request on a List of Equity Symbols
Data shape Results from a History Request on the Closing Price:

Warming Your Algorithm

Warming up your algorithm causes data from before the algorithm starts to be fed into the conventional data event handlers. This system of warm up lends itself to stream style algorithms.

The data flowing to your algorithm will seamlessly pass from historical to live data. The flag bool IsWarmingUp can be used to monitor this transition.

During the warm up period you cannot submit orders. Warm up also does not support universe selection while the algorithm is warming up.

SetWarmUp(200); //Warm up 200 bars for all subscribed data.
SetWarmUp(TimeSpan.FromDays(7)); //Warm up 7 days of data.
self.SetWarmUp(200) //Warm up 200 bars for all subscribed data.
self.SetWarmUp(timedelta(7)) //Warm up 7 days of data.

You can also see our Tutorials and Videos. You can also get in touch with us via Chat.

Did you find this page Helpful ?