Algorithm Reference

Consolidating Data

Introduction

Consolidating data allows you to create bars of any length from smaller bars. Commonly, this is used to combine one-minute price bars into longer bars such as 10-20 minute bars. Using the price movement over a longer period can sometimes reduce the noise of the markets to make trading more efficient.

To achieve this QuantConnect allows you to create Consolidator objects and register them for data. Using a consolidator makes creating longer periods easier and reduces the chance of bugs. In the following sections, we will introduce the different types of consolidators and show you how to shape data into any form.

Data Shapes and Sizes

Consolidated output data is in the same format as the input. Small TradeBars are aggregated into large TradeBars, and small QuoteBars are aggregated into larger QuoteBars.

Many asset-types in QuantConnect have data for both trades and quotes. In this case, we default to generating bars for the TradeBar data source. Forex is the exception here which only has data for quotes, and we so default to generating QuoteBars.

Consolidating Periods

The most common request is to create bars based on a period of time. QuantConnect has created a helper for this called Consolidate()self.Consolidate(). The consolidate method looks up the Symbol requested, creates a consolidator for the given period, and passes the output to the provided function event handler. With just one line of code you can create data in any format required. The consolidate helper accepts a timedelta, Resolution, or Calendar period specifier:

// Consolidate 1min SPY -> 45min Bars
Consolidate("SPY", TimeSpan.FromMinutes(45), FortyFiveMinuteBarHandler)

// Consolidate 1min SPY -> 1-Hour Bars
Consolidate("SPY", Resolution.Hour, HourBarHandler)

// Consolidate 1min SPY -> 1-Week Bars
Consolidate("SPY", Calendar.Weekly, WeekBarHandler)
# Consolidate 1min SPY -> 45min Bars
self.Consolidate("SPY", timedelta(minutes=45), self.FortyFiveMinuteBarHandler)

# Consolidate 1min SPY -> 1-Hour Bars
self.Consolidate("SPY", Resolution.Hour, self.HourBarHandler)

# Consolidate 1min SPY -> 1-Week Bars
self.Consolidate("SPY", Calendar.Weekly, self.WeekBarHandler)

The event handler function of Consolidate accepts just one argument, the resulting bar. For most data sources this defaults to TradeBar format. For a full example see the demonstration algorithmdemonstration algorithm.

// Example event handler from Consolidate helper.
void FortyFiveMinuteBarHandler(TradeBar consolidated) {
    Log($"{consolidated.EndTime:o} 45 minute consolidated.");
}
# Example event handler from Consolidate helper.
def FortyFiveMinuteBarHandler(self, consolidated):
      self.Log(f"{consolidated.EndTime} >> FortyFiveMinuteBarHandler >> {consolidated.Close}")

Most Common Error: Adding Braces

The most common error is to put braces "()" at the end of your function call when defining the event handler. Using braces causes the method to be executed, and the result passed in as the event handler. Remember to simply pass the name of your function to the event system. i.e. It should be self.EventHandler not self.EventHandler().

Consolidating Data for Indicators

Consolidated data can easily be used with indicators along the period-resolution boundaries. This is possible with one line of code by the basic indicator API as shown below. Using these helper methods the required consolidators are created and the output bar is automatically used to update the indicator. See the Indicators documentation for more information.

 // Consolidating minute SPY into 14-bar daily indicators
var ema = EMA("SPY", 14, Resolution.Daily);
var sma = SMA("SPY", 14, Resolution.Daily);
 # Consolidating minute SPY into 14-bar daily indicators
ema = self.EMA("SPY", 14, Resolution.Daily)
sma = self.SMA("SPY", 14, Resolution.Daily)

A common request is to use consolidators with indicators to create indicators with exotic data (e.g. 35-minute EMA). To do this you will need to create the indicator and register it to receive updates. This is done with the RegisterIndicator function. Registering the indicator wires it up to get data updates from LEAN automatically.

// Generate 7 minute bars; then SMA-10 generates the average of last 10 bars.
AddEquity("SPY", Resolution.Minute);
var sma = new SimpleMovingAverage(10);
RegisterIndicator("SPY", sma, TimeSpan.FromMinutes(7));
# Generate 7 minute bars; then SMA-10 generates the average of last 10 bars.
self.AddEquity("SPY", Resolution.Minute)
self.sma = SimpleMovingAverage(10)
self.RegisterIndicator("SPY", self.sma, timedelta(minutes=7))

Rolling Window of Consolidated Bars

Manually Consolidating Bar Count

Manually Consolidating Periods

Renko Volume Consolidation

Data Consolidation Events

The result of the consolidation is passed into an event handler. An event handler is a function in your algorithm designed to receive the bar. It can have any name but must have the required parameters. Depending on how you're using the consolidator system you must use one of the method patterns below:

// self.Consolidate() Event Handler
void FortyFiveMinuteBarHandler(TradeBar consolidated) {
}

// Manually Created Event Handler
void ThirtyMinuteBarHandler(object sender, TradeBar consolidated) {
}
# self.Consolidate() Event Handler
def FortyFiveMinuteBarHandler(self, consolidated):
      pass

# Manually Created Event Handler
def ThirtyMinuteBarHandler(self, sender, consolidated):
      pass

Removing a Consolidator

If you manually create a consolidator for a universe subscription you should remember to remove it again later once the security leaves your universe. If you do not "tidy up" these can compound internally causing your algorithm to slow down and eventually die once it runs out of RAM.

You will need to save a reference to the consolidator to remove it cleanly. We recommend using a class to organize all of the symbol-specific objects created over the lifetime of a security in your universe. See this exampleexample Alpha as an example of removing consolidators from universe subscriptions.

// Remove a consolidator instance from subscription manager
algorithm.SubscriptionManager.RemoveConsolidator(symbol, myConsolidator)
# Remove a consolidator instance from subscription manager
algorithm.SubscriptionManager.RemoveConsolidator(self.symbol, self.myConsolidator)

You can also see our Tutorials and Videos. You can also get in touch with us via Chat.

Did you find this page helpful?

Contribute to the documentation: