Consolidating Data

Getting Started

Introduction

Consolidating data allows you to create bars of any length from smaller bars. Consolidation is commonly used to combine one-minute price bars into longer bars such as 10-20 minute bars. Consolidated bars are helpful because price movement over a longer period can sometimes contain less noise and bars of exotic length are less researched than standard bars, so they may present more opportunities to capture alpha.

To consolidate data, create a Consolidator object and register it for data. The built-in consolidators make it easy to create consolidated bars without introducing bugs. In the following sections, we will introduce the different types of consolidators and show you how to shape data into any form.

Basic Usage

The following code snippet demonstrates a simple consolidator that aggregates 1-minute bars into 10-minutes bars:

// Add some class members
private Symbol _symbol;
private TradeBarConsolidator _consolidator;

// In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates
_symbol = AddEquity("SPY", Resolution.Minute).Symbol;
_consolidator = Consolidate(_symbol, TimeSpan.FromMinutes(10), ConsolidationHandler);

// Define the consolidation handler
void ConsolidationHandler(TradeBar consolidatedBar)
{
    
}
# In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates
self.symbol = self.AddEquity("SPY", Resolution.Minute).Symbol
self.consolidator = self.Consolidate(self.symbol, timedelta(minutes=10), self.consolidation_handler)

# Define the consolidation handler
def consolidation_handler(self, consolidated_bar: TradeBar) -> None:
    pass

Data Shapes and Sizes

Consolidators usually produce output data that is the same format as the input data. They aggregate small TradeBar objects into a large TradeBar, small QuoteBar objects into a large QuoteBar, and Tick objects into either a TradeBar or QuoteBar.

Many asset classes in QuantConnect have data for both trades and quotes. By default, consolidators aggregate data into TradeBar objects. Forex data is the only exception, which consolidators aggregate into QuoteBar objects since Forex data doesn't have TradeBar objects.

Consolidator Types

There are many different types of consolidators you can create. The process to create and update the consolidator depends on the input data format, output data format, and the consolidation technique.

Time Period Consolidators

Time period consolidators aggregate data based on a period of time like a number of seconds, minutes, or days. If a time period consolidator aggregates data over multiple days, the multi-day aggregation cycle starts and repeats based on the time stamp of the first data point you use to update the consolidator.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarTradeBarConsolidator
QuoteBarQuoteBarQuoteBarConsolidator
TickTradeBarTickConsolidator
TickQuoteBarTickQuoteBarConsolidator

Calendar Consolidators

Calendar consolidators aggregate data based on specific periods of time like a weekly basis, quarterly basis, or any schedule with specific start and end times. In contrast to time period consolidators, the bars that calendar consolidators produce always end on the same schedule, regardless of the first data point the consolidator receives.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarTradeBarConsolidator
QuoteBarQuoteBarQuoteBarConsolidator
TickTradeBarTickConsolidator
TickQuoteBarTickQuoteBarConsolidator

Count Consolidators

Count consolidators aggregate data based on a number of bars or ticks. This type of consolidator aggregates n samples together, regardless of the time period the samples cover. Managing count consolidators is similiar to managing time period consolidators, except you create count consolidators with an integer argument instead of a time-based argument.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarTradeBarConsolidator
QuoteBarQuoteBarQuoteBarConsolidator
TickTradeBarTickConsolidator
TickQuoteBarTickQuoteBarConsolidator

Mixed-Mode Consolidators

Mixed-mode consolidators are a combination of count consolidators and time period consolidators. This type of consolidator aggregates n samples together or aggregates samples over a specific time period, whichever happens first.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarTradeBarConsolidator
QuoteBarQuoteBarQuoteBarConsolidator
TickTradeBarTickConsolidator
TickQuoteBarTickQuoteBarConsolidator

Renko Consolidators

Renko consolidators aggregate bars based on a fixed price movement. The RenkoConsolidator produces Renko bars by their traditional definition. A RenkoConsolidator with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 above the high of the previous bar or $1 below the low of the previous bar. If the price jumps multiple dollars in a single tick, the RenkoConsolidator produces multiple $1 bars in a single time step.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarRenkoConsolidator
QuoteBarQuoteBarRenkoConsolidator
TickTradeBarRenkoConsolidator
TickQuoteBarRenkoConsolidator

Classic Renko Consolidators

Renko consolidators aggregate bars based on a fixed price movement. The ClassicRenkoConsolidator produces a different type of Renko bars than the RenkoConsolidator. A ClassicRenkoConsolidator with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 away from the close of the previous bar. If the price jumps multiple dollars in a single tick, the ClassicRenkoConsolidator only produces one bar per time step where the open of each bar matches the close of the previous bar.

The following table shows which consolidator type to use based on the data format of the input and output:

InputOutputClass Type
TradeBarTradeBarClassicRenkoConsolidator
QuoteBarQuoteBarClassicRenkoConsolidator
TickTradeBarClassicRenkoConsolidator
TickQuoteBarClassicRenkoConsolidator

Sequential Consolidators

Sequential consolidators wire two internal consolidators together such that the output of the first consolidator is the input to the second consolidator and the output of the second consolidator is the output of the sequential consolidator.

For more information about sequential consolidators, see Combining Consolidators.

Execution Sequence

The algorithm manager calls events in the following order:

  1. Scheduled Events
  2. Consolidation event handlers
  3. OnData event handler

This event flow is important to note. For instance, if your consolidation handlers or OnData event handler appends data to a RollingWindow and you use that RollingWindow in your Scheduled Event, when the Scheduled Event executes, the RollingWindow won't contain the most recent data.

The consolidators are called in the order that they are updated. If you register them for automatic updates, they are updated in the order that you register them.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: