Consolidating Data

Consolidator Types

Introduction

There are many types of consolidators you can create. This page describes them and demonstrates how to create them.

Time Period Consolidators

Time period consolidators aggregate data based on a period of time. The consolidator time period must be greater than or equal to the resolution of the security subscription. For instance, you can aggregate minute bars into 10-minute bars, but you can't aggregate hour bars into 10-minute bars. To set the time period for the consolidator, you can use either a timedeltaTimeSpan, Resolution, or CalendarInfo object.

timedelta Periods

TimeSpan Periods

If you define the time period with a timedeltaTimeSpan object, the time starts from the beginning of the day, not the beginning of the market open or the first data point. For example, if you use timedelta(minutes=7)TimeSpan.FromMinutes(7), the 7-minute counter starts at midnight. Additionally, the time period is relative to the data time zone, not the algorithm time zone. If you consolidate Crypto data into daily bars, the event handler receives the consolidated bars at midnight 12:00 AM Coordinated Universal Time (UTC), regardless of the algorithm time zone.

Resolution Periods

The Resolution enumeration has the following members:

If you consolidate on an hourly basis, the consolidator ends at the top of the hour, not every hour after the market open. For US Equities, that's 10 AM Eastern Time (ET), not 10:30 AM ET.

CalendarInfo Periods

You can use the built-in CalendarInfo objects or create your own. The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

MethodDescription
Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time

If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

var consolidator = new TradeBarConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
# Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
consolidator = TradeBarConsolidator(self.consolidation_period)

If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

Create Time Period Consolidators

To create a time period consolidator, pass the time period to the consolidator constructor.

// TimeSpan argument
_timeSpanTradeBar = new TradeBarConsolidator(TimeSpan.FromDays(1));
_timeSpanQuoteBar = new QuoteBarConsolidator(TimeSpan.FromDays(1));
_timeSpanTradeTick = new TickConsolidator(TimeSpan.FromSeconds(1));
_timeSpanQuoteTick = new TickQuoteBarConsolidator(TimeSpan.FromSeconds(1));

// Resolution argument (only works for TradeBarConsolidator);
_resolutionTradeBar = new TradeBarConsolidator.FromResolution(Resolution.Daily);

// Calendar method argument
_calendarTradeBar = new TradeBarConsolidator(Calendar.Yearly);
_calendarQuoteBar = new QuoteBarConsolidator(Calendar.Quarterly);
_calendarTradeTick = new TickConsolidator(Calendar.Monthly);
_calendarQuoteTick = new TickQuoteBarConsolidator(Calendar.Weekly);

// Custom method argument
_customConsolidator = new TradeBarConsolidator(CustomPeriod);
# timedelta argument
self.timedelta_trade_bar = TradeBarConsolidator(timedelta(days=1))
self.timedelta_quote_bar = QuoteBarConsolidator(timedelta(days=1))
self.timedelta_trade_tick = TickConsolidator(timedelta(seconds=1))
self.timedelta_quote_tick = TickQuoteBarConsolidator(timedelta(seconds=1))

# Resolution argument (only works for TradeBarConsolidator)
self.resolution_trade_bar = TradeBarConsolidator.FromResolution(Resolution.Daily)

# Calendar method argument
self.calendar_trade_bar = TradeBarConsolidator(Calendar.Yearly)
self.calendar_quote_bar = QuoteBarConsolidator(Calendar.Quarterly)
self.calendar_trade_tick = TickConsolidator(Calendar.Monthly)
self.calendar_quote_tick = TickQuoteBarConsolidator(Calendar.Weekly)

# Custom method argument
self.custom_consolidator = TradeBarConsolidator(self.CustomPeriod)

Shortcut Method

The Consolidate method is a helper method to create time period consolidators for algorithms with static universes. With just one line of code, you can create data in any time period with a timedeltaTimeSpan, Resolution, or CalendarInfo object. To create a consolidator with the shortcut method, call Consolidate with a Symbol, time period, and event handler. If you don't pass the method a Symbol, it looks up the Symbol internally.

// Consolidate 1min SPY -> 45min Bars
_timespanConsolidator = Consolidate("SPY", TimeSpan.FromMinutes(45), FortyFiveMinuteBarHandler);

// Consolidate 1min SPY -> 1-Hour Bars
_resolutionConsolidator = Consolidate("SPY", Resolution.Hour, HourBarHandler);

// Consolidate 1min SPY -> 1-Week Bars
_calendarConsolidator = Consolidate("SPY", Calendar.Weekly, WeekBarHandler);
# Consolidate 1min SPY -> 45min Bars
self.timedelta_consolidator = self.Consolidate("SPY", timedelta(minutes=45), self.FortyFiveMinuteBarHandler)

# Consolidate 1min SPY -> 1-Hour Bars
self.resolution_consolidator = self.Consolidate("SPY", Resolution.Hour, self.HourBarHandler)

# Consolidate 1min SPY -> 1-Week Bars
self.calendar_consolidator = self.Consolidate("SPY", Calendar.Weekly, self.WeekBarHandler)

If your data subscription provides both trades and quotes, you can pass a TickType to the Consolidate method to specify the data format you want to consolidate.

var symbol = AddEquity("SPY", Resolution.Minute).Symbol;

// Create QuoteBar objects
_consolidator = Consolidate(symbol, Resolution.Hour, TickType.Quote, ConsolidationHandler);
symbol = self.AddEquity("SPY", Resolution.Minute).Symbol

# Create QuoteBar objects
self.consolidator = self.Consolidate(symbol, Resolution.Hour, TickType.Quote, self.consolidation_handler)

When the consolidator receives a bar that reaches or passes the consolidation period, it passes the consolidated bar to the event handler.

Count Consolidators

Count consolidators aggregate data based on a number of bars or ticks. This type of consolidator aggregates n samples together, regardless of the time period the samples cover. To create a count consolidator, pass the number of samples to the consolidator constructor.

self.trade_bar_consolidator = TradeBarConsolidator(10)

self.quote_bar_consolidator = QuoteBarConsolidator(10)

self.tick_trade_consolidator = TickConsolidator(10)

self.tick_quote_consolidator = TickQuoteBarConsolidator(10)
_tradeBarConsolidator = new TradeBarConsolidator(10);

_quoteBarConsolidator = new QuoteBarConsolidator(10);

_tickTradeConsolidator = new TickConsolidator(10);

_tickQuoteConsolidator = new TickQuoteBarConsolidator(10);

When the consolidator receives the n-th bar or tick, it passes the consolidated bar to the event handler.

Mixed-Mode Consolidators

Mixed-mode consolidators are a combination of count consolidators and time period consolidators. This type of consolidator aggregates n samples together or aggregates samples over a specific time period, whichever happens first. To create a mixed-mode consolidator, pass the number of samples and the time period to the consolidator constructor.

self.trade_bar_consolidator = TradeBarConsolidator(5, timedelta(days=5))

self.quote_bar_consolidator = QuoteBarConsolidator(5, timedelta(hours=5))

self.tick_trade_consolidator = TickConsolidator(50, timedelta(seconds=5))

self.tick_quote_consolidator = TickQuoteBarConsolidator(50, timedelta(seconds=5))
_tradeBarConsolidator = new TradeBarConsolidator(5, TimeSpan.FromDays(5));

_quoteBarConsolidator = new QuoteBarConsolidator(5, TimeSpan.FromHours(5));

_tickTradeConsolidator = new TickConsolidator(50, TimeSpan.FromSeconds(5));

_tickQuoteConsolidator = new TickQuoteBarConsolidator(50, TimeSpan.FromSeconds(5));

When the consolidator receives the n-th sample or the time period lapses, it passes the consolidated bar to the event handler.

Renko Consolidators

Renko consolidators aggregate bars based on a fixed price movement. To create Renko bars, you can use the RenkoConsolidator or the ClassicRenkoConsolidator.

Renko Consolidator

The RenkoConsolidator produces Renko bars by their traditional definition. A RenkoConsolidator with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 above the high of the previous bar or $1 below the low of the previous bar. If the price jumps multiple dollars in a single tick, the RenkoConsolidator produces multiple $1 bars in a single time step. To create this type of Renko consolidator, pass the bar size to the RenkoConsolidator constructor.

# Create a Renko consolidator that emits a bar when the price moves $1
self.consolidator = RenkoConsolidator(1)
// Create a Renko consolidator that emits a bar when the price moves $1
_consolidator = new RenkoConsolidator(1m);

When the consolidator produces a new Renko bar, it passes the consolidated bar to the event handler.

Classic Renko Consolidator

The ClassicRenkoConsolidator produces a different type of Renko bars than the RenkoConsolidator. A ClassicRenkoConsolidator with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 away from the close of the previous bar. If the price jumps multiple dollars in a single tick, the ClassicRenkoConsolidator only produces one bar per time step where the open of each bar matches the close of the previous bar. To create this type of Renko consolidator, pass the bar size to the ClassicRenkoConsolidator constructor.

# Create a Classic Renko consolidator that emits a bar when the price moves $1
self.consolidator = ClassicRenkoConsolidator(1)
// Create a Classic Renko consolidator that emits a bar when the price moves $1
_consolidator = new ClassicRenkoConsolidator(1m);

The ClassicRenkoConsolidator has the following default behavior:

  • It uses the Value property of the IBaseData object it receives to build the Renko bars
  • It ignores the volume of the input data
  • It enforces the open and close of each bar to be a multiple of the bar size

To build the Renko bars with a different property than the Value of the IBaseData object, provide a selector argument. The selector should be a function that receives the IBaseData object and returns a decimal value.

self.consolidator = ClassicRenkoConsolidator(1, selector = lambda data: data.High)
_consolidator = new ClassicRenkoConsolidator(1m, data => (data as TradeBar).High);

To add a non-zero Volume property to the Renko bars, provide a volumeSelector argument. The volumeSelector should be a function that receives the IBaseData object and returns a decimal value.

self.consolidator = ClassicRenkoConsolidator(1, volumeSelector = lambda data: data.Volume)
_consolidator = new ClassicRenkoConsolidator(1m, null, data => (data as TradeBar).Volume);

To relax the requirement that the open and close of the Renko bars must be a multiple of bar size, disable the evenBars argument. If you disable evenBars, the open value of the first Renko bar is set to the first value from the selector. The following opening and closing Renko bar values are all multiples of the first value from the selector

self.consolidator = ClassicRenkoConsolidator(1, evenBars = False)
_consolidator = new ClassicRenkoConsolidator(1m, evenBars: false);

The preceding arguments enable you to create Renko bars that aggregate the excess liquidity on the bid.

# Create Renko consolidator
self.consolidator = ClassicRenkoConsolidator(10, lambda data: data.Value, lambda data: data.LastBidSize - data.LastAskSize)

# Update the consolidator with QuoteBar objects
self.consolidator.Update(slice.QuoteBars[self.symbol])
// Create Renko consolidator
_consolidator = new ClassicRenkoConsolidator(10, null, 
    data => (data as QuoteBar).LastBidSize - (data as QuoteBar).LastAskSize);
    
# Update the consolidator with QuoteBar objects
_consolidator.Update(slice.QuoteBars[self.symbol]);

To view a full example of a ClassicRenkoConsolidator, see the ClassicRenkoConsolidatorAlgorithmClassicRenkoConsolidatorAlgorithm in the LEAN GitHub repository.

When the consolidator produces a new Renko bar, it passes the consolidated bar to the event handler.

It's not currently possible to use the Renko consolidators to create volume bars. To be notified when volume bars are available, subscribe to GitHub Issue #3754.

Sequential Consolidators

A sequential consolidator wires two internal consolidators together such that the output of the first consolidator is the input to the second consolidator and the output of the second consolidator is the output of the sequential consolidator. To create a sequential consolidator, create two consolidators and then pass them to the SequentialConsolidator constructor.

# This first consolidator produces a consolidated bar after a day passes
one_day_consolidator = TradeBarConsolidator(timedelta(days=1))

# This second consolidators produces a consolidated bar after it sees 3 samples
three_count_consolidator = TradeBarConsolidator(3)

# This sequential consolidator aggregates three 1-day bars together
self.consolidator = SequentialConsolidator(one_day_consolidator, three_count_consolidator)
// This first consolidator produces a consolidated bar after a day passes
var oneDayConsolidator = new TradeBarConsolidator(TimeSpan.FromDays(1));

// This second consolidators produces a consolidated bar after it sees 3 samples
var threeCountConsolidator = new TradeBarConsolidator(3);

// This sequential consolidator aggregates three 1-day bars together
_consolidator = new SequentialConsolidator(oneDayConsolidator, threeCountConsolidator);

When the second internal consolidator produces a new bar, the SequentialConsolidator passes the consolidated bar to the event handler.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: