Consolidator Types

Calendar Consolidators

Introduction

Calendar consolidators aggregate data based on specific periods of time like a weekly basis, quarterly basis, or any schedule with specific start and end times. In contrast to time period consolidators, the bars that calendar consolidators produce always end on the same schedule, regardless of the first data point the consolidator receives.

Consolidate Trade Bars

TradeBar consolidators aggregate TradeBar objects into TradeBar objects of the same size or larger. Follow these steps to create and manage a TradeBar consolidator based on custom start and end periods:

  1. Create the consolidator.

    2. To set the time period for the consolidator, you can use the built-in CalendarInfo objects or create your own. The following list describes each technique:

    • Standard Periods

      • The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

      MethodDescription
      Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
      Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
      Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
      Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time
       
      _consolidator = new TradeBarConsolidator(Calendar.Weekly);
// Alias:
// _consolidator = CreateConsolidator(Calendar.Weekly, typeof(TradeBar));
      self.consolidator = TradeBarConsolidator(Calendar.Weekly)
# Alias:
# self.consolidator = self.CreateConsolidator(Calendar.Weekly, TradeBar)

      To use the Calendar class, you may need to add the following code to the imports section because of ambiguity with .NET Calendar Class:

      using Calendar = QuantConnect.Data.Consolidators.Calendar;
    • Custom Periods

      • If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

      _consolidator = new TradeBarConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
      # Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
self.consolidator = TradeBarConsolidator(self.consolidation_period)
  2. Add an event handler to the consolidator.
    3. _consolidator.DataConsolidated += ConsolidationHandler;
    self.consolidator.DataConsolidated += self.consolidation_handler

    LEAN passes consolidated bars to the consolidator event handler in your algorithm. The most common error when creating consolidators is to put parenthesis () at the end of your method name when setting the event handler of the consolidator. If you use parenthesis, the method executes and the result is passed as the event handler instead of the method itself. Remember to pass the name of your method to the event system. Specifically, it should be ConsolidationHandlerself.consolidation_handler, not ConsolidationHandler()self.consolidation_handler().

  3. Define the consolidation handler.
    4. void ConsolidationHandler(object sender, TradeBar consolidatedBar)
{

}
    def consolidation_handler(self, sender: object, consolidated_bar: TradeBar) -> None:
    pass

    If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

  4. Update the consolidator.

    5. You can automatically or manually update the consolidator.

    • Automatic Updates

      • To automatically update a consolidator with data from the security subscription, call the AddConsolidator method of the Subscription Manager.

      self.SubscriptionManager.AddConsolidator(self.symbol, self.consolidator)
      SubscriptionManager.AddConsolidator(_symbol, _consolidator);
    • Manual Updates

      • Manual updates let you control when the consolidator updates and what data you use to update it. If you need to warm up a consolidator with data outside of the warm-up period, you can manually update the consolidator. To manually update a consolidator, call its Update method with a TradeBar object. You can update the consolidator with data from the Slice object in the OnData method or with data from a history request.

      # Example 1: Update the consolidator with data from the Slice object
def OnData(self, slice: Slice) -> None:
    trade_bar = slice.Bars[self.symbol]
    self.consolidator.Update(trade_bar)

# Example 2: Update the consolidator with data from a history request
history = self.History[TradeBar](self.symbol, 30, Resolution.Minute)
for trade_bar in history:
    self.consolidator.Update(trade_bar)
      // Example 1: Update the consolidator with data from the Slice object
public override void OnData(Slice slice)
{
    var tradeBar = slice.Bars[_symbol];
    _consolidator.Update(tradeBar);
}

// Example 2: Update the consolidator with data from a history request
var history = History<TradeBar>(_symbol, 30, Resolution.Minute);
foreach (var tradeBar in history)
{
    _consolidator.Update(tradeBar);
}
  5. If you create consolidators for securities in a dynamic universe and register them for automatic updates, remove the consolidator when the security leaves the universe.
    6. SubscriptionManager.RemoveConsolidator(_symbol, _consolidator);
    self.SubscriptionManager.RemoveConsolidator(self.symbol, self.consolidator)

    If you have a dynamic universe and don't remove consolidators, they compound internally, causing your algorithm to slow down and eventually die once it runs out of RAM. For an example of removing consolidators from universe subscriptions, see the GasAndCrudeOilEnergyCorrelationAlphaGasAndCrudeOilEnergyCorrelationAlpha in the LEAN GitHub repository.

You can also use the Consolidate helper method to create calendar consolidators and register them for automatic updates.

    _consolidator = Consolidate(_symbol, Calendar.Weekly, ConsolidationHandler);
    self.consolidator = self.Consolidate(self.symbol, Calendar.Weekly, self.consolidation_handler)

If you use the Consolidate helper method, the consolidation handler doesn't receive an object argument.

void ConsolidationHandler(TradeBar consolidatedBar)
{

}
def consolidation_handler(self, consolidated_bar: TradeBar) -> None:
    pass

Consolidate Quote Bars

QuoteBar consolidators aggregate QuoteBar objects into QuoteBar objects of the same size or larger. Follow these steps to create and manage a QuoteBar consolidator based on custom start and end periods:

  1. Create the consolidator.

    2. To set the time period for the consolidator, you can use the built-in CalendarInfo objects or create your own. The following list describes each technique:

    • Standard Periods

      • The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

      MethodDescription
      Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
      Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
      Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
      Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time
       
      _consolidator = new QuoteBarConsolidator(Calendar.Weekly);
// Alias:
// _consolidator = CreateConsolidator(Calendar.Weekly, typeof(QuoteBar));
      self.consolidator = QuoteBarConsolidator(Calendar.Weekly)
# Alias:
# self.consolidator = self.CreateConsolidator(Calendar.Weekly, QuoteBar)

      To use the Calendar class, you may need to add the following code to the imports section because of ambiguity with .NET Calendar Class:

      using Calendar = QuantConnect.Data.Consolidators.Calendar;
    • Custom Periods

      • If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

      _consolidator = new QuoteBarConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
      # Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
self.consolidator = QuoteBarConsolidator(self.consolidation_period)
  2. Add an event handler to the consolidator.
    3. _consolidator.DataConsolidated += ConsolidationHandler;
    self.consolidator.DataConsolidated += self.consolidation_handler

    LEAN passes consolidated bars to the consolidator event handler in your algorithm. The most common error when creating consolidators is to put parenthesis () at the end of your method name when setting the event handler of the consolidator. If you use parenthesis, the method executes and the result is passed as the event handler instead of the method itself. Remember to pass the name of your method to the event system. Specifically, it should be ConsolidationHandlerself.consolidation_handler, not ConsolidationHandler()self.consolidation_handler().

  3. Define the consolidation handler.
    4. void ConsolidationHandler(object sender, QuoteBar consolidatedBar)
{

}
    def consolidation_handler(self, sender: object, consolidated_bar: QuoteBar) -> None:
    pass

    If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

  4. Update the consolidator.

    5. You can automatically or manually update the consolidator.

    • Automatic Updates

      • To automatically update a consolidator with data from the security subscription, call the AddConsolidator method of the Subscription Manager.

      self.SubscriptionManager.AddConsolidator(self.symbol, self.consolidator)
      SubscriptionManager.AddConsolidator(_symbol, _consolidator);
    • Manual Updates

      • Manual updates let you control when the consolidator updates and what data you use to update it. If you need to warm up a consolidator with data outside of the warm-up period, you can manually update the consolidator. To manually update a consolidator, call its Update method with a QuoteBar object. You can update the consolidator with data from the Slice object in the OnData method or with data from a history request.

      # Example 1: Update the consolidator with data from the Slice object
def OnData(self, slice: Slice) -> None:
    quote_bar = slice.QuoteBars[self.symbol]
    self.consolidator.Update(quote_bar)

# Example 2: Update the consolidator with data from a history request
history = self.History[QuoteBar](self.symbol, 30, Resolution.Minute)
for quote_bar in history:
    self.consolidator.Update(quote_bar)
      // Example 1: Update the consolidator with data from the Slice object
public override void OnData(Slice slice)
{
    var quoteBar = slice.QuoteBars[_symbol];
    _consolidator.Update(quoteBar);
}

// Example 2: Update the consolidator with data from a history request
var history = History<QuoteBar>(_symbol, 30, Resolution.Minute);
foreach (var quoteBar in history)
{
    _consolidator.Update(quoteBar);
}
  5. If you create consolidators for securities in a dynamic universe and register them for automatic updates, remove the consolidator when the security leaves the universe.
    6. SubscriptionManager.RemoveConsolidator(_symbol, _consolidator);
    self.SubscriptionManager.RemoveConsolidator(self.symbol, self.consolidator)

    If you have a dynamic universe and don't remove consolidators, they compound internally, causing your algorithm to slow down and eventually die once it runs out of RAM. For an example of removing consolidators from universe subscriptions, see the GasAndCrudeOilEnergyCorrelationAlphaGasAndCrudeOilEnergyCorrelationAlpha in the LEAN GitHub repository.

You can also use the Consolidate helper method to create calendar consolidators and register them for automatic updates.

    _consolidator = Consolidate<QuoteBar>(_symbol, Calendar.Weekly, TickType.Quote, ConsolidationHandler);
    self.consolidator = self.Consolidate(self.symbol, Calendar.Weekly, TickType.Quote, self.consolidation_handler)

If you use the Consolidate helper method, the consolidation handler doesn't receive an object argument.

void ConsolidationHandler(QuoteBar consolidatedBar)
{

}
def consolidation_handler(self, consolidated_bar: QuoteBar) -> None:
    pass

Consolidate Trade Ticks

Tick consolidators aggregate Tick objects into TradeBar objects. Follow these steps to create and manage a Tick consolidator based on custom start and end periods:

  1. Create the consolidator.

    2. To set the time period for the consolidator, you can use the built-in CalendarInfo objects or create your own. The following list describes each technique:

    • Standard Periods

      • The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

      MethodDescription
      Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
      Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
      Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
      Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time
       
      _consolidator = new TickConsolidator(Calendar.Weekly);
// Alias:
// _consolidator = CreateConsolidator(Calendar.Weekly, typeof(Tick));
      self.consolidator = TickConsolidator(Calendar.Weekly)
# Alias:
# self.consolidator = self.CreateConsolidator(Calendar.Weekly, Tick)

      To use the Calendar class, you may need to add the following code to the imports section because of ambiguity with .NET Calendar Class:

      using Calendar = QuantConnect.Data.Consolidators.Calendar;
    • Custom Periods

      • If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

      _consolidator = new TickConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
      # Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
self.consolidator = TickConsolidator(self.consolidation_period)
  2. Add an event handler to the consolidator.
    3. _consolidator.DataConsolidated += ConsolidationHandler;
    self.consolidator.DataConsolidated += self.consolidation_handler

    LEAN passes consolidated bars to the consolidator event handler in your algorithm. The most common error when creating consolidators is to put parenthesis () at the end of your method name when setting the event handler of the consolidator. If you use parenthesis, the method executes and the result is passed as the event handler instead of the method itself. Remember to pass the name of your method to the event system. Specifically, it should be ConsolidationHandlerself.consolidation_handler, not ConsolidationHandler()self.consolidation_handler().

  3. Define the consolidation handler.
    4. void ConsolidationHandler(object sender, TradeBar consolidatedBar)
{

}
    def consolidation_handler(self, sender: object, consolidated_bar: TradeBar) -> None:
    pass

    If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

  4. Update the consolidator.

    5. You can automatically or manually update the consolidator.

    • Automatic Updates

      • To automatically update a consolidator with data from the security subscription, call the AddConsolidator method of the Subscription Manager.

      self.SubscriptionManager.AddConsolidator(self.symbol, self.consolidator)
      SubscriptionManager.AddConsolidator(_symbol, _consolidator);
    • Manual Updates

      • Manual updates let you control when the consolidator updates and what data you use to update it. If you need to warm up a consolidator with data outside of the warm-up period, you can manually update the consolidator. To manually update a consolidator, call its Update method with a Tick object. You can update the consolidator with data from the Slice object in the OnData method or with data from a history request.

      # Example 1: Update the consolidator with data from the Slice object
def OnData(self, slice: Slice) -> None:
    ticks = slice.Ticks[self.symbol]
    for tick in ticks:
    	self.consolidator.Update(tick)

# Example 2: Update the consolidator with data from a history request
ticks = self.History[Tick](self.symbol, timedelta(minutes=3), Resolution.Tick)
for tick in ticks:
    self.consolidator.Update(tick)
      // Example 1: Update the consolidator with data from the Slice object
public override void OnData(Slice slice)
{
    var ticks = slice.Ticks[_symbol];
    foreach (var tick in ticks)
    {
    	_consolidator.Update(tick);
    }
}

// Example 2: Update the consolidator with data from a history request
var ticks = History<Tick>(_symbol, TimeSpan.FromMinutes(3), Resolution.Tick);
foreach (var tick in ticks)
{
    _consolidator.Update(tick);
}
  5. If you create consolidators for securities in a dynamic universe and register them for automatic updates, remove the consolidator when the security leaves the universe.
    6. SubscriptionManager.RemoveConsolidator(_symbol, _consolidator);
    self.SubscriptionManager.RemoveConsolidator(self.symbol, self.consolidator)

    If you have a dynamic universe and don't remove consolidators, they compound internally, causing your algorithm to slow down and eventually die once it runs out of RAM. For an example of removing consolidators from universe subscriptions, see the GasAndCrudeOilEnergyCorrelationAlphaGasAndCrudeOilEnergyCorrelationAlpha in the LEAN GitHub repository.

You can also use the Consolidate helper method to create calendar consolidators and register them for automatic updates.

    _consolidator = Consolidate(_symbol, Calendar.Weekly, ConsolidationHandler);
    self.consolidator = self.Consolidate(self.symbol, Calendar.Weekly, self.consolidation_handler)

If you use the Consolidate helper method, the consolidation handler doesn't receive an object argument.

void ConsolidationHandler(TradeBar consolidatedBar)
{

}
def consolidation_handler(self, consolidated_bar: TradeBar) -> None:
    pass

Consolidate Quote Ticks

Tick quote bar consolidators aggregate Tick objects that represent quotes into QuoteBar objects. Follow these steps to create and manage a Tick quote bar consolidator based on custom start and end periods:

  1. Create the consolidator.

    2. To set the time period for the consolidator, you can use the built-in CalendarInfo objects or create your own. The following list describes each technique:

    • Standard Periods

      • The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

      MethodDescription
      Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
      Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
      Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
      Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time
       
      _consolidator = new TickQuoteBarConsolidator(Calendar.Weekly);
// Alias:
// _consolidator = CreateConsolidator(Calendar.Weekly, typeof(Tick), TickType.Quote);
      self.consolidator = TickQuoteBarConsolidator(Calendar.Weekly)
# Alias:
# self.consolidator = self.CreateConsolidator(Calendar.Weekly, Tick, TickType.Quote)

      To use the Calendar class, you may need to add the following code to the imports section because of ambiguity with .NET Calendar Class:

      using Calendar = QuantConnect.Data.Consolidators.Calendar;
    • Custom Periods

      • If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

      _consolidator = new TickQuoteBarConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
      # Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
self.consolidator = TickQuoteBarConsolidator(self.consolidation_period)
  2. Add an event handler to the consolidator.
    3. _consolidator.DataConsolidated += ConsolidationHandler;
    self.consolidator.DataConsolidated += self.consolidation_handler

    LEAN passes consolidated bars to the consolidator event handler in your algorithm. The most common error when creating consolidators is to put parenthesis () at the end of your method name when setting the event handler of the consolidator. If you use parenthesis, the method executes and the result is passed as the event handler instead of the method itself. Remember to pass the name of your method to the event system. Specifically, it should be ConsolidationHandlerself.consolidation_handler, not ConsolidationHandler()self.consolidation_handler().

  3. Define the consolidation handler.
    4. void ConsolidationHandler(object sender, QuoteBar consolidatedBar)
{

}
    def consolidation_handler(self, sender: object, consolidated_bar: QuoteBar) -> None:
    pass

    If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

  4. Update the consolidator.

    5. You can automatically or manually update the consolidator.

    • Automatic Updates

      • To automatically update a consolidator with data from the security subscription, call the AddConsolidator method of the Subscription Manager.

      self.SubscriptionManager.AddConsolidator(self.symbol, self.consolidator)
      SubscriptionManager.AddConsolidator(_symbol, _consolidator);
    • Manual Updates

      • Manual updates let you control when the consolidator updates and what data you use to update it. If you need to warm up a consolidator with data outside of the warm-up period, you can manually update the consolidator. To manually update a consolidator, call its Update method with a Tick object. You can update the consolidator with data from the Slice object in the OnData method or with data from a history request.

      # Example 1: Update the consolidator with data from the Slice object
def OnData(self, slice: Slice) -> None:
    ticks = slice.Ticks[self.symbol]
    for tick in ticks:
    	self.consolidator.Update(tick)

# Example 2: Update the consolidator with data from a history request
ticks = self.History[Tick](self.symbol, timedelta(minutes=3), Resolution.Tick)
for tick in ticks:
    self.consolidator.Update(tick)
      // Example 1: Update the consolidator with data from the Slice object
public override void OnData(Slice slice)
{
    var ticks = slice.Ticks[_symbol];
    foreach (var tick in ticks)
    {
    	_consolidator.Update(tick);
    }
}

// Example 2: Update the consolidator with data from a history request
var ticks = History<Tick>(_symbol, TimeSpan.FromMinutes(3), Resolution.Tick);
foreach (var tick in ticks)
{
    _consolidator.Update(tick);
}
  5. If you create consolidators for securities in a dynamic universe and register them for automatic updates, remove the consolidator when the security leaves the universe.
    6. SubscriptionManager.RemoveConsolidator(_symbol, _consolidator);
    self.SubscriptionManager.RemoveConsolidator(self.symbol, self.consolidator)

    If you have a dynamic universe and don't remove consolidators, they compound internally, causing your algorithm to slow down and eventually die once it runs out of RAM. For an example of removing consolidators from universe subscriptions, see the GasAndCrudeOilEnergyCorrelationAlphaGasAndCrudeOilEnergyCorrelationAlpha in the LEAN GitHub repository.

You can also use the Consolidate helper method to create calendar consolidators and register them for automatic updates.

    _consolidator = Consolidate<QuoteBar>(_symbol, Calendar.Weekly, TickType.Quote, ConsolidationHandler);
    self.consolidator = self.Consolidate(self.symbol, Calendar.Weekly, TickType.Quote, self.consolidation_handler)

If you use the Consolidate helper method, the consolidation handler doesn't receive an object argument.

void ConsolidationHandler(QuoteBar consolidatedBar)
{

}
def consolidation_handler(self, consolidated_bar: QuoteBar) -> None:
    pass

Consolidate Other Data

<dataType> consolidators aggregate various types of data objects into <dataType> objects of the same size or larger. If you consolidate custom data or alternative datasets, check the definition of the data class to see its data type. Follow these steps to create and manage a <dataType> consolidator based on custom start and end periods:

  1. Create the consolidator.

    2. To set the time period for the consolidator, you can use the built-in CalendarInfo objects or create your own. The following list describes each technique:

    • Standard Periods

      • The following table describes the helper methods that the Calendar class provides to create the built-in CalendarInfo objects:

      MethodDescription
      Calendar.WeeklyComputes the start of week (previous Monday) of the given date/time
      Calendar.MonthlyComputes the start of month (1st of the current month) of the given date/time
      Calendar.QuarterlyComputes the start of quarter (1st of the starting month of the current quarter) of the given date/time
      Calendar.YearlyComputes the start of year (1st of the current year) of the given date/time
       
      _consolidator = new DynamicDataConsolidator(Calendar.Weekly);
// Alias:
// _consolidator = CreateConsolidator(Calendar.Weekly, typeof(<dataType>));
      self.consolidator = DynamicDataConsolidator(Calendar.Weekly)
# Alias:
# self.consolidator = self.CreateConsolidator(Calendar.Weekly, <dataType>)

      To use the Calendar class, you may need to add the following code to the imports section because of ambiguity with .NET Calendar Class:

      using Calendar = QuantConnect.Data.Consolidators.Calendar;
    • Custom Periods

      • If you need something more specific than the preceding time periods, define a method to set the start time and period of the consolidated bars. The method should receive a datetimeDateTime object that's based in the data time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. The following example demonstrates how to create a custom consolidator for weekly bars:

      _consolidator = new DynamicDataConsolidator(datetime => {
    var period = TimeSpan.FromDays(7);

    var timeSpan = new TimeSpan(17, 0, 0);
    var newDateTime = datetime.Date + timeSpan;
    var delta = 1 + (int)newDateTime.DayOfWeek;
    if (delta > 6)
    {
        delta = 0;
    }
    var start = newDateTime.AddDays(-delta);

    return new CalendarInfo(start, period);
});
      # Define a consolidation period method
def consolidation_period(self, dt: datetime) -> CalendarInfo:
    period = timedelta(7)

    dt = dt.replace(hour=17, minute=0, second=0, microsecond=0)
    delta = 1+dt.weekday()
    if delta > 6:
        delta = 0
    start = dt-timedelta(delta)

    return CalendarInfo(start, period)

# Create the consolidator with the consolidation period method
self.consolidator = DynamicDataConsolidator(self.consolidation_period)
  2. Add an event handler to the consolidator.
    3. _consolidator.DataConsolidated += ConsolidationHandler;
    self.consolidator.DataConsolidated += self.consolidation_handler

    LEAN passes consolidated bars to the consolidator event handler in your algorithm. The most common error when creating consolidators is to put parenthesis () at the end of your method name when setting the event handler of the consolidator. If you use parenthesis, the method executes and the result is passed as the event handler instead of the method itself. Remember to pass the name of your method to the event system. Specifically, it should be ConsolidationHandlerself.consolidation_handler, not ConsolidationHandler()self.consolidation_handler().

  3. Define the consolidation handler.
    4. void ConsolidationHandler(object sender, <dataType> consolidatedBar)
{

}
    def consolidation_handler(self, sender: object, consolidated_bar: <dataType>) -> None:
    pass

    If you use a custom consolidation period method, LEAN passes the consolidated bar to the consolidation handler when the consolidation period ends. The Time and EndTime properties of the consolidated bar reflect the data time zone, but the Time property of the algorithm still reflects the algorithm time zone.

  4. Update the consolidator.

    5. You can automatically or manually update the consolidator.

    • Automatic Updates

      • To automatically update a consolidator with data from the data subscription, call the AddConsolidator method of the Subscription Manager.

      self.SubscriptionManager.AddConsolidator(self.symbol, self.consolidator)
      SubscriptionManager.AddConsolidator(_symbol, _consolidator);
    • Manual Updates

      • Manual updates let you control when the consolidator updates and what data you use to update it. If you need to warm up a consolidator with data outside of the warm-up period, you can manually update the consolidator. To manually update a consolidator, call its Update method with a <dataType> object. You can update the consolidator with data from the Slice object in the OnData method or with data from a history request.

      # Example 1: Update the consolidator with data from the Slice object
def OnData(self, slice: Slice) -> None:
    if self.symbol in slice:
        self.consolidator.Update(slice[self.symbol])

# Example 2: Update the consolidator with data from a history request
history = self.History[<dataType>](self.symbol, 30, Resolution.Daily)
for data in history:
    self.consolidator.Update(data)
      // Example 1: Update the consolidator with data from the Slice object
public override void OnData(Slice slice)
{
    if slice.ContainsKey(_symbol)
    {
        _consolidator.Update(slice[_symbol]);
    }
}

// Example 2: Update the consolidator with data from a history request
var history = History<<dataType>>(_symbol, 30, Resolution.Daily);
foreach (var data in history)
{
    _consolidator.Update(data);
}
  5. If you create consolidators for securities in a dynamic universe and register them for automatic updates, remove the consolidator when the security leaves the universe.
    6. SubscriptionManager.RemoveConsolidator(_symbol, _consolidator);
    self.SubscriptionManager.RemoveConsolidator(self.symbol, self.consolidator)

    If you have a dynamic universe and don't remove consolidators, they compound internally, causing your algorithm to slow down and eventually die once it runs out of RAM. For an example of removing consolidators from universe subscriptions, see the GasAndCrudeOilEnergyCorrelationAlphaGasAndCrudeOilEnergyCorrelationAlpha in the LEAN GitHub repository.

You can also use the Consolidate helper method to create calendar consolidators and register them for automatic updates.

    _consolidator = Consolidate(_symbol, Calendar.Weekly, ConsolidationHandler);
    self.consolidator = self.Consolidate(self.symbol, Calendar.Weekly, self.consolidation_handler)

If you use the Consolidate helper method, the consolidation handler doesn't receive an object argument.

void ConsolidationHandler(<dataType> consolidatedBar)
{

}
def consolidation_handler(self, consolidated_bar: <dataType>) -> None:
    pass

Examples

The following examples are typical calendar functions you may want.

The function should receive a datetimeDateTime object that's based in the algorithm time zone and should return a CalendarInfo object, which contains the start time of the bar in the data time zone and the duration of the consolidation period. This function is evaluated when the duration of the consolidation period has passed and at the following time step.

The preceding sections of this page provide a typical calendar function that consolidates data weekly, starting at 5:00 PM. If you consolidate US Equity and Crypto data, the event handler triggers at different times since the data time zone of US Equity is America/New York and Crypto is UTC.

Example 1: Consolidate data into 20-minute TradeBar objects that start at 9:30 AM

This is a typical case for the US markets because they open every business day at 9:30 PM Eastern Time (ET).

public CalendarInfo EveryTwentyMinuteAfterEquityMarketOpen(DateTime datetime)
{
    var period = TimeSpan.FromMinutes(20);
    var openTime = datetime.Date + new TimeSpan(9, 30, 0);
    var start = openTime + (int)((datetime - openTime) / period) * period;            
    return new CalendarInfo(start, period);
}
def every_twenty_minute_after_equity_market_open(self, dt: datetime) -> CalendarInfo:
    period = timedelta(minutes=20)
    open_time = dt.replace(hour=9, minute=30, second=0, microsecond=0)
    start = open_time + int((dt - open_time) / period) * period
    return CalendarInfo(start, period)

To consolidate data into another period, change the period variable. For example, to consolidate into 4-hour bars, replace timedelta(minutes=20)TimeSpan.FromMinutes(20) with timedelta(hours=4)TimeSpan.FromHours(4).

Example 2: Consolidate data in daily QuoteBar objects that start at 5 PM:

This is a typical Forex case because the Forex market opens on Sunday at 5 PM ET.

public CalendarInfo CustomDailyForex(DateTime datetime)
{
    var start = datetime.Date
        .AddHours(datetime.Hour < 17 ? -7 : 17);
    return new CalendarInfo(start, TimeSpan.FromDays(1));
}
def custom_daily_forex(self, dt: datetime) -> CalendarInfo:
start = dt.replace(hour=17, minute=0, second=0)
if dt.hour < 17:
    start -= timedelta(1)
return CalendarInfo(start, timedelta(1))

Example 3: Consolidate data into monthly TradeBar objects that start at midnight:

public CalendarInfo CustomMonthly(DateTime datetime)
{
    var start = dt.AddDays(1 - dt.Day).Date;
    var end = dt.AddMonths(1);
    end = new DateTime(end.Year, end.Month, 1);
    return new CalendarInfo(start, (end - start));
}
def custom_monthly(self, dt: datetime) -> CalendarInfo:
    start = dt.replace(day=1).date()
    end = dt.replace(day=28) + timedelta(4)
    end = (end - timedelta(end.day - 1)).date()
    return CalendarInfo(start, end - start)
Demonstration Algorithms
DataConsolidationAlgorithm.py Python DataConsolidationAlgorithm.cs C#

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: