I want to improve performance by moving the heavy processing of daily bars and indicator generation outside the on_data method. The plan is to simply collect the data slice within on_data, then schedule a separate event to handle the intensive processing one minute after market close.
However, I'm concerned about the timing: if process_daily_bars takes a long time to complete, and on_data proceeds to the next trading day in the meantime, it may cause daily_bars to be overwritten or emptied before process_daily_bars finishes.
Could you please confirm whether the scheduled after-market event and the on_data method are able to safely handle different dates in parallel, especially when one of them runs significantly slower? Example of the code
def on_data(self, data):
"""Cache daily bars and update VIX values."""
if self.vix in data:
vix_bar = data[self.vix]
if isinstance(vix_bar, TradeBar):
self.current_vix = vix_bar.Close
else:
self.current_vix = getattr(vix_bar, 'Value', np.nan)
for symbol, bar in data.Bars.items():
if symbol in self.symbol_data:
self.daily_bars[symbol] = bar def process_daily_bars(self):
"""Process cached daily bars after market close."""
if self.IsWarmingUp or not self.daily_bars:
return
self._process_slice(self.daily_bars)
self.daily_bars = {} # Process daily bars and indicators after market close
self.Schedule.On(
self.DateRules.EveryDay(),
self.TimeRules.AfterMarketClose("SPY", 1),
self.process_daily_bars
)
Mia Alissi
Dharmesh, here’s a clear, practical summary for your scenario:
So you can confidently cache data as you do in on_data, and safely process all your daily bars in process_daily_bars after the close, exactly as your sample code outlines. No risk of overlap, no concurrency issues.
Your approach is solid and safe in QuantConnect!
Sources: Getting Started - QuantConnect.com, Periods - QuantConnect.com, Algorithm Engine - QuantConnect.com, Scheduled Events - QuantConnect.com, Time Period Consolidators - QuantConnect.com
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dharmesh Khalasi
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!