A python algorithm that selects 400 securities at minute resolution, with empty OnData does ~250K points per second.

Adding a consolidator for each security and registering it with the SubscriptionManager reduces it to ~50K points per second.

If we instead manually call Consolidator's Update() method in OnData(), it improves to ~125K points per second.

All reproducible in attached backtest.

Questions:

  • Why does using the SubscriptionManager cause such a drastic slowdown?
  • Is there anything that can be done to improve performance of TradeBarConsolidator? The aggregation logic does not appear overly complex so it is surprising that it reduces throughput by 50%.