Share New Research 
Dear all,
I am experimenting with Futures using daily prices. Since Futures do not support Daily resolution, I am consolidating minute data following the same logic as suggested here
https://www.quantconnect.com/forum/discussion/8902/futures-tick-data-consolidationHowever, as you can see in the attached backtest, even without any logic in the consolidator, when there are about 15 futures (3 contracts each, so the filter is 180 days), the algorithm is quite slow. And with the consolidation part commented out (lines 24-26), algo is executing at a normal pace. Am I doing something wrong?
Arthur Asenheimer
Hi Aram,
I have two suggestions to speed up your algorithm.
1. Remove consolidator subscriptions for contracts you no longer need via
self.SubscriptionManager.RemoveConsolidator(contract.Symbol, dailyConsolidator)2. Reduce frequency for creating the contract list. The OnData() method is called every minute and so popular_contracts is recreated every minute. I recommend putting an if-statement in front of this part of the code so that it only runs once a day, or alternatively you can also use ScheduleEvents.
Let me know if that helped.
Aram Darakchian
Hi Arthur,
Thank you for your suggestions!
After implementing both, there is no significant reduction.
Unsubscribing ununsed contracts wll prevent algo time from exploding when run for say 5 years, but in this case 3 month example, it does not have any impact.
As for the frequency reduction, I am not sure why it did not have any meaningful impact.
Here is the comparison with and without the consolidators:
Algorithm Id:(78237ca400941cdea307cfa0c53b6b38) completed in 119.90 seconds at 110k data points per second. Processing total of 13,146,385 data points.Algorithm Id:(631d2be7a915a4b8610c417563566dbd) completed in 44.13 seconds at 298k data points per second. Processing total of 13,146,385 data points.
Arthur Asenheimer
>> Unsubscribing ununsed contracts wll prevent algo time from exploding when run for say 5 years, but in this case 3 month example, it does not have any impact. <<
Yes, exactly. I assumed that you attached a backtest with only 3 months period for demo purposes only.
>> As for the frequency reduction, I am not sure why it did not have any meaningful impact. <<
Yes, I think because the OnData() call is not very expensive here. Unfortunately, I can't think of any other ways to speed up the algorithm. But you correctly pointed out that it is related to the Consolidator. I have also observed that the Consolidator has a significant impact on the backtesing duration and appears to increase proportionally with the numbers of Symbols.
Maybe some else has an idea ..... ?
Arthur Asenheimer
*someone
P Chen
On inspection I would guess the sheer number of asset data subscriptions at minute resolution makes slow backtests unavoidable. If you can live with using only daily data for testing purposes it might be worth rejigging the algorithm to use QC's access to Quandl data?
Derek Melchin
Hi Aram,
Adding consolidators increases the workload of Lean. As a consequence, we experience longer backtests.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Aram Darakchian
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!