Hi,
I have a backtest that analyzes the SP 500 for the past 10 years and finds the top 5 gainers and losers for each month. It buys the top 5 gains and shorts the top 5 losers at the beginning of the month and the liquidates at the end of the month. It works but very, very slowly. Is there any way optimize my algorithm?
Right now it checks the date and if it's the first of the month it buys/shorts and then does another check to see if it's the end of the month and liquidates if it is. It doesn't need anything better than daily resolution but I believe that's not an option currently? Also, I initially tried this so that the algo was analyzing the sp500 and finding the top/worst 5. But I thought the slowness was due to this so I did a bunch of data crunching and hardcoded the top/worst 5 in my algo. Unfortunately, I didn't see an improvement in speed.
Thanks!
Jocelyn
Back
Backtest Optimization
Hi Jocelyn,
I guess the only way to see why the backtests is slow, is watching the actual code. Can you share it?
I guess the only way to see why the backtests is slow, is watching the actual code. Can you share it?
0
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
@Jocelyn - sharing would help, but also we're launching Daily data which should fit your use case well. Daily data is roughly 100x faster than minute data :) The data has finally finished processing so it should be live by tomorrow.
Regarding actual *optimization* support; we don't yet have variable optimization support but it is on the roadmap!
Regarding actual *optimization* support; we don't yet have variable optimization support but it is on the roadmap!
0
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hi,
Attached is my code.
I also had a question regarding the stock data- for instance one of the stocks I was using was SUN. According to the data manager, data is available for the dates between 1/98 and 10/12. When I was backtesting, it said it was missing tradebars data but the dates of my backtest were 10/2005-5/2006. Do you know why it would be missing tradebars data at that point? I wasn't sure if it had to do with my code as I'd tried to be a bit fancy with finding the start and end dates of the month.
Thank you for your help!
Jocelyn
Attached is my code.
I also had a question regarding the stock data- for instance one of the stocks I was using was SUN. According to the data manager, data is available for the dates between 1/98 and 10/12. When I was backtesting, it said it was missing tradebars data but the dates of my backtest were 10/2005-5/2006. Do you know why it would be missing tradebars data at that point? I wasn't sure if it had to do with my code as I'd tried to be a bit fancy with finding the start and end dates of the month.
Thank you for your help!
Jocelyn
0
Hmm... wondering if i did in fact attach a project... lemme try again.
Oh and I ended up removing SUN from my stock list.
Oh and I ended up removing SUN from my stock list.
1
Jocelyn, great work!
I'm guessing here, but maybe the performance problem is because the algorithm is running with too many stocks with minute resolution.
I run a short backtest and the console output shows:
Another backtest, with a 3 years period:
And your algorithm:
So, in one hand, seems the engine is slower as the data points increases. In the other, your algo uses <7M data for just 3 months!
That's my two cents. JJ
I'm guessing here, but maybe the performance problem is because the algorithm is running with too many stocks with minute resolution.
I run a short backtest and the console output shows:
2015-01-30 16:00:00 Algorithm Id:() completed in 12.15 seconds at 20k data points per second. Processing total of 245,704 data points.Another backtest, with a 3 years period:
2015-05-29 16:00:00 Algorithm Id:() completed in 143.27 seconds at 15k data points per second. Processing total of 2,193,810 data points.And your algorithm:
2005-03-31 16:00:00 Algorithm Id:() completed in 599.70 seconds at 12k data points per second. Processing total of 7,315,466 data points.So, in one hand, seems the engine is slower as the data points increases. In the other, your algo uses <7M data for just 3 months!
That's my two cents. JJ
0
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hi Juan,
Thanks for taking the time to look at this! Yeah, my algo does use a lot of data. That's why I was asking about optimizing it. I'm new to their API (and I haven't touched C# in 8-9 years!) so I wasn't sure if it was something wrong with my code or if there was something I could do to make it run faster without changing the core idea of the backtest. But if they are rolling out daily tickers that will be a huge help in terms of time and speed.
Jocelyn
Thanks for taking the time to look at this! Yeah, my algo does use a lot of data. That's why I was asking about optimizing it. I'm new to their API (and I haven't touched C# in 8-9 years!) so I wasn't sure if it was something wrong with my code or if there was something I could do to make it run faster without changing the core idea of the backtest. But if they are rolling out daily tickers that will be a huge help in terms of time and speed.
Jocelyn
1
Daily resolution has finished processing now. Since you're not using intraday data you can use Resolution.Daily or Resolution.Hour which should make those backtests much much faster :)
1
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
0
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Loading...
This discussion is closed
QuantConnect™ 2019. All Rights Reserved
