Back

Varying backtest results

I have been trying to implement a momentum strategy. When I first deployed my first backtest about 2 days ago, the returns were relatively positive. My start date was 2018-8-1 and I did not set an end date (set to default) and this gave me 90% return over the period with a 100000 starting capital. 
The strategy ranks the top momentum stocks and longs them with a monthly rebalancement. Anyway, after I tweaked around a few parameters, the backtest results went way off and so I changed the entire code back to the original one (literally copy and pasted the same code). However, the results were completely different. I tried closing the project, creating the algorithm again and still the backtest results were completely different even though I was using the same original code. I went to look into the orders of the new backtest and the orders made were completely different even though I was using the same exact code. For example, in the original backtest with the positive results I would have bought Tesla a couple of times in 2020 but instead the new backtest results did not contain a single Tesla and I was buying completely random and different stocks.

 

Has anyone come across this problem before? Any help is appreciated! Thank you

Update Backtest







0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


I'm having this same exact problem. Backtest was doing great yesterday, running the same exact code today gives major errors and performs very differently.

After doing some research, it seems that the newest update (night of 25-8 to 26-8) caused some critical bug. The bug seems to be related to stock price data not being available in the "OnData" function when backtesting. 

1

I also have a problem with the OnData, it just never gets called in the backtest resulting in zero trades. Yesterday was fine and I changed nothing to the algorithm but today it is broken.

0

Yes indeed, sounds like the exact same problem. Could you post your algorithm (since this is your thread)? Then readers will be able to reproduce our errors.

 

0

Just for context, here is my backtest. It gives error messages about "The security does not have an accurate price as it has not yet received a bar of data." when executing "MarketOrder" for most stocks. The price is available in the data however and this same backtest did work yesterday.

0


This is my failed backtest result that is completely off to my original backtest result even though I executed the same code. 

0


Problem seems fixed for me in the last couple of hours. Didn't change anything in the code, yet the backtest performance is back to normal.

0

Hi there. Our cloud host rebooted part of our infrastructure. Generally please report issues like this to support@quantconnect.com as it needs to be reviewed by QC team

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Will do, thanks for the response!

0

So, I've now been running the same backtest, back to back, all day. Every time I run it, I get a vastly new result.

In the last few hours, it seems to have at least stabilized a bit, but still does not compare to the original backtest from yesterday evening. 

Does anyone have any insight into what is going on?

0


I'm also seeing inconsistent backtest results.  Live algorithms appear to be impacted too.  I'm getting unexpected (wrong) trade signals and errors on algorithms that have been live without issue for a few months now.  This seems like a major issue and we appreciate the attention of the QC team to develop a quick resolution.

1

Hi Tristan, Robert. There are no known issues and all tests are green. And for sure, nothing is impacting live trading -- live is literally a completely independent tech stack on physical servers we own, which cannot be rebooted (which was the issue from 3am PST 27th August). Please follow the support processes we have in place for tracking and debugging issues constructively. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed