Back

Same algorithm different results during back testing

I'm running a futures algorithm with Renko Consolidator and keep running into an issue where the same algorithm produces different results when back-test over the same period. My expectation is the results to be the same if I rerun the same algorithm twice with no code change. 

Update Backtest







0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Bo,

Could you attach a backtest? It will make it easier for us to identify the problem. For now, here are some ideas for the differing results:

  • The algorithm is nondeterministic, one example is it uses a Machine Learning model that have random initial states/parameters
  • The end date isn’t set (likely not the problem)

Best,
Shile Wen

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


I'vee opened up a support ticket since it seems a few bugs in the framework. However, before we close this discussion out. 

How is the framework handling orders executions in backtesting? The algorithm is deterministic in my specific case. So I would expect to see the same results given that the same data coming in. No different than the framework unit test work.

If the framework is handling order execution via a happy path (i.e. immediate full order) then the results should be the same.

1

Hi Bo Vargas ,

Yes, the results should be the same since the backtest should be deterministic.
We haven't had any report on this issue (non-deterministic backtest) so we need to take a look at the algorithm you shared with support@quantconect.com in particular.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed