Hi everyone, 

I noticed something strange. I get different backtest results for identical algorithms. Start Date and End Date are fixed, so that's not the reason here. 

At first, I suspected that there was a bug somewhere in my code, which leads to a non-deterministic behaviour of the algorithm. But I couldn't find anything. Then I found out that sometimes NaN values appeared for particular securities in my dataframes. So the algorithm couldn't find any data for this security. This in itself is not a big deal and could be intercepted by a "data.ContainsKey(symbol)" for example. What bothers me more is that it finds data one time and another time it doesn't. As if it depends on coincidence... It looks to me a little bit like a non-deterministic behaviour is created somewhere under the hood. And I don't like that at all because it would undermine confidence in QC as a platform for backtesting and live trading. 

We obviously expect that the same algorithm will always produce the same results. If this is not the case, then backtest results lose all meaning. 

I don't want to share the original code here because it's intended for the alpha streams marketplace. But I created a minimalistic example with the security "INTU" which reproduces the error. 

Please perform multiple backtests to get different results. For me this was usually the case after 3-5 runs. 

I really hope that I just missed something and that there is a simple explanation for it. 

I am grateful for any support. 

Best regards, 

Arthur

Author