In the latest episode of Better System Trader, the guest, Dave Walton, made a compelling case arguing AGAINST some of the most popular backtesting strategies:  Specifically, out-of-sample testing, walk-forward analysis, and Monte Carlo analysis.  He states these approaches are a very poor indicator of future results.  He advocates instead for something he calls System Parameter Permutation.  Basically, it's a way to model many possible permutations of the same system but with a range of different parameter values for its rules (ie. entry and exit rules).  Then, by looking at all of the possible outcomes, you can draw better conclusions on the probability of how your strategy will work in the future.  

You can listen to the podcast and read a much more detailed explaination here:
http://bettersystemtrader.com/051-dave-walton/

So, as a community, what do all of you think?  Does he make some valid points?  Most importantly, is this something that can be done with QuantConnect?  What about the Lean Engine?  If not, are there other platforms that offer this type of analysis? 

Author