An interesting paper was published recently that drew the conclusion that most traders produce poor backtest optimizations compared to suitably trained machine learning classifiers (based on out of sample performance).

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2745220

I'm sure I'm not alone in developing too good to be true algorithms that turn out to be just that in production trading. I notice there is an AWS ML classifier service and I'm wondering whether it would be suitably adapted as a backtest classifier/optimizer? In general, does anyone with experience in this area have an estimate on the kind of time investment required to produce an ML classifier model suitable for backtesting? What level of re-use across an entire portfolio could be acheived? Would it require a customized model for each algorithm for instance?

Author