Back

Machine Learning for Equity Price Trend Prediction

Coming from Python and being relatively new in C#, I thought it would be helpful to have an example strategy that utilizes the Accord.NET machine learning library. Since I couldn't find any good examples, I coded one myself.

This strategy trains a Linear SVM (Support Vector Machine) with historical returns. Then at the open of the market, it attempts to predict whether the market will close UP or DOWN. If the trend is predicted to be UP, we enter a LONG position. If the trend is DOWN, we exit the market. If we are already in a LONG position, we do nothing. As mentioned, I am quite new to C#, so there might be bugs and there is certainly room for improvement. Would be nice to discuss possible improvements/ideas here.
Update Backtest








Somewhat predictably, using a simpler normalized Bayesian algorithm performs acceptably to allow online re-learning of a set of adequate size. I imagine it's possible to optimize an svm to achieve better performance, but this is more a matter for the experts than for someone like myself that’s just dabbling to satisfy curiosity.

 

0

This paper has findings relevant to this topic:

http://www.jonathankinlay.com/Articles/ONE%20MILLION%20MODELS.pdf

In the study, they automated backtesting of a wide random selection of technical indicators and concluded that machine learning algorithms perform poorly for trend prediction on out of sample data and debunks papers with findings to the contrary as being subject to period fitting.

They do however, find that out of sample performance is superior for algorithms that focus on price, volume and order book data rather than technical indicator inputs.

0

      

Correction to previous post: the findings of the study were that volume was not very effective for predictions.

0

Thank you James, it's a nice and easy to read paper. IMO they're not using a good window period (1000 days) or sample weighting (uniform). Market rarely cares what happened 3 years ago...

0

      

I think the length of the training window may be to demonstrate that they have not period fit to a particular regime but that the model should be able to adapt to switches in trend.

I have this dilemma myself; that the length of the training window has a dramatic effect on results. My finding however is that the model tends to be struck with indecision with an overly long lookback.

Is there any consensus on methods to determine a valid range for the lookback period? I just rely on maximizing Sharpe.

0

I'm convinced one actually should fit to the most recent data and regime. They've certainly managed to show that the opposite didn't work for them, at least. My highly informal experience is that in the context of daily bars, looking more than a few months back is detrimental to most signals. Yeah, there won't be a lot of training data - that is a necessary problem, for if it was not, someone would already be detecting and trading away the pattern.

I'm not aware of what the consensus to determining lookback periods is (if any), I try to determine it using typical parameter optimization to maximize CAGR/MaxDD. I should add I'm not live trading any of the algos I've tried so I'm not an authority on the subject.

 

0

Awesome discussion!

First I’d like to focus on the million backtest paper. I really like the first part, where they test the use of SVM.

But I have serious doubts about the second part, the one where they run 1,000,000 backtests. 11Ants is a ML software specialized in customer loyalty. And the model ranking is based in a black box "combination of goodness of fit measures”. I find kind of disappointment the difference between the care and details if the first part with the bulk-I-have-no-idea-what-happens-here second part.

As many paper in this subject, the critical element is assumed to be correct, the backtesting. In this case there is a subtle detail that shows some looking ahead bias in the second part backtests. They standardized the data by “subtracting the in-sample mean and dividing by the standard deviation, also estimated from the in-sample data.” So, in the out-of-sample backtest, they should use the whole out-of-sample data to standardize the first observation. I’m not sure if this detail is significant but, in principle, shows that the experiment design has some flaws. Finally, I think everyone who uses ML techniques knows that in-sample performance is always better than out of sample.

Now respect to the training window, I recall a quote from the Ehlers’s book Cycle Analytics for Traders where he compares the prices to the meander of a river:

Viewed as an aerial photograph, every river in the world meanders. […] Every meander in a river is independent of other meanders, and thus all are completely random. If we were to look at all the meanders as an ensemble, overlaying one on top of the other like a multiple-exposure photograph, the meander randomness would also become apparent. […] However, if we are in a given meander, we are virtually certain of the general path of the river for a short distance downstream. The result is that the river can be described as having a short-term coherency but is random over the longer span.

River meanders are the kind of cycles we have in the market. We can measure and use these short-term cycles to our advantage if we realize they can come and go in the longer term.

Those are my two cents.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


I see what you mean about the two parts of the paper. They methodically demonstrate that most ML studies are not conscientious in their out of sample testing, which is a truism to the point of cliche. It may be that it's ill-advised trying to proceed from this kind of common knowledge towards an induction proof of the wider inadequacy of a whole breed of models; the problem arena is simply too vast to be captured by any set of experimental conditions. I'm curious what aspects of their experimental bias meant that only price data was an effective indicator. Sounds like you're suggesting that the cohort ranking is questionable, and the other thing out of place is the lookback period. It would still be interesting (if possible) to correct the bias of the experimental conditions in order to rank technical indicators by their predictive power.

"having a short-term coherency but is random over the longer span"

This is interesting as several of the ML papers I've come across conclude (from their in-sample testing) trend prediction is far more accurate on longer time scales than short. I suppose you could attribute this to the greater ease of curve fitting with aggregated bars over long timescales. This brings us back to random meandering: the effective strategies at short timescales proceed from the acceptance that price is analogous to a stochastic process which is coherent only in retrospect.

0

James Smith  you nailed it! That’s what I meant. But after a second review I realized the paper is from 2011, maybe then the insight was more informative than now.

Sounds like you're suggesting that the cohort ranking is questionable.

In some sense yes, is a black box. The authors said nothing about how the ranking is defined.

What continues is a bar conversation:

Respect to the short term coherency, I can’t find an article I saw time ago about the behavior of the prices at different time frames. I can’t even recall if the test was unit-root, randomness (kind of Kolmogorov-Smirnov) or returns autocorrelation. But the findings were that with lower time frames the prices show more coherency (or less randomness), thus is potentially more exploitable.

In the same line of reasoning, some days ago Patrick Star said in this post:

I was finally able to test some of my best performer algorithms with Minute and Second and I must say TA works just fine.

Finally,

…the effective strategies at short timescales proceed from the acceptance that price is analogous to a stochastic process which is coherent only in retrospect.

I love it! That’s why I think the best ML technique to explore is reinforcement learning, we need to teach a AI to safely gamble ;)

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Yes, again as I mentioned there, your biggest enemy is news (in general, news, events, reports, or any external happenings). In daily data there is a long period of time between two bars that can easily imact the price drastically while the market is not open. A conflict, a political issue, a diplomatic agreement, an election, passing a new law in other countries, a report, that your machine does not know about but they are significant to humans. There is no way for any ML algorithm to predict how impactful any of these events are to the next day's open price just by looking at the past. Especially when even two very experienced, well aware traders could interpret the same event in two completely different ways!

1

Hi,

@Gene Wildhart (and others), Can you implement your algo in Python?

Is there any example algo with machine learning in Python here at QC?

Are we able to import sk-learn library in Python?

Thanks :) 

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed