Back

Machine Learning in Python?

Hi all,

At QC, can we import sk-learn and other libraries that we need to implement Machine Learning based algorithms in Python?  And can anyone attach some example algos of ML here? Thanks :) 

Update Backtest








We can find the white-listed libraries at the FAQ:
"The following python libraries are white-listed for use on QuantConnect:: numpy, pandas, scipy, sklearn, blaze, cvxopt, cvxpy, pykalman, statsmodels, statistics, copulalib, keras, theano, stats, tensorflow and xgboost."
QuantConnect team hasn't developped a ML algorithm yet, but you can find a research example here.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


One note of caution: ML algorithms usually require quite a bit of data to fit the model. Using Schedule function, one can do this as infrequently as needed. However, in my experience, the RAM consumption is _cumulative_: if you fit a model once a month using 5000 observations on something like 60 variables, you will run out of RAM after some time. If you fit the same model on 10000 observations, you will run out of RAM in half the time.

Alexandre Catarino could you suggest on how a memory can be force-freed in order to prevent this from happening - maybe it is not a problem in C#, but it certainly is in Python

0

Hey Karen, please send through an example we can use to debug to support and we'll make sure it gets fixed =)

Would it happen with a simple array? e.g. Creating a new vector of 1M random items each OnData?

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Yes, here is the example I cooked up: the algo is using Schedule.On to

evaluate a signal 16 times a day (every 15 minutes starting at 11:40am). On

each evaluation, it request a history of 128 minute bars.



https://www.quantconnect.com/terminal/processCache?request=embedded_backtest_4d37b562c0c7f5e7823d68cf86c4a9bb.html



Runs out of memory sometime mid-test.
1

Karen Chaltikian just an update; changes to our core technology in the backend have addressed this issue. You can now complete the backtest =)

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed