Back

Using machine learning in backtesting

Hello,

I have trained a sklearn svm regressor in the Research Environment.

Now I want to use this trained regressor in backtesting. How can I do it?

Thanks

Daniele

Update Backtest







Did you ever figure this out?

0

Sorry at this time it's not possible to save objects and retrieve those objects. It's a feature we've designed out and are almost done implementing but it is not ready quite yet sorry!

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks for quick reply! I'm new to QuantConnect -- is it possible to use train and use an ML algorithm within the standard algorithm framework versus the research framework? 

0

Yes, many people do this; training in warm-up etc and then running with the model in memory. The missing piece we're working on is saving that model so you don't need to run the training each time.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


May be what you can do is to persist your model into a text using pickle. I tested on Research and it works.Let me give you an example.

Research training phase:

import pickle
import codecs
import pandas as pd
from sklearn.svm import SVR

# Build the data
df = pd.DataFrame({'x': list(range(100)), 'y': list(range(100))})
df = df.sample(frac=1)
df_train = df.iloc[:70]

# Build the SVM regressor
model = SVR(kernel='linear')
model.fit(
df_train['x'].values.reshape(-1, 1),
df_train['y'].values.reshape(-1, 1)
)

# Test the SVM
print(model.predict([[1], [10], [100]]))

# Persist to text
pickle_content = codecs.encode(pickle.dumps(model), "base64").decode()
pickle_content

Last line will print the serialization in UTF. Then on backtests or another Research you have to load the pickle content. For example

import pickle
import codecs

pickle_content = 'gANjc2tsZWFybi5zdm0uY2xhc3NlcwpTVlIKcQApgXEBfXECKFgFAAAAX2ltcGxxA1gLAAAAZXBz\naWxvbl9zdnJxBFgGAAAAa2VybmVscQVYBgAAAGxpbmVhcnEGWAYAAABkZWdyZWVxB0sDWAUAAABn\nYW1tYXEIWAQAAABhdXRvcQlYBQAAAGNvZWYwcQpHAAAAAAAAAABYAwAAAHRvbHELRz9QYk3S8an8\nWAEAAABDcQxHP/AAAAAAAABYAgAAAG51cQ1HAAAAAAAAAABYBwAAAGVwc2lsb25xDkc/uZmZmZmZ\nmlgJAAAAc2hyaW5raW5ncQ+IWAsAAABwcm9iYWJpbGl0eXEQiVgKAAAAY2FjaGVfc2l6ZXERS8hY\nDAAAAGNsYXNzX3dlaWdodHESTlgHAAAAdmVyYm9zZXETiVgIAAAAbWF4X2l0ZXJxFEr/////WAwA\nAAByYW5kb21fc3RhdGVxFU5YBwAAAF9zcGFyc2VxFolYDQAAAGNsYXNzX3dlaWdodF9xF2NudW1w\neS5jb3JlLm11bHRpYXJyYXkKX3JlY29uc3RydWN0CnEYY251bXB5Cm5kYXJyYXkKcRlLAIVxGkMB\nYnEbh3EcUnEdKEsBSwCFcR5jbnVtcHkKZHR5cGUKcR9YAgAAAGY4cSBLAEsBh3EhUnEiKEsDWAEA\nAAA8cSNOTk5K/////0r/////SwB0cSRiiUMAcSV0cSZiWAYAAABfZ2FtbWFxJ0c/8AAAAAAAAFgI\nAAAAc3VwcG9ydF9xKGgYaBlLAIVxKWgbh3EqUnErKEsBSwKFcSxoH1gCAAAAaTRxLUsASwGHcS5S\ncS8oSwNoI05OTkr/////Sv////9LAHRxMGKJQwghAAAAIwAAAHExdHEyYlgQAAAAc3VwcG9ydF92\nZWN0b3JzX3EzaBhoGUsAhXE0aBuHcTVScTYoSwFLAksBhnE3aCKJQxAAAAAAAMBYQAAAAAAAAAAA\ncTh0cTliWAoAAABuX3N1cHBvcnRfcTpoGGgZSwCFcTtoG4dxPFJxPShLAUsChXE+aC+JQwj4y0L3\nXn8AAHE/dHFAYlgKAAAAZHVhbF9jb2VmX3FBaBhoGUsAhXFCaBuHcUNScUQoSwFLAUsChnFFaCKJ\nQxAr8rfEI6WEPyvyt8QjpYS/cUZ0cUdiWAoAAABpbnRlcmNlcHRfcUhoGGgZSwCFcUloG4dxSlJx\nSyhLAUsBhXFMaCKJQwjNmJmZmZm5P3FNdHFOYlgGAAAAcHJvYkFfcU9oGGgZSwCFcVBoG4dxUVJx\nUihLAUsAhXFTaCKJaCV0cVRiWAYAAABwcm9iQl9xVWgYaBlLAIVxVmgbh3FXUnFYKEsBSwCFcVlo\nIoloJXRxWmJYCwAAAGZpdF9zdGF0dXNfcVtLAFgKAAAAc2hhcGVfZml0X3FcS0ZLAYZxXVgLAAAA\nX2ludGVyY2VwdF9xXmgYaBlLAIVxX2gbh3FgUnFhKEsBSwGFcWJoIolDCM2YmZmZmbk/cWN0cWRi\nWAsAAABfZHVhbF9jb2VmX3FlaERYEAAAAF9za2xlYXJuX3ZlcnNpb25xZlgGAAAAMC4xOS4xcWd1\nYi4=\n'

# Load the model
model = pickle.loads(codecs.decode(pickle_content.encode(), "base64"))

# Test it
print(model.predict([[1], [10], [100]]))

Of course you cannot persist 100 MB ensemble, but except you are working with K-NN, most models don't weight too much. Most of my GBM models, which are boosting models, weights near 1 MB.

Hope it helps.

1

Can we actually save a trained model on an external source, for example a dropbox http url ?

0

You're right Adrien. May be you can store the Pickle serialization in dropbox and load it on backtest with self.Download method.

https://www.quantconnect.com/docs/algorithm-reference/importing-custom-data#Importing-Custom-Data-Loading-Reference-Data
1

Hi Adrien, you cannot export the serialization to dropbox unless it is a tiny volume of data (less than 10kb). Exporting data isn't allowed sadly due to our data licensing agreements that make it possible to provide the data. 

We are working on a way to save and fetch objects; it'll be ready soon. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed