Hi,
I have an sklearn model that I trained on my local machine and uploaded to dropbox as a pickle object. Is there anyway to use it in my backtests on QC?
I tried downloading using
response = urllib.urlopen("URL/to/model")
self.model = pickle.loads(response.read())But got the following error:
Build Warning: File: n/a Line:0 Column:0 - Sorry, 'open(' is not whitelisted within QuantConnect.
Remove this code statementThe material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
A robust way to work with such files for models is on the feature list(help welcome from us community members). I'm unsure if there is a big security issue with using pickled files from outside the cloud(such as locally trained Sklearn or Deap models), or even TensorFlow's binary checkpoint files. However, saving a model from a Research notebook, Backtest, or Live algo to ensure a persistable model will be great as the financial data exists in the QC cloud.
I think something like this is allowed in C# (if I not totally reading this wrong):
https://www.quantconnect.com/forum/discussion/385/uploading-data-files
I was hoping there would be some equivalent code to this (as mentioned on the other discussion):
var client = new WebClient();
var json = client.DownloadString (address);
var customObject = JsonConvert.DeserializeObject(json);Along those lines, you may have luck manually working with & iimporting your sklean model in json format:
*I have not tested this, I just came across this url when trying to save to json vs pickle(only way suggested in sklearn docs).
Yeah, I could do that. But that still doesn't solve the problem of accessing a web url from within python.
Hi Ahmad; We've got a feature coming soon to replace fetcher. Its not merged yet but we'll deploy it this week.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
The only way I know of accessing outside data via python is with the custom data importing from docs which is a bit heavy handed vs the quick json downlload mentioned in that other post. Perhaps others can suggest more ideas.
QCAlgorithm has the Download method that accepts an url and returns the data from that url. Here is an example.
The QuantConnect team hasn't tackled the issue #1040 that implemensts a virtual file system or object storage yet.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
I am facing the same problem right now, Wondering has anyone sucessfully load trained sklearn model yet?Â
Hi Xiaolei,
Thank you for your report, we are installing new data protection technology and dropbox downloading will start working again soon
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hi. Do you have a date for the modified "Download()" to work again, Link Liang ?
Hi Laurent,
The Download() method is available to use now. Try it out!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Does it work in research?
Â
Hi Lexx7,
Sorry, we cannot fetch external data in Research so far. Please visit our Research documentation to take advantage of all the functions we already provide. We hope you can enjoy performing exploratory research in the Research environment. If you have further request, please let us know. Thank you!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
|