QUANTCONNECT COMMUNITY
Hi,
I have an sklearn model that I trained on my local machine and uploaded to dropbox as a pickle object. Is there anyway to use it in my backtests on QC?
I tried downloading using
Share New Research 
response = urllib.urlopen("URL/to/model")
self.model = pickle.loads(response.read())But got the following error:
Build Warning: File: n/a Line:0 Column:0 - Sorry, 'open(' is not whitelisted within QuantConnect.
Remove this code statement 
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
QuantConnect™ 2023. All Rights Reserved
|
|
Derek Tishler
A robust way to work with such files for models is on the feature list(help welcome from us community members). I'm unsure if there is a big security issue with using pickled files from outside the cloud(such as locally trained Sklearn or Deap models), or even TensorFlow's binary checkpoint files. However, saving a model from a Research notebook, Backtest, or Live algo to ensure a persistable model will be great as the financial data exists in the QC cloud.
Ahmad Emad
I think something like this is allowed in C# (if I not totally reading this wrong):
https://www.quantconnect.com/forum/discussion/385/uploading-data-files
I was hoping there would be some equivalent code to this (as mentioned on the other discussion):
var client = new WebClient(); var json = client.DownloadString (address); var customObject = JsonConvert.DeserializeObject(json);Derek Tishler
Along those lines, you may have luck manually working with & iimporting your sklean model in json format:
*I have not tested this, I just came across this url when trying to save to json vs pickle(only way suggested in sklearn docs).
Ahmad Emad
Yeah, I could do that. But that still doesn't solve the problem of accessing a web url from within python.
Jared Broad
Hi Ahmad; We've got a feature coming soon to replace fetcher. Its not merged yet but we'll deploy it this week.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Derek Tishler
The only way I know of accessing outside data via python is with the custom data importing from docs which is a bit heavy handed vs the quick json downlload mentioned in that other post. Perhaps others can suggest more ideas.
Alexandre Catarino
QCAlgorithm has the Download method that accepts an url and returns the data from that url. Here is an example.
The QuantConnect team hasn't tackled the issue #1040 that implemensts a virtual file system or object storage yet.
Xiaolei
I am facing the same problem right now, Wondering has anyone sucessfully load trained sklearn model yet?Â
Link Liang
Hi Xiaolei,
Thank you for your report, we are installing new data protection technology and dropbox downloading will start working again soon
Laurent Crouzet
Hi. Do you have a date for the modified "Download()" to work again, Link Liang ?
Link Liang
Hi Laurent,
The Download() method is available to use now. Try it out!
Lexx7
Does it work in research?
Â
Daniel Chen
Hi Lexx7,
Sorry, we cannot fetch external data in Research so far. Please visit our Research documentation to take advantage of all the functions we already provide. We hope you can enjoy performing exploratory research in the Research environment. If you have further request, please let us know. Thank you!
Ahmad Emad
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!