Back

Revisit Allowing System.IO.MemoryStream

Last September someone wanted to import custom data, and because of security restrictions, QuantConnect does not allow System.IO. I would like to revisit that restriction please.

I am writing an Alorithm for Abhi Goal, a fellow member, and he wants to read in some custom data upon which to base his trades. Call it a custom indicator. QC allows me to get data from an http:// call using WebClient and I would like to process the return data as a Stream. I would like to use StreamReader on the return from the call, but I could live with putting the returned csv data into a String and reading it into a MemoryStream and pass the MemoryStream to my parsing class. Streams are nice because I can grab a line at a time with one call.

I suppose could rewrite the parser to get the lines from the string variable, but that is painful what with having to parse for newlines and substring the result.

Thanks,

Nicholas Stein
Update Backtest






The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



Hey Nicholas,

If the custom data is string csv data then I would recommend just defining custom data type. Have a peak at the Bitcoin implementation available in github. If you use this pattern you don't need to download the file at all, the engine will take care of all of that.

Is there another reason as to why you'd like to control the downloading of the file?
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


It is not market data, it is a custom indicator that is generated by a proprietary source somewhere. However there is a date column and I might be able to gin up a bitcoin-like object.
0

It is not market data, it is a custom indicator that is generated by a proprietary source somewhere. However there is a date column and I might be able to gin up a bitcoin-like object.
0

The custom data is csv data with a date column, but it is not market data but an indicator. He wants QC to do some logic and generate the order for backtest. The data is not regularly spaced in time though. There are only a 800 daily signals since 2010.

Thanks for the idea, I will try your approach and look at quandl too.. Maybe I could use a consolidator to combine the source with a trade bar?
0

Even if it's irregularly spaced you can still use it. Create the custom data type MyIndicator and add it using:
AddData("MyIndicator", Resolution.Daily);
It doesn't need to be market data, we have examples that import weather data! The defaults for AddData< T > set fill forward=false, so you'll receive data in your OnData(MyIndicator data) only when there's new data on the irregular spacing. When that method fires, place your trade. It should be pretty simple. If you'd like more help, share the csv format.

I wouldn't try to lift that data into a QC indicator type or try and merge it with the trade bar information. I think that may be unneeded complexity.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


@MichaelH I implemented your approach and it gets the custom data both from a local file during debug and from the url during live. The code is much cleaner and I love the approach. Thank you.

When the OnData fires, the Security I am trying to trade has not been added to the Portfolio.Securities collection. The only Security I have in there is the one that was added when I did.
AddData("ABHIGOAL", Resolution.Daily);
It looks like the only place I can add the Security is in the Initialize function. In the OnData handler tried adding the Security using
AddSecurity(SecurityType.Equity, symbol, Resolution.Daily);
but it did not add it. Is there any way to add a Security after Initializing the algo and it is running? When I read the whole list in during Initialize, I was able to loop through and add all the symbols.

Of course I do not know the symbols until I get the data.

Thanks,

Nick
0

There's currently no way to add symbols during run time. On our development road map is a feature we call universe selection. This is the ability to select your symbols based on some conditions. When this feature is available (should be pretty soon) you'll be able to request those data streams while the algorithm is running.

For your current testing, I would recommend limiting the orders placed to a set of symbols that are initialized in the Initialize() method.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed