Changing resolution during backtest

I've realised I can't use option data in Daily resolution, so a new plan: Subscribe to the Minute data, then unsubscribe after the first slice. Resubscribe again later.

So now the question - how to I remove a subscription to the option and equity. I've added them with AddEquity and AddOption, but there looks like there is no equivalent "RemoveEquity" that I could find.

Note that I'm just testing at this point - at some stage I'd likely make this an Alpha that can take any Universe - so any insight into how to Add/Remove subscriptions in that model would be fantastic!

I've found self.RemoveSecurity, however it does not seem to have the desired affect.

I currently add the securities with AddEquity and AddOption. Removing them with:

self.RemoveSecurity(chain.Underlying.Symbol.Value)

Does not seem to work, and OnData still receives slices for those symbolsI did find this discussion, which has me leaning towards using a universe function. Any assistance or suggestions well appreciated (particularly how this might work as a stand alone alpha, where somebody else provides a custom universe from which I want to control the regularity of option data given to OnData.

You could try subscribing to Minute data, then consolidating the data into Daily. When the consolidated bar triggers something, set a flag to True then work with OnData.

I had set the code up to do something similar - subscribe to Minute data, and set a flag to True once a day to signal that I should work with OnData. This works, but the OnData event firing every Minute for each ticker still slows things down - even when all I do is check if the flag is True then return.

This discussion might be helpful:

how about using [history dataframe].unstack(level=0).resample('D').last()

That can work, but if subscribing to minutely data and returning is already too slow, calling history repeatedly will be even slower

One thing that seems to make things a little faster is simply checking the time. 

The below function is modified from somebody else - but it simply returns the current minute.

If I then call (for sake of argument)

The result is that the code will only execute on every thirtieth minute. You can do the same with hourly (in fact the function originally was "HourMinuteIs". This seems faster than other methods I have tried (for whatever reason!). I have a test algorithm using this currently - it subscribes to Minute options data, then uses MinuteIs to only both doing something every ten minutes. I will report back as I explore

The biggest slowdown would likely be subscribing to minutely data in the first place when you're not using it. RemoveSecurity() should work if the universe is defined manually by AddEquity() calls, then re-subscribing as you had tried. Are you using a Coarse-Fine Universe Selection?

Currently I'm using AddEquity followed by AddOption to manually add them. RemoveSecurity does not seem to work - OnData still receives events. I even set up a scheduled event each day to iterate through ActiveSecurities and RemoveSecurity for each. It still doesn't stop OnData receiving that data. More experimenting needed.

The Limit to specific times is a workaround for now, but the speed is challenging!

Hmm interesting.

Can you try printing out the securities in the UniverseManager before and after you call RemoveSecurity()?

IIRC, something like

At the moment it seems to output only null or zero length strings both before and after running RemoveSecurity (debug output has been trimmed by QC):

162 | 18:06:17: Universe 0: [, , ,

163 | 18:06:18: Universe 0: [, , ,

164 | 18:06:19: Universe 0: [, , ,

165 | 18:06:20: Universe 0: [, , ,

166 | 18:06:20: Universe 0: [, , ,

(nb, I had to make slight modifications to your code (removing the str() in defining security_ids) )

I will experiment more a little later and see what I can uncover

Hi Daniel, 

I couldn't reproduce your issue. I've attached a backtest that shows RemoveSecurity is working (see the Debug statements in your console). 

Could you please attach a backtest, too? I think that's the only way to get ahead here. The other option would be to contact the support directly. 

btw, RemoveSecurity is working for securities you've added via AddEquity as well as for securities you've added via manual UniverseSelection. 

A simplified backtest would be helpful as Arthur mentioned.

Made a typo in that above:

should be:

And try it with a smaller universe (a few assets) and maybe self.Log instead

I scrapped my earlier test algorithm and decided to strip it back to basics, starting from your example above, Arthur. I have determined that it works flawlessly when subscribing to equity data, but as soon as I subscribe to option data with AddOption, those subscriptions never get removed (depsite being enumerated by the removeSecurities function. 

For simplicity I've made this unsubscribe at 9:35 every day, and never resubscribe. The AddOption lines are commented out below, so you would need to remove the comment to see the failure mode compared.

Ok, I have managed to get this to work. It turns out that when you iterate ActiveSecurities, you enumerate the actual Options themselves. You need to instead call RemoveSecurity using the option symbol object. See the below modified backtest.

Would also love any ideas you may have about how best to structure this type of algorithm - it seems kind of janky to subscribe one minute and unsubscribe the next, but I'm not sure there's a much neater way

Hi Daniel,

When we use AddOption, an option universe is created for that security, which adds and removes options based on the filter. Using RemoveSecurity on the options managed by this universe will not work as intended. Instead, I suggest adding options through AddOptionContract using a scheduled event 5 minutes before market close, then using another scheduled event on market open and get the open interest and remove the unwanted contracts with RemoveSecurity.

Best,
Shile Wen

Thanks Shile,

I read somewhere that AddOptionContract does not support greeks, is that right? 

Disregard my earlier comment - I think I found the answer and will give AddOptionContract a try