Creating rolling windows for multiple stocks by Christian Lauer

Christian Lauer INVESTOR

| Reward Discussion

Hello everyone,

thanks to all of your help I got my first simple algorithm working. Now I'm thinking of modifying it, so I can trade more than one stock. But I don't know how to create rolling windows for multiple stocks. I thought I'd use a list and iterate through it, but then I think I'd overwrite the same rolling window again and again. So how is this correctly done? This is the part of my code I'm unsure about:

         self.AddEquity("SPY", Resolution.Second)



         consolidator_daily = TradeBarConsolidator(timedelta(1))

         consolidator_daily.DataConsolidated += self.OnDailyData

         self.SubscriptionManager.AddConsolidator("SPY", consolidator_daily)



         consolidator_minute = TradeBarConsolidator(60)

         consolidator_minute.DataConsolidated += self.OnMinuteData

         self.SubscriptionManager.AddConsolidator("SPY", consolidator_minute)



         self.daily_rw = RollingWindow[TradeBar](2)

         self.minute_rw = RollingWindow[TradeBar](2)

         self.window = RollingWindow[TradeBar](2)



         self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.AfterMarketOpen('SPY', 1),

                         Action(self.one_minute_after_open_market))



         self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.BeforeMarketClose('SPY', 1),

                         Action(self.before_close_market))



    # Add daily bar to daily rolling window

    def OnDailyData(self, sender, bar):

        self.daily_rw.Add(bar)



    def OnMinuteData(self, sender, bar):

        self.minute_rw.Add(bar)

Thank you very much for your help.

Kind regards,

Christian Lauer

Update Backtest

Christian Lauer

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Alexandre Catarino

Pro ,

In this scenario, we often advice users to create a class with the common members (rolling windows) and use a dictionary to access the objects:

# In Initialize

self.symbolData = dict()



for ticker in ["SPY", "FB", "TWTR"]:

    symbol = AddEquity(ticker).Symbol   

    # Consolidators part...

    self.symbolData[symbol] = SymbolData()

where SymbolData holds the rolling windows:

class SymbolData(object):	

    def __init__(self):

        self.daily_rw = RollingWindow[TradeBar](2)

        self.minute_rw = RollingWindow[TradeBar](2)

        self.window = RollingWindow[TradeBar](2)

Here is a simplified version of your algorithm with the mechanics on using the dictionaris to update the rolling windows.

Christian Lauer

2k Pro ,

Hello Alexandre,

thank you very much for your help. I've tried to modify my algorithm, but I must have done something wrong. Now I get

AttributeError : 'dict' object has no attribute 'window'

Kind regards,

Christian Lauer

from QuantConnect.Data.Market import TradeBar

from datetime import timedelta

from System import *

from QuantConnect import *

from QuantConnect.Algorithm import *

from QuantConnect.Indicators import *

import decimal as d





class MyAlgorithm(QCAlgorithm):

	

    def Initialize(self):

        self.SetStartDate(2013, 05, 1)  # Set Start Date

        self.SetEndDate(2013, 06, 01)

        self.SetCash(100000)  # Set Strategy Cash

        

        self.symbolData = dict()

        

        for ticker in ["SPY", "FB", "TWTR"]:

			symbol = self.AddEquity(ticker, Resolution.Second).Symbol



			consolidator_daily = TradeBarConsolidator(timedelta(1))

			consolidator_daily.DataConsolidated += self.OnDailyData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_daily)



			consolidator_minute = TradeBarConsolidator(60)

			consolidator_minute.DataConsolidated += self.OnMinuteData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_minute)

			

			self.symbolData[symbol] = SymbolData()



        self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.AfterMarketOpen('SPY', 1),

                         Action(self.one_minute_after_open_market))

                         

        self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.BeforeMarketClose('SPY', 1),

                         Action(self.before_close_market))



    # Add daily bar to daily rolling window

    def OnDailyData(self, sender, bar):

        self.symbolData[bar.Symbol].daily_rw.Add(bar)



    def OnMinuteData(self, sender, bar):

        self.symbolData[bar.Symbol].minute_rw.Add(bar)



    def one_minute_after_open_market(self):

        """

        At 9:31 check if there has been a gap at the market open from the previous day.

        If so and the stock is gapping up and the first minute bar is negative, create a short selling signal.

        If the stock is gapping down and the first minute bar is positive, create a buying signal.

        """



        if not (self.symbolData.window.IsReady and self.symbolData.daily_rw.IsReady and self.symbolData.minute_rw.IsReady): return

        for k in self.symbolData: 

         last_close = self.symbolData[k].window[0].Close

         yesterday_daily_close = self.symbolData[k].daily_rw[1].Close

         first_minute_close = self.symbolData[k].minute_rw[1].Close

         first_minute_open = self.symbolData[k].minute_rw[1].Open

        

         gap = last_close - yesterday_daily_close

         first_minute_bar = first_minute_close - first_minute_open



         if not self.Portfolio["SPY"].Invested:

            # If the stock is gapping down and the first minute bar is positive, create a buying signal.

            if gap < 0 and first_minute_bar > 0:

                self.SetHoldings("SPY", 0.333)

                self.Log('GOING LONG')

            # If the stock is gapping up and the first minute bar is negative, create a short selling signal

            elif gap > 0 and first_minute_bar < 0:

                self.SetHoldings("SPY", -0.333)

                self.Log('GOING SHORT')



    def before_close_market(self):

    	"""

    	At the end of the day, if there is a short position, close it.

    	"""

    	for stock in self.Portfolio.Invested:

            self.Liquidate(stock)

            self.Log('LIQUIDATE SHORT End of Day')





    def OnData(self, data):

        if data["SPY"] is None:

            return

        if not (self.symbolData.window.IsReady):

            return

        for stock in Portfolio.Invested:

         factor = d.Decimal(1.01)

         currBar = self.window[0].Close

        

        # Every second, check the price and if it's higher than the price the stock was bought for times 1.01, close the position.

         if self.Portfolio[stock].AveragePrice * factor < currBar:

            self.Liquidate(stock)

            self.Log('LIQUIDATE AT THRESHOLD REACHED.')

    

    def OnEndOfDay(self):

    	self.Plot("Portfolio", "MarginRemaining", self.Portfolio.MarginRemaining)

    

    def OnEndOfAlgorithm(self):

    	self.Liquidate()

    	self.Log('LIQUIDATE AT End Of Algorithm.')

    	

class SymbolData(object):

    		

    def __init__(self):

		self.daily_rw = RollingWindow[TradeBar](2)

		self.minute_rw = RollingWindow[TradeBar](2)

		self.window = RollingWindow[TradeBar](2)

Alexandre Catarino

Pro ,

You need to substitute all the variables that refer to the past rolling window (e.g.: self.window) for the rolling window in the SymbolData (e.g.: self.symbolData[symbol].window).

By the way, please attach the backtest instead of copy&paste the code, it is more practical.
Use code snippets for explaining the issue.

Christian Lauer

2k Pro ,

Hello Alexandre,

thank you for your answer. There really was one self.window which I had forgot to substitute. Still I somehow get the same error when backtesting. As far as attaching the backtests is concerned, I'd really like to do that but sometimes it's not working. I think this is because the bug occurs so early that there is nothing to run and thus no backtest. I'm sorry to do it again.

Kind regards,

Christian Lauer

from QuantConnect.Data.Market import TradeBar

from datetime import timedelta

from System import *

from QuantConnect import *

from QuantConnect.Algorithm import *

from QuantConnect.Indicators import *

import decimal as d





class MyAlgorithm(QCAlgorithm):

	

    def Initialize(self):

        self.SetStartDate(2013, 05, 1)  # Set Start Date

        self.SetEndDate(2013, 06, 01)

        self.SetCash(100000)  # Set Strategy Cash

        

        self.symbolData = dict()

        

        for ticker in ["SPY", "FB", "TWTR"]:

			symbol = self.AddEquity(ticker, Resolution.Second).Symbol



			consolidator_daily = TradeBarConsolidator(timedelta(1))

			consolidator_daily.DataConsolidated += self.OnDailyData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_daily)



			consolidator_minute = TradeBarConsolidator(60)

			consolidator_minute.DataConsolidated += self.OnMinuteData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_minute)

			

			self.symbolData[symbol] = SymbolData()



        self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.AfterMarketOpen('SPY', 1),

                         Action(self.one_minute_after_open_market))

                         

        self.Schedule.On(self.DateRules.EveryDay(),

                         self.TimeRules.BeforeMarketClose('SPY', 1),

                         Action(self.before_close_market))



    # Add daily bar to daily rolling window

    def OnDailyData(self, sender, bar):

        self.symbolData[bar.Symbol].daily_rw.Add(bar)



    def OnMinuteData(self, sender, bar):

        self.symbolData[bar.Symbol].minute_rw.Add(bar)



    def one_minute_after_open_market(self):

        """

        At 9:31 check if there has been a gap at the market open from the previous day.

        If so and the stock is gapping up and the first minute bar is negative, create a short selling signal.

        If the stock is gapping down and the first minute bar is positive, create a buying signal.

        """



        if not (self.symbolData.window.IsReady and self.symbolData.daily_rw.IsReady and self.symbolData.minute_rw.IsReady): return

        for k in self.symbolData: 

         last_close = self.symbolData[k].window[0].Close

         yesterday_daily_close = self.symbolData[k].daily_rw[1].Close

         first_minute_close = self.symbolData[k].minute_rw[1].Close

         first_minute_open = self.symbolData[k].minute_rw[1].Open

        

         gap = last_close - yesterday_daily_close

         first_minute_bar = first_minute_close - first_minute_open



         if not self.Portfolio["SPY"].Invested:

            # If the stock is gapping down and the first minute bar is positive, create a buying signal.

            if gap < 0 and first_minute_bar > 0:

                self.SetHoldings("SPY", 0.333)

                self.Log('GOING LONG')

            # If the stock is gapping up and the first minute bar is negative, create a short selling signal

            elif gap > 0 and first_minute_bar < 0:

                self.SetHoldings("SPY", -0.333)

                self.Log('GOING SHORT')



    def before_close_market(self):

    	"""

    	At the end of the day, if there is a short position, close it.

    	"""

    	for stock in self.Portfolio.Invested:

            self.Liquidate(stock)

            self.Log('LIQUIDATE SHORT End of Day')





    def OnData(self, data):

        if data["SPY"] is None:

            return

        if not (self.symbolData.window.IsReady):

            return

        for stock in Portfolio.Invested:

         factor = d.Decimal(1.01)

         currBar = self.symbolData[stock].window[0].Close

        

        # Every second, check the price and if it's higher than the price the stock was bought for times 1.01, close the position.

         if self.Portfolio[stock].AveragePrice * factor < currBar:

            self.Liquidate(stock)

            self.Log('LIQUIDATE AT THRESHOLD REACHED.')

    

    def OnEndOfDay(self):

    	self.Plot("Portfolio", "MarginRemaining", self.Portfolio.MarginRemaining)

    

    def OnEndOfAlgorithm(self):

    	self.Liquidate()

    	self.Log('LIQUIDATE AT End Of Algorithm.')

    	

class SymbolData(object):

    		

    def __init__(self):

		self.daily_rw = RollingWindow[TradeBar](2)

		self.minute_rw = RollingWindow[TradeBar](2)

		self.window = RollingWindow[TradeBar](2)

Alexandre Catarino

Pro ,

Please look at the pattern:
self.daily_rw became self.symbolData[bar.Symbol].daily_rw
self.minute_rw became self.symbolData[bar.Symbol].minute_rw
In your code, there are still self.symbolData.window where there is no symbol to access the value in the dictionary.

Christian Lauer

2k Pro ,

Hello Alexandre,

thank you for your answer. The backtest is now running. Now I got the problem that the algorithm either is not trading anything (like in the attached backtest, I think this is because my windows aren't ready somehow) or there is a bug and it says:

Runtime Error: Python.Runtime.PythonException: KeyNotFoundException : 'SPY' wasn't found in the Slice object, likely because there was no-data at this moment in time and it wasn't possible to fillforward historical data. Please check the data exists before accessing it with data.ContainsKey("SPY") at QuantConnect.Data.Slice.get_Item (QuantConnect.Symbol symbol) [0x0002d] in <fd76fb15bc8d4db9b879580b87c41291>:0 at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in <dca3b561b8ad4f9fb10141d81b39ff45>:0 at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in <14452d8e618b4a588d1da22cca9cdbfe>:0 at Python.Runtime.PyObject.InvokeMethod (System.String name, Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00007] in <14452d8e618b4a588d1da22cca9cdbfe>:0 at Python.Runtime.PyObject.TryInvokeMember (System.Dynamic.InvokeMemberBinder binder, System.Object[] args, System.Object& result) [0x0003e] in <14452d8e618b4a588d1da22cca9cdbfe>:0 at (wrapper dynamic-method) System.Object:CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object,QuantConnect.Data.Slice) at QuantConnect.AlgorithmFactory.Python.Wrappers.AlgorithmPythonWrapper.OnData (QuantConnect.Data.Slice slice) [0x000c6] in <9189cbd909fc4acdb7506d0665715861>:0 at QuantConnect.Lean.Engine.AlgorithmManager.Run (QuantConnect.Packets.AlgorithmNodePacket job, QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.IDataFeed feed, QuantConnect.Lean.Engine.TransactionHandlers.ITransactionHandler transactions, QuantConnect.Lean.Engine.Results.IResultHandler results, QuantConnect.Lean.Engine.RealTime.IRealTimeHandler realtime, QuantConnect.Lean.Engine.Server.ILeanManager leanManager, System.Threading.CancellationToken token) [0x012d0] in <3e70d397bd794c0f8471ddd804025edf>:0 (Open Stacktrace)

Kind regards,

Christian Lauer

2k Pro ,

Despite the problem of my algorithm not working I also think I could improve the code at some places. For example, I think it would be better to iterate through the positions in my portfolio for checking if the closing conditions are met instead of iterating through all the stocks I want to trade and checking if they're are invested. How is that done? Also I'd be really glad for help with the general problem.

Kind regards,

Christian Lauer

Alexandre Catarino

Pro ,

We can iterate the Portfolio object:

for holding in self.Portfolio:

     if holding.Invested:

         pass

     else:

         pass

     symbol = holding.Symbol

Please checkout the docs for the more information about the Portfolio object.

In the attached backtest, you can find a working example of your algorithm.

Christian Lauer

2k Pro ,

from QuantConnect.Data.Market import TradeBar

from datetime import timedelta

from System import *

from QuantConnect import *

from QuantConnect.Algorithm import *

from QuantConnect.Indicators import *

import decimal as d



class MyAlgorithm(QCAlgorithm):

	

    def Initialize(self):

		self.SetStartDate(2017, 9, 11)  # Set Start Date

		self.SetEndDate(2017, 9, 13)

		self.SetCash(100000)  # Set Strategy Cash

        

		self.symbolData = dict()

        

		for ticker in ["SPY", "AAPL", "NVDA", "FB", "BABA", "MU", "MYL", "JD", "TEVA", "TVIX", "AMD"]:

			symbol = self.AddEquity(ticker, Resolution.Second).Symbol



			consolidator_daily = TradeBarConsolidator(timedelta(1))

			consolidator_daily.DataConsolidated += self.OnDailyData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_daily)



			consolidator_minute = TradeBarConsolidator(60)

			consolidator_minute.DataConsolidated += self.OnMinuteData

			self.SubscriptionManager.AddConsolidator(symbol, consolidator_minute)

			

			self.symbolData[symbol] = SymbolData()



        #self.Schedule.On(self.DateRules.EveryDay(),

        #                 self.TimeRules.AfterMarketOpen('SPY', 1),

        #                 Action(self.one_minute_after_open_market))

                         

        #self.Schedule.On(self.DateRules.EveryDay(),

        #                 self.TimeRules.AfterMarketOpen('SPY', 5),

        #                 Action(self.before_close_market))

        

		#self.Schedule.On(self.DateRules.EveryDay(), 

		#				self.TimeRules.Every(timedelta(seconds=10)), 

		#				Action(self.Tensec))



    # Add daily bar to daily rolling window

    def OnDailyData(self, sender, bar):

		self.symbolData[bar.Symbol].daily_rw.Add(bar)

		

    def OnMinuteData(self, sender, bar):

		self.symbolData[bar.Symbol].minute_rw.Add(bar)



    def OnData(self, data):

		for symbol in data.Keys:

			self.Log(str(symbol))

			if data[symbol] is None: continue

    	    # Create local variable to readability

			window = self.symbolData[symbol].window

    		# Update the window. If not ready, continue

			window.Add(data[symbol])

			minute = self.symbolData[symbol].minute_rw

			if not (window.IsReady and minute.IsReady): continue

    		

    		if self.Portfolio[symbol].Invested:

				last_close = window[0].Close

				last_min_open = minute[1].Open

				last_min_close = minute[1].Close

				last_bar = minute[1].Close - minute[1].Open

				second_bar = minute[2].Close - minute[2].Open

				fourth_bar = minute[4].Close - minute[4].Open

				self.Log(str(symbol)+str(last_min_open)+", "+str(last_close))

				if (last_close > last_min_close):

					self.Log(str(last_min_open)+", "+str(last_close))

					self.Liquidate(symbol)

					self.Log('LIQUIDATE AT THRESHOLD REACHED.')

    		

    		if not self.Portfolio[symbol].Invested:

				last_bar = minute[1].Close - minute[1].Open

				second_bar = minute[2].Close - minute[2].Open

				third_bar = minute[3].Close - minute[3].Open

				fourth_bar = minute[4].Close - minute[4].Open

				last_close = window[0].Close

				last_min_open = minute[1].Open

				last_min_close = minute[1].Close

				#self.Log(str(last_bar)+", "+", "+str(second_bar)+", "+str(third_bar)+", "+str(last_close)+", "+str(minute[0].Open))

				if (last_bar < 0 and last_min_close - last_close > d.Decimal(0.05)):  

					self.SetHoldings(symbol, -1.0/3.0)

					self.Log('shorting')

					

    #def Tensec(self):

    	

    #	for symbol in self.symbolData:

    	    # Create local variable to readability

	#		window = self.symbolData[symbol].window

    		# Update the window. If not ready, continue

	#		minute = self.symbolData[symbol].minute_rw

	#		if not (window.IsReady and minute.IsReady): continue	

    

    def OnEndOfDay(self):

		self.Plot("Portfolio", "MarginRemaining", self.Portfolio.MarginRemaining)

    

    def OnEndOfAlgorithm(self):

		self.Liquidate()

		self.Log('LIQUIDATE AT End Of Algorithm.')

    	

class SymbolData(object):

    		

    def __init__(self):

		self.daily_rw = RollingWindow[TradeBar](5)

		self.minute_rw = RollingWindow[TradeBar](5)

		self.window = RollingWindow[TradeBar](5)

Hello Alexandre,

thank you for your help again. Thanks to you I finally got to the point where I can test things without getting errors every 10 minutes. I still got a new problem though. This next algorithm I'm working on is simpler. It should iterate through a list of stock every second. If not invested and the last minute bar was negative and the difference between the current price and the last minute's closing price is more than 0.05, it should make a short position. If the stock is invested and the last price is bigger than the last minute's close, it should close the position. This has been working if I iterate in OnData for the rolling windows and in another method which I had called every ten seconds for making the positions. When I put it all in OnData I get the following error:

Runtime Error: Python.Runtime.PythonException: ArgumentOutOfRangeException : Must be between 0 and -1 Parameter name: i Actual value was 1. at QuantConnect.Indicators.RollingWindow`1[T].get_Item (System.Int32 i) [0x0004c] in <38253e1f0adf48debed06f8d2e7759e5>:0 at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in <dca3b561b8ad4f9fb10141d81b39ff45>:0 at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in <387056c9810b431d9b668f2df5d6c027>:0 at Python.Runtime.PyObject.InvokeMethod (System.String name, Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00007] in <387056c9810b431d9b668f2df5d6c027>:0 at Python.Runtime.PyObject.TryInvokeMember (System.Dynamic.InvokeMemberBinder binder, System.Object[] args, System.Object& result) [0x0003e] in <387056c9810b431d9b668f2df5d6c027>:0 at (wrapper dynamic-method) System.Object:CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object,QuantConnect.Data.Slice) at System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2[T0,T1] (System.Runtime.CompilerServices.CallSite site, T0 arg0, T1 arg1) [0x00128] in <63992662b765477a898ef49cdcc99ee2>:0 at QuantConnect.AlgorithmFactory.Python.Wrappers.AlgorithmPythonWrapper.OnData (QuantConnect.Data.Slice slice) [0x000c6] in <a4a88fb3668b4590893c272615df630f>:0 at QuantConnect.Lean.Engine.AlgorithmManager.Run (QuantConnect.Packets.AlgorithmNodePacket job, QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.IDataFeed feed, QuantConnect.Lean.Engine.TransactionHandlers.ITransactionHandler transactions, QuantConnect.Lean.Engine.Results.IResultHandler results, QuantConnect.Lean.Engine.RealTime.IRealTimeHandler realtime, QuantConnect.Lean.Engine.Server.ILeanManager leanManager, System.Threading.CancellationToken token) [0x012d0] in <b6042ddb9cf849bda5636f8ee179370d>:0 (Open Stacktrace)

Also I've noticed that the algorithm is crashing if I remove the SPY from the stock list. Why is that?

Kind regards,

Christian Lauer

2k Pro ,

I'd be really glad for help as I'm pretty stuck here.

Christian Lauer INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

232,047 Quants.

Online Now

Stay Connected