Data requesting & resampling question by Igor Kartashev

Igor Kartashev

| Reward Discussion

Hello, looking for some wisdom!

What I am trying to do for a couple of days is to find an efficient and lightweight way to simply request 65 30-minutes bars each monday before trading starts, make calculations on this dataset and never request any more data during the week, so no need in additional data requests and rolling windows. Btw, I don't even need the full 30min bars, just 30min closes would be enough.

One more complication is that I need to make it not for just one vehicle but for the list of symbols.

How u would approach this guys?

thank an advance, sorry I've been unable to find a solution by myself.

Update Backtest

Igor Kartashev

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Rahul Chowdhury

STAFF ,

Hi Igor,

Here is a solution which stores the last 65 30-minute bars for each symbol in a rolling window. We use consolidators to create 30 minute bars and then store them in our windows for further use. We can use a scheduled event to make our calculations every Monday.

Igor Kartashev

954 ,

Thank you Sir, that's exactly what I was looking for!

My current problem is I'm not sure I've understood properly how to access the values of my rolling windows.

I read the whole doc, but never found any info about how to access multiple bars' values.

In MakeCalculations func I'm trying to extract close prices of the rolling window (close prices of the last 65 30-minute bars), pass in to "closes" list variable, and then make calculations & send the results to pandas dataframe "Levels".

Moreover, I'm also not sure if I call the most recent elements of the rolling window correctly in my OnData func.

Igor

import numpy as np

import pandas as pd



class ModulatedDynamicFlange(QCAlgorithm):



    def Initialize(self):

        

        #env, masterlist, datafeed, calculations

        

        self.SetStartDate(2019, 7, 12)

        self.SetEndDate(2019, 7, 15)

        self.SetCash(100000)

        tickers = ["SPY" , "AAPL", "TSLA", "AMZN"]

        self.data = {}

        self.Levels = pd.DataFrame(index=tickers, columns=['LC','HC','MeanC'])

        self.Traded = dict.fromkeys(tickers, False)

        

        for ticker in tickers:

            symbol = self.AddEquity(ticker, Resolution.Minute).Symbol

            self.data[symbol] = RollingWindow[TradeBar](65)

            consolidator = TradeBarConsolidator(timedelta(minutes = 30))

            self.SubscriptionManager.AddConsolidator(ticker, consolidator)

            consolidator.DataConsolidated += self.OnConsolidated

        

        self.Schedule.On(self.DateRules.Every(DayOfWeek.Monday), self.TimeRules.AfterMarketOpen("SPY", 1), self.MakeCalculations)

        

            

    def OnConsolidated(self, sender, bar):

        self.data[bar.Symbol].Add(bar)



    def MakeCalculations(self):

        if not all([window.IsReady for window in self.data.values()]):

            return

        

        #list of close prices of the last 65 30-minute bars

        closes = []

        

        for symbol in self.data.keys():

            symbolData = self.data[symbol]

            closes = symbolData[0:65].Close

            self.Levels['LC'][symbol] = LC = min(closes)

            self.Levels['HC'][symbol] = HC = max(closes)

            self.Levels['MeanC'][symbol] = MeanC = np.mean(closes)

            

    def OnData(self,data):

        

        #Trading

        

        for symbol in self.data.keys():

            symbolData = self.data[symbol]

            if self.Traded[symbol] == False:

                    if symbolData[0].Low <= self.Levels['LC'][symbol] and symbolData[0].Close >= self.Levels['LC'][symbol]:

                        self.MarketOrder(symbol, 1)

                        self.Traded[symbol] = True

Igor Kartashev

954 ,

Okay, I came up with following solution. Unfortunately it's not pretty and I don't know if multiple consolidators for multiple symbols can be added, (for example, both 30min and 5min consolidators for each asset), but the current implementation works.

I created a "for i in range" loop to add close prices to the list variable, and it turned out that I had to make another string version of "symbol" var in order to be able to access my pandas dataframes.

I still can't understand tho how to properly add WarmUp period in case you are using consolidators, especially multiple ones like 30m & 5m for each asset.

And one more question, in case of using Universe selection, is it possible to make your Universe give you the assets it chose in list format?

Current solution:

I still dont like these two lines of code:

for i in range (0,65):
closes.append(bars[i].Close)

I still think it can be done better. Kinda perfectionist I am, I guess.

def MakeCalculations(self):

        if not all([window.IsReady for window in self.data.values()]):

            return

        

        #list of close prices of the last 65 30-minute bars

        closes = []

        

        for symbol in self.data.keys():

            bars = self.data[symbol]

            sym = str(symbol)

            for i in range (0,65):

                closes.append(bars[i].Close)

            self.Levels['LC'][sym] = LC = min(closes)

            self.Levels['HC'][sym] = HC = max(closes)

            self.Levels['MeanC'][sym] = MeanC = np.mean(closes)

            closes[:] = []

            

    def OnData(self,data):

        if not all([window.IsReady for window in self.data.values()]):

            return

        

        for symbol in self.data.keys():

            bars = self.data[symbol]

            sym = str(symbol)

            if self.Traded[sym] == False:

                if bars[0].Low <= self.Levels['LC'][sym] and bars[0].Close >= self.Levels['LC'][sym]:

                        self.MarketOrder(symbol, 1)

                        self.Traded[sym] = True

Rahul Chowdhury

STAFF ,

Hey Igor,

1. Data will be consolidated during the warm up period. You should set your warm up period to the largest time period which ensures all your rolling windows are ready. You can set your warm up period for your algorithm like this

self.SetWarmUp(timedelta(days = 15))

2. You can access the securities in your universe by using self.ActiveSecurities.

for kvp in self.ActiveSecurities:

    symbol = kvp.Key

    security = kvp.Value

3. You can make those 2 lines of code more concise by rewriting it this way:

[closes.append(bar.Close) for bar in bars]

Best
Rahul

Igor Kartashev

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!