History Timeout Exception - Live Trading by Andrzej Fudala

Andrzej Fudala

| Reward Discussion

Hello,

during live implementation of my algorithm I'm trying to get history of last 300000 SPY trade bars:

self.History(["SPY"], 300000, Resolution.Minute).loc["SPY"]

I've got following error:

The history request took too long to process. History requests have a 20 second timeout.

I've checked it more precisely and 20s time frame is enough only to get 150000 trade bars. I am blocked and I think the best would be to somehow make time limit dependent on the history length request. We're going more and more in the machine learning applications, so we need to access quite big data sets.

Update Backtest

Andrzej Fudala

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Andrzej Fudala

1.1k ,

If somebody is interested - below potential solution for minute resolution which splits the request.

self.splitted_history_request(self.stock, 300000, 100000)



def splitted_history_request(self, stock, periods, single_request_period):

        security = self.Securities[stock]

        dates = []

        dates.append(Time.GetStartTimeForTradeBars(security.Exchange.Hours, self.UtcTime, Time.OneMinute, periods, security.IsExtendedMarketHours))

        for i in reversed(range(periods / single_request_period)):

            splitted_periods = (i + 1) * single_request_period

            if splitted_periods != periods:

                dates.append(Time.GetStartTimeForTradeBars(security.Exchange.Hours, self.UtcTime, Time.OneMinute, splitted_periods, security.IsExtendedMarketHours))

        dates.append(self.UtcTime)

        

        hist = []

        for i in range(len(dates[:-1])):

            hist.append(self.History([stock.Value], dates[i], dates[i+1], Resolution.Minute).loc[stock.Value])

        hist = pd.concat(hist)

        return dates

Dan Whitnable

1.5k ,

Agreed. It would be helpful to have an efficient History method. I'm sure the QC folks are going to mention RollingWindows, however there are two issues with that approach. First, it doesn't work easily with a dynamic universe selection. Second, RollingsWindows are not updated for stock splits in live trading (ie previous entries in the window are not adjusted for a current split). The ability to fetch historical data is a must. I would suggest that a large portion of algorithms don't need 'real time' historicaal data. Fetching data could occur overnight or have a long timeout during trading hours.

Andrzej Fudala

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

207,709 Quants.

Online Now

Stay Connected