Backtest does not work first day, then goes swimmingly thereafter by Ted Murphy

Ted Murphy INVESTOR

| Reward Discussion

Hello all,

My algorithm does not work the first day, then goes fine the following day and thereafter. This was ok in backtesting, because I simply started a day earlier. When I run the algorithm live, however, it does not kick off properly:

I call the AddUniverse, then I schedule an event in Initialize():

# get the universe of stocks every morning
self.AddUniverse(self.CoarseSelectionFunction)

# schedule an event to fire every trading day, 10 minutes before market close
self.Schedule.On(self.DateRules.EveryDay("SPY"), self.TimeRules.BeforeMarketClose("SPY", 10), self.TradeBeforeMarketClose)

But that universe of stocks is not available the first day that I run the scheduled event (and the information is not available in the Live implementation). The next day, the universe is available for backtesting purposes.

I think if I can get a backtest to work on the same day, it will work Live:

self.SetStartDate(2018,6,18) #Set Start Date
self.SetEndDate(2018,6,18) #Set End Date

May have something to do with Warmup? Please let me know your thoughts.

Update Backtest

Ted Murphy

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Ted Murphy

180 ,

Here is a sample that fails on the first day, trades on the second:

class MyAlgorithm(QCAlgorithm):



    def Initialize(self):



        self.SetStartDate(2018,6,15)    #Set Start Date

        self.SetEndDate(2018,6,18)      #Set End Date

        self.SetCash(100000)            # Account value



        self.AddEquity("SPY", Resolution.Second)



        self.AddUniverse(self.CoarseSelectionFunction)



        self.Schedule.On(self.DateRules.EveryDay("SPY"), self.TimeRules.BeforeMarketClose("SPY", 10), self.TradeBeforeMarketClose)



    def CoarseSelectionFunction(self, coarse):



        # price > 5 and volume > 500k and sector data available

        selected = [x for x in coarse if (float(x.Price) >= 5 and x.Volume > 5000000) ]



        # this is our universe

        self.Debug("selected:" + str(len(selected)))



        # subscribe to these stocks, add list of symbols to self

        self.MySymbols = []

        for x in selected:

            self.MySymbols.append(self.AddEquity(x.Symbol.Value, Resolution.Minute).Symbol)



        # return the list of symbols for consistency with "fine", but they are already saved in self.MySymbols[]

        return [ x.Symbol for x in selected ]



    # trade routine

    def TradeBeforeMarketClose(self):



        # make sure we have our ticker list

        noCustomData = 0

        try:

            self.Debug("Selected symbols in universe: " + str(len(self.MySymbols)))

        except:

            self.Debug("Fail: Universe of stocks not selected")

            noCustomData = 1

        if noCustomData:

            return



        self.MarketOrder("SPY", 100)



        return



    # order notifications

    def OnOrderEvent(self, fill):

        order = self.Transactions.GetOrderById(fill.OrderId)

        self.Debug("{0} - {1}:TEST: {2}".format(self.Time, order.Type, fill))

Output from log:

2018-06-15 00:00:00 Launching analysis for 4533cde97c4089c165f4e46e5a80ad54 with LEAN Engine v2.4.0.0.3879

2018-06-15 15:50:00 Fail: Universe of stocks not selected

2018-06-16 00:00:00 selected:173

2018-06-18 15:50:00 Selected symbols in universe: 173

2018-06-18 15:50:00 2018-06-18 15:50:00 - 0:TEST: Time: 6/18/2018 7:50:00 PM OrderID: 1 Symbol: SPY Status: Submitted

2018-06-18 15:50:00 2018-06-18 15:50:00 - 0:TEST: Time: 6/18/2018 7:50:00 PM OrderID: 1 Symbol: SPY Status: Filled Quantity: 100 FillPrice: 276.66 USD OrderFee: 1 USD

2018-06-19 00:00:00 Algorithm Id:(4533cde97c4089c165f4e46e5a80ad54) completed in 13.10 seconds at 311k data points per second. Processing total of 4,080,171 data points.

Gurumeher Sawhney

16.5k ,

Universe selection is called at midnight each day, as shown through the log above, and those symbols are available to trade only after the selection. As a result the first day does not acquire the relevant data. The solution below skips skips over the day universe selection is empty.

Ted Murphy

180 ,

Sounds like I need to feed the starting universe into the algorithm if I want to go live.

Jared Broad

No, you will just need the algorithm to be able to wait for it to be emitted at 6am the following morning. The way the algorithm was written it threw an exception because it assumed the universe was ready -- if its tweaked like Guru showed above it won't throw at the end of the day.

If you prefer not to make any code changes then you can simply deploy after 4pm.

Ted Murphy INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

231,924 Quants.

Online Now

Stay Connected