Lean manual universe bar data error with too many symbols

SetWarmup does not appear to fix this error

Hi CAPO,

It looks like your algo is trying to trade an asset before some data as been given to your algo for that asset. Here, it is AAPL.

Depending on how your algo is laid out, it can happen. “You start trading too soon” is the long story short.

Usually culprit is lacking of sanity checks in OnData. Another one is if you are not subscribed to AAPL anymore but still want to trade it. If you are not invested in AAPL and AAPL leaves your Universe, you won't be subscribed anymore to it. If your algo keeps a ref to AAPL and trades it - bang!

Without code, it is difficult to tell you more, anything more precise…

Fred

Attaching Backtest 1 of 2, 

The error appears to be related to the number of securities being added to the universe, and I suspect is related to the RAM or CPU limitations of the backtest node, because on my local lean I can handle up to 350 symbols without getting the error, while on quantconnect cloud it required far less symbols before the error appeared. In this example the error occurs since I am adding too many equities. The same happens if I create symbols instead and use manualuniverse selection.

Backtest Handled Error: AAPL: The security does not have an accurate price as it has not yet received a bar of data. Before placing a trade (or using SetHoldings) warm up your algorithm with SetWarmup, or use slice.Contains(symbol) to confirm the Slice object has price before using the data. Data does not necessarily all arrive at the same time so your algorithm should confirm the data is ready before using it. In live trading this can mean you do not have an active subscription to the asset class you're trying to trade. If using custom data make sure you've set the 'Value' property.

Attaching Backtest 2 of 2, 

In this second example the error does not appear since I only add equities from tickers[:10] instead of the full list of 450 tickers. I believe there is some type of RAM/CPU limitation that I would like to understand. If someone has a deeper understanding of what causes this issue, beyond just recommending I use less symbols in my universe, that would be appreciated. I've also tried checking for data (if not data.ContainsKey(s):    # Check for symbol) and that only causes my trading logic to no longer execute .setholdings. There is nothing in the code that removes the security, it seems like there is a data limitation against so many equities added.

Hi CAPO,

I tried running the following code but there seems to be a problem right now. Have you tried running this?:

from AlgorithmImports import *


class VirtualOrangeAlbatross(QCAlgorithm):
    def Initialize(self):
        self.SetStartDate(2020, 2, 1)  # Set Start Date
        self.SetEndDate(2020, 3, 1)  # Set End Date
        self.SetCash(100000)  # Set Strategy Cash
        
        tickers = ['AAPL', 'MSFT', 'AMZN', 'GOOGL', 'GOOG', 'TSLA', 'NVDA', 'BRK.B', 'UNH', 'JNJ', 'JPM', 'XOM', 'BAC', 'CVX', 'DIS', 'PFE', 'ABBV', 'AVGO', 'CSCO', 'COST', 'PEP', 'ADBE', 'TMO', 'CMCSA', 'ABT', 'WFC', 'CRM', 'AMD', 'ACN', 'LLY', 'INTC', 'MRK', 'QCOM', 'WMT', 'MCD', 'DHR', 'NKE', 'NFLX', 'UNP', 'TXN', 'NEE', 'RTX', 'UPS', 'LOW', 'BMY', 'LIN', 'MDT', 'CVS', 'INTU', 'PYPL', 'HON', 'AMGN', 'SCHW', 'COP', 'ORCL', 'AXP', 'AMAT', 'NOW', 'IBM', 'ANTM', 'SBUX', 'PLD', 'LMT', 'BLK', 'ISRG', 'AMT', 'CAT', 'TGT', 'ZTS', 'MDLZ', 'SPGI', 'BKNG', 'SYK', 'ADP', 'ADI', 'MMM', 'CME', 'PNC', 'TFC', 'TJX', 'LRCX', 'MMC', 'BDX', 'DUK', 'USB', 'GILD', 'CSX', 'CHTR', 'TMUS', 'CCI', 'ICE', 'FCX', 'EOG', 'NOC', 'REGN', 'COF', 'AON', 'EQIX', 'ATVI', 'SHW', 'BSX', 'NSC', 'PGR', 'ITW', 'ETN', 'HCA', 'VRTX', 'FISV', 'FIS', 'HUM', 'EMR', 'FDX', 'PXD', 'PSA', 'SLB', 'MRNA', 'KLAC', 'NEM', 'APD', 'MCO', 'ILMN', 'AIG', 'NXPI', 'LHX', 'ADSK', 'MET', 'CNC', 'MPC', 'SNPS', 'ROP', 'FTNT', 'MAR', 'TEL', 'SRE', 'JCI', 'AEP', 'APH', 'SPG', 'CTSH', 'IDXX', 'SYY', 'IQV', 'KMB', 'ECL', 'ADM', 'DOW', 'ORLY', 'INFO', 'CMG', 'BAX', 'TRV', 'PRU', 'CDNS', 'MCK', 'HLT', 'EXC', 'MSCI', 'GIS', 'DXCM', 'HPQ', 'MCHP', 'AZO', 'GPN', 'CARR', 'DLR', 'WMB', 'PAYX', 'CTVA', 'ALGN', 'OXY', 'DVN', 'NUE', 'PSX', 'MSI', 'AFL', 'WELL', 'XEL', 'DFS', 'RMD', 'YUM', 'SIVB', 'STZ', 'TDG', 'ALL', 'APTV', 'VLO', 'EBAY', 'IFF', 'AMP', 'AVB', 'OTIS', 'SBAC', 'CTAS', 'KMI', 'AJG', 'TROW', 'PEG', 'FITB', 'ROST', 'CBRE', 'MTD', 'WBA', 'DLTR', 'PCAR', 'MNST', 'PPG', 'HAL', 'BIIB', 'GLW', 'FRC', 'ROK', 'MTCH', 'AME', 'FAST', 'OKE', 'HSY', 'CMI', 'ODFL', 'EQR', 'EXPE', 'BLL', 'KEYS', 'TWTR', 'WST', 'WEC', 'VRSK', 'ANSS', 'HES', 'DHI', 'WTW', 'ANET', 'AWK', 'CERN', 'TSN', 'DAL', 'ZBH', 'EFX', 'SWK', 'KHC', 'ARE', 'LUV', 'CPRT', 'LYB', 'EXR', 'RSG', 'BKR', 'EIX', 'LEN', 'VMC', 'STE', 'CHD', 'FANG', 'MKC', 'NTRS', 'MLM', 'SYF', 'HIG', 'MTB', 'DTE', 'MAA', 'CDW', 'URI', 'KEY', 'STT', 'FTV', 'TSCO', 'ALB', 'SWKS', 'PKI', 'HBAN', 'DOV', 'CFG', 'ZBRA', 'AEE', 'VTR', 'BBY', 'ENPH', 'ABC', 'ETR', 'STX', 'HPE', 'ESS', 'VRSN', 'GWW', 'ULTA', 'RJF', 'GNRC', 'DRE', 'COO', 'ETSY', 'NDAQ', 'TDY', 'MPWR', 'PPL', 'MOS', 'TER', 'WAT', 'CTRA', 'FLT', 'DRI', 'VFC', 'TTWO', 'RCL', 'CZR', 'CMS', 'CLX', 'CINF', 'POOL', 'OMC', 'MRO', 'HOLX', 'KMX', 'SBNY', 'AMCR', 'PARA', 'MGM', 'TRMB', 'CTLT', 'PFG', 'AKAM', 'BXP', 'TYL', 'EXPD', 'NTAP', 'GPC', 'WAB', 'CNP', 'SEDG', 'CAG', 'JBHT', 'PEAK', 'NLOK', 'GRMN', 'PAYC', 'LYV', 'TXT', 'XYL', 'DGX', 'NVR', 'WDC', 'EMN', 'TFX', 'UDR', 'BRO', 'DPZ', 'PWR', 'TECH', 'CCL', 'CAH', 'LVS', 'FDS', 'QRVO', 'CEG', 'CRL', 'FMC', 'SJM', 'ATO', 'KIM', 'MKTX', 'LNT', 'IEX', 'AVY', 'IPG', 'LDOS', 'IRM', 'ABMD', 'EVRG', 'BBWI', 'HWM', 'AES', 'PKG', 'MAS', 'BIO', 'VTRS', 'HRL', 'UAL', 'RHI', 'APA', 'HST', 'FOXA', 'JKHY', 'CTXS', 'AAP', 'CHRW', 'PHM', 'WRB', 'LKQ', 'HAS', 'CMA', 'CBOE', 'FFIV', 'NDSN', 'WHR', 'WRK', 'HSIC', 'INCY', 'BF.B', 'LNC', 'PTC', 'XRAY', 'FBHS', 'NWSA', 'EPAM', 'TPR', 'SNA', 'JNPR', 'ZION', 'UHS', 'TAP', 'ALLE', 'AAL', 'REG', 'SEE', 'BWA', 'AIZ', 'PNR', 'OGN', 'CPB', 'PBCT', 'NRG', 'AOS', 'NWL', 'CDAY', 'DISCK', 'LUMN', 'BEN', 'HII', 'FRT', 'PNW', 'WYNN', 'DISH', 'MHK', 'IVZ', 'ALK', 'DXC', 'PENN', 'ROL', 'DVA', 'PVH', 'NCLH', 'FOX', 'VNO', 'NLSN', 'DISCA', 'IPGP', 'UAA', 'GSKY', 'LEN.B', 'NWS']
        
        for symbol in tickers:
            self.AddEquity(symbol, Resolution.Minute, fillDataForward = True, leverage = 0, extendedMarketHours = False)
                    

    def OnData(self, data):
        """OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
            Arguments:
                data: Slice object keyed by symbol containing the stock data
        """
        if data.Bars.ContainsKey("AAPL") and data["AAPL"] is not None:
            if not self.Portfolio.Invested:
                self.SetHoldings("AAPL", 1)
                self.Debug("Purchased Stock")

Fred

Thank you Fred, the logic in your above code seems to prevent the bactest handling error with the full amount of tickers, thank you! I'll need to change my trading logic a bit in order to repeat this when looping over all the symbols, but this appears to work.