Key Error

Back

Hi friends,

I'm having troubles when making backtest. I find quite error messages related to data for some symbols, so I would like you to help me.

I have the following strategy:

import numpy as np

class BetaAlgorithm(QCAlgorithm):

def Initialize(self):
self.SetStartDate(2020, 8, 20) # Set Start Date
self.SetEndDate(2020, 10, 26) # Set End Date
self.SetCash(10000) # Set Strategy Cash

self.ReductionCoeff = 0.98 # Coefficient to avoid leverage
self.SetWarmup(22)
self.PositionsNumber = 5 # Max. number of active positions for this strategy

self.spy = self.AddEquity("QQQ", Resolution.Daily) # Add equity to use for trends

# Dow 30 companies.
self.symbols = [self.AddEquity(ticker).Symbol

for ticker in [
'FVRR', # FIVERR
'GH', # Guardant Health
'GWRE', # Guidewire Software
'HUBS', # HubSpot
'HUYA', # HUYA
'IBKR', # Interactive Brokers Group
'INCY', # Incyte Corporation
'IONS', # Ionis Pharmaceuticals
'IOVA', # Iovance Biotherapeutics
'IRDM', # Iridium Communications
'KTOS', # Kratos Defense & Security Solutions
'LMND', # Lemonade
'LSPD', # Lightspeed POS
'MCRB', # Seres Therapeutics
'MKL', # Markel Corporation
] ]

# Benchmark - Security used as reference to beat. In this case QQQ (NASDAQ index)
self.benchmark = Symbol.Create('QQQ', SecurityType.Equity, Market.USA)

# Set number days to trace back
self.lookback = 1

# Schedule Event: trigger the event at the begining of each month.
self.Schedule.On(self.DateRules.EveryDay(),
self.TimeRules.AfterMarketOpen(self.symbols[0]),
self.Rebalance)

#QQQ trend filter: only trades if benchmark trends
# Very fast moving average of benchmark
self.spy_ma_vfast = self.EMA("QQQ", 5)
# Fast moving average of benchmark
self.spy_ma_fast = self.EMA("QQQ", 13)
# Slow moving average of benchmark
self.spy_ma_slow = self.EMA("QQQ", 21)


def Rebalance(self):

# Defines benchmark in a trend (fast ma over slow ma)
self.trend_up = self.spy_ma_vfast > self.spy_ma_fast and self.spy_ma_fast > self.spy_ma_slow

# Fetch the historical data to perform the linear regression

history = self.History(

self.symbols + [self.benchmark],
self.lookback,
Resolution.Daily).close.unstack(level=0)

symbols = self.SelectSymbols(history)

# Liquidate positions that are not held by selected symbols and liquidates if benchmark is not in a trend
for holdings in self.Portfolio.Values:
symbol = holdings.Symbol
if symbol not in symbols and holdings.Invested:
self.Liquidate(symbol)

# Invest in the selected symbols is benchmark/reference is in a trend
for symbol in symbols:

if self.trend_up:
# Position size is calculated based on active strategies, max positions number and some coeffs to avoid leverage
self.SetHoldings(symbol, (1*self.ReductionCoeff)/self.PositionsNumber)

if not self.trend_up:
self.Liquidate(symbol)


def SelectSymbols(self, history):
'''Select symbols with the highest intercept/alpha to the benchmark'''

alphas = dict()

# Get the benchmark returns
benchmark = history[self.benchmark].pct_change().dropna()

# Conducts linear regression for each symbol and save the intercept/alpha
for symbol in self.symbols:

# Get the security returns
returns = history[symbol].pct_change().dropna()
returns = np.vstack([returns, np.ones(len(returns))]).T

# Simple linear regression function in Numpy
result = np.linalg.lstsq(returns, benchmark)
alphas[symbol] = result[0][1]

# Select symbols and number of them with the highest intercept/alpha to the benchmark
selected = sorted(alphas.items(), key=lambda x: x[1], reverse=True)[:self.PositionsNumber]
return [x[0] for x in selected]

 

But I'm getting the following error:

BacktestingRealTimeHandler.Run(): There was an error in a scheduled event EveryDay: FVRR: 0 min after MarketOpen. The error was KeyError : 'LSPD XHRUTPA72AG5'

If I try an older Start Date, it shows more error in different tickers.

Is there any way to fix this or pass the error? It happens even in other strategies using coarse and fine selection but at this moment this is the one I would like to fix.

Thanks in advance and kind regards.

 

Update Backtest







 
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Saúl,

if you really need to get them working it is possible to except the non-working symbols. It is probably better to try an understand why the non-working symbols are not working, maybe the failing ones have a different first ticker? These symbols can be logged and further investigate them.

0


Just a quick glance, but the error looks to be a Pandas KeyError from this line

for symbol in self.symbols:
returns = history[symbol].pct_change().dropna()

which means that `FVRR` is not in the `history` dataframe. A simple solution would be to increase the `lookback` variable from 1 day to say, 3 days, then slicing it by the last sample. 

My guess is when the Rebalance function is being triggered on Monday, the resulting history dataframe with 1 day lookback is empty, throwing the error. Probably best to add some data integrity checks instead for stability though, in case of market holidays etc which could fail the 3 day lookback anyways.

Alternatively, if you just need the prices of previous day you could just store them in a dict and update them at the end of each day. This would solve the data integrity issue and also speed up your backtests by avoiding the (expensive) call to self.History every day.

0

Hi Hector Barrio and Adam W !

Thank you very much for your help, it will help me to debug better and solve some mistakes.

Héctor, I follow your work at Ostirion.net and it's very inspiring. Will follow even closer :-) 

Kind regards.

1

You are welcome Saúl, glad you find it helpful!

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!