I am running into this runtime issue that I couldn't figure out the problem:
2022-02-08 09:00:00 Runtime Error: Trying to retrieve an element from a collection using a key that does not exist in that collection throws a KeyError exception. To prevent the exception, ensure that the key exist in the collection and/or that collection is not empty.
at wrapped_function
raise KeyError(f"No key found for either mapped or original key. Mapped Key: {mKey}; Original Key: {oKey}")
File "./cache/algorithm/main.py" in PandasMapper.py: line 75
Checking out this forum, it seems history calls creating this issue but I still don't know the reason.
in my setup during backtesting, I add 100 symbols every day using manual selection. On the first day, everything seems ok. The second day I see this run time error. I also put try-catch everywhere but it does not help to find out the location.
For example, I am creating symbol objects like:
def OnSecuritiesChanged(self, changes):
self.Log("OnSecuritiesChanged -> {}: {}".format(self.Time, changes))
try:
newSymbols = []
for security in changes.AddedSecurities:
symbol = security.Symbol
if symbol not in self.symbolDict:
newSymbols.append(symbol)
self.Log(f'OnSecuritiesChanged -> newSymbols: {[sym.Value for sym in sorted(newSymbols)]}')
if newSymbols:
history = self.History(newSymbols, self.dailyLength, Resolution.Daily)
historyH = self.History(newSymbols, self.hourlyLength, Resolution.Hour)
self.Log(f'OnSecuritiesChanged -> history: { [sym.split()[0] for sym in sorted(set(history.index.levels[0]))] }')
self.Log(f'OnSecuritiesChanged -> historyH:{ [sym.split()[0] for sym in sorted(set(historyH.index.levels[0]))] }')
for symbol in newSymbols:
if str(symbol) in history.index.levels[0] and str(symbol) in historyH.index.levels[0]:
self.symbolDict[symbol] = SymbolData(symbol, self, history, historyH)
else:
self.Log(f'history does not have symbol: {str(symbol)}')
# Remove consolidators for removed symbols and also remove symbol from symbolDict
for security in changes.RemovedSecurities:
symbol = security.Symbol
if symbol in self.symbolDict:
if self.symbolDict[symbol].isGood:
self.SubscriptionManager.RemoveConsolidator(symbol, self.symbolDict[symbol].oneHourConsolidator)
self.SubscriptionManager.RemoveConsolidator(symbol, self.symbolDict[symbol].oneDayConsolidator)
self.symbolDict.pop(symbol, None)
except Exception as e:
self.Log(f'something wrong with OnSecuritiesChanged: {str(e)}')
If history has the symbol, then I create the symbol object.
Another interesting observation looking at the logs is that history has some symbols that are not included in newSymbols list:
2022-02-07 09:00:00 OnSecuritiesChanged -> newSymbols: ['AAPL', 'ACGL', 'ACLS', 'ACT', 'ADM', 'AER', 'ALK', 'AMP', 'ANET', 'ATI', 'AU', 'BKR', 'BMO', 'BNS', 'BPOP', 'BTI', 'BY', 'BYD', 'CADE', 'CBT', 'CCMP', 'CENX', 'CIEN', 'CINF', 'CMC', 'CME', 'CPA', 'CSX', 'CUBI', 'CWK', 'ECPG', 'EDR', 'ENTG', 'ESI', 'EWBC', 'FINW', 'FITB', 'FNV', 'FUN', 'HGV', 'HLI', 'HSBC', 'HSY', 'HUN', 'HWC', 'HWKN', 'HWM', 'IMKTA', 'JNPR', 'KEY', 'KO', 'KSS', 'LITE', 'LPLA', 'LYB', 'LYV', 'MLI', 'MU', 'MUSA', 'NEM', 'NMIH', 'NMRK', 'NTAP', 'NXST', 'ORI', 'ORLY', 'PACW', 'PLAB', 'PLNT', 'RDN', 'RE', 'RGLD', 'RJF', 'RRR', 'RS', 'SEAS', 'SF', 'SFM', 'SGH', 'SIX', 'SNV', 'STX', 'TD', 'TGH', 'TLK', 'TRTN', 'TU', 'UBS', 'UNP', 'UNVR', 'VIRT', 'VLRS', 'WBS', 'WFC', 'WHD', 'WLK', 'WRB', 'ZION']
2022-02-07 09:00:00 OnSecuritiesChanged -> history: ['AA', 'AAPL', 'ACLS', 'ACT', 'ADM', 'AER', 'ALK', 'ALT', 'AMP', 'ANET', 'AU', 'BHGE', 'BKLY', 'BMO', 'BNS', 'BPOP', 'BTI', 'BY', 'BYD', 'CADE', 'CBT', 'CCMP', 'CENX', 'CIEN', 'CINF', 'CMC', 'CME', 'CMT', 'CPA', 'CSX', 'CUBI', 'CWK', 'EDR', 'ENTG', 'ESI', 'EWBC', 'FCBP', 'FINW', 'FITB', 'FNV', 'FUN', 'HBC', 'HBHC', 'HGV', 'HLI', 'HSY', 'HUN', 'HWKN', 'IMKTA', 'JNPR', 'KEY', 'KO', 'KSS', 'LITEV', 'LPLA', 'LYB', 'LYV', 'MCMC', 'MLI', 'MU', 'MUSA', 'NEM', 'NMIH', 'NMRK', 'NOB', 'NTAP', 'NXST', 'ORI', 'ORLY', 'PLAB', 'PLNT', 'RCHI', 'RE', 'RGLD', 'RJF', 'RRR', 'RS', 'SEAS', 'SF', 'SFM', 'SGH', 'SIX', 'SNV', 'STX', 'TAL', 'TD', 'TGH', 'TLK', 'TU', 'UBS', 'UNP', 'UNVR', 'VIRT', 'VLRS', 'WBST', 'WHD', 'WLK', 'ZION']
For example, AA and ALT are not in newSymbols list but it is in History?
I really appreciate any help here!
Thanks,
Cole S
Hey Mustafa,
It would be much easier to help if you will attach a backtest. Then we can see the full code and which line is throwing the exception.
-Cole
Fred Painchaud
Hi Mustafa,
Cole is right.
In the meantime, know that line 75 in your main.py file throws the exception. That line will include a dict get, like “self.my_dict[my_key]”. That key “my_key” is not in the dict. The reason why it is not there while your algo assumes it is can be many and to explain that, yes, we need your code. Or you can also investigate yourself if you prefer.
A quick hack is to add a guard before the access, like so: if my_key in self.my_dict:
That will make sure the error is not thrown anymore but it is much better to understand why your algo's logic is trying to access a dict with a non-existent key since it means there's an aspect of the logic which is not mastered.
Fred
Mustafa Oz
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!