KeyError exception with custom-universe. Data in dictionary not sustained

Hi Ahmed,

The error message you're seeing is typically thrown when you're trying to access a key in a dictionary that doesn't exist. In your case, it seems to be happening when you're trying to access self.discount_purchase[symbol] in your OnData method.

One possible reason for this could be that the symbol you're trying to access in self.discount_purchase is not present in the dictionary at the time of selling. This could happen if the symbol was not added to the dictionary at the time of buying, or if it was somehow removed from the dictionary between the time of buying and selling.

To prevent this error, you can add a check to ensure that the symbol exists in the dictionary before trying to access it. Here's how you can do it:

if symbol in self.discount_purchase:
    self.Debug(f"Selling {symbol} at {self.Time}, Price: {price}, Purchase discount: {self.discount_purchase[symbol]}")
else:
    self.Debug(f"Selling {symbol} at {self.Time}, Price: {price}, but no purchase discount found")

This will ensure that the KeyError is not thrown, and will also give you a debug message when a symbol is being sold that was not present in the discount_purchase dictionary at the time of purchase.

If you're still facing issues, it would be helpful to see the complete code or a backtest which reproduces the issue. This would allow us to better understand the problem and provide more specific guidance.

For more information on handling KeyErrors, you can refer to these discussions on the QuantConnect forum: Discussion 12130, Discussion 14244, Discussion 14098, Discussion 7633, Discussion 14787, Discussion 13285, Discussion 14724, Discussion 928, Discussion 13816.

I need to understand why the error is being thrown, the idea is not to avoid it by preventing the error but rather to know why this behavior is happening? Your solution above does not solve the issue.

I understand your concern, Ahmed. The KeyError is thrown when a dictionary is accessed with a key that does not exist in it. In your case, the error is thrown when you're trying to access self.discount_purchase[symbol] in your OnData method.

The reason for this could be that the symbol you're trying to access in self.discount_purchase is not present in the dictionary at the time of selling. This could happen if the symbol was not added to the dictionary at the time of buying, or if it was somehow removed from the dictionary between the time of buying and selling.

One possible scenario could be that the symbol was not added to the dictionary because the condition value < self.lower_limit and not self.Portfolio[symbol].Invested was not met. If this condition is not met, the symbol will not be added to the discount_purchase dictionary, but it could still be sold later if the condition aro_value > self.upper_limit and self.Portfolio[symbol].Invested is met, leading to a KeyError.

Another possible scenario could be that the symbol was removed from the discount_purchase dictionary somewhere else in your code that is not shown in the snippets you provided.

To debug this issue, you could add more logging to your code to track the state of the discount_purchase dictionary and the symbols that are being added and removed from it. This could help you identify where and why the symbol is not in the dictionary at the time of selling.

If you're still facing issues, it would be helpful to see the complete code or a backtest which reproduces the issue. This would allow us to better understand the problem and provide more specific guidance.

Algo attached for reference…your help is much appreciated on this!

From the code you've provided, it seems like the KeyError could be occurring due to the way you're handling the self.cef_purchase_discount dictionary.

In your OnData method, you're adding a symbol to the self.cef_purchase_discount dictionary only when you're buying a security:

if aro_value < self.aro_dptl_lower_limit and not self.Portfolio[symbol].Invested:
    self.cef_purchase_discount[symbol] = self.cef_discount_data[symbol]
    self.SetHoldings(self.portfolioTargets)

However, you're trying to access the same symbol from the dictionary when you're selling a security:

elif aro_value > self.aro_dptl_upper_limit and self.Portfolio[symbol].Invested:
    self.Liquidate(symbol)

The KeyError could occur if a symbol is being sold that was not added to the self.cef_purchase_discount dictionary at the time of purchase. This could happen if the condition for buying a security (aro_value < self.aro_dptl_lower_limit and not self.Portfolio[symbol].Invested) was not met for that symbol.

To fix this issue, you could add a check to ensure that the symbol exists in the self.cef_purchase_discount dictionary before trying to access it:

elif aro_value > self.aro_dptl_upper_limit and self.Portfolio[symbol].Invested:
    if symbol in self.cef_purchase_discount:
        self.Liquidate(symbol)
    else:
        self.Debug(f"Symbol {symbol} not found in cef_purchase_discount dictionary")

This will prevent the KeyError from being thrown and will also give you a debug message when a symbol is being sold that was not present in the self.cef_purchase_discount dictionary at the time of purchase.