Price data error even though checking data.ContainsKey

Hi Justin,

Done some modifs in your code (attached) but it did not solve the issue. Since in 2010 there is only 1 asset on 1 day (most likely the day of a rebalance) that triggers the inaccurate price, it looks to me like some error in the data for that security.

Done some speed improvements in the code but one change is crucial as before Python 3.7, dictionaries cannot be sorted (deterministically - in other words, the order of insertions in dictionaries are not guaranteed to be preserved). Thus, converting the sorted tuples list into a dict (by dict(sorted(ranked.items(), key=lambda x: x[1], reverse=True))) will work most of the time (preserve order. thus sort) but CAN change sometimes………………………….. one reason one can wonder why the exact same algo sometimes can return different results.

Fred

Thanks for the info on dictionary sorting! That's super helpful. 

As for the error, I'm going to see about reworking this using the algorithm framework and do some more digging. Is it common for data errors to come up now and then?

Re sorting dicts, once Python gets updated in LEAN, it won't be an issue any longer - from 3.7+, dictionaries do guarantee preservation of insertion order and iteration order.

Data errors do happen, no data in trading is perfect. Check out: https://www.quantconnect.com/forum/discussions/1/dataissues. If you come to the same conclusion than myself after going framework style, you can report a potential error at https://www.quantconnect.com/data/issues/open. If using the framework makes the error disappear (for the same assets (some selection criteria) and same period of time), please report back as I would find that very interesting to know.

Fred

Hi Justin,
The above error occurs because the security has not received a data bar. Data does not necessarily all arrive simultaneously, so your algorithm should confirm the data is ready before using it. To check if there is trade data, we recommend using:
 

if symbol in data.Bars:
	#logic

Note that data.ContainsKey(symbol) checks for whether the current slice object has data for the given symbol, which can be of trade and non-trade type(splits, dividends).
Best,
Varad Kabade

Thanks, Varad. Now that I'm seeing it, that makes total sense. Appreciate the help!