Pandas Symbol Remapper - Easier Access to Data Frames by Jared Broad

Jared Broad

| Reward Discussion

Hello All,

We merged an update to LEAN to map symbol objects to the underlying row of a Pandas DataFrame. This should allow you to use the symbol object instead of hacks like "str(symbol.ID)". The underlying DataFrame is still indexed by the raw symbol hash which is guaranteed to be truly unique.

We tried to do this last week but had to roll back the implementation as we had too many bugs. This new merge represents a lot of effort by the LEAN team and we have complete coverage of the Pandas Unit Tests so we're pretty sure its backward compatible and also supports symbols on all indexes even when you chain calls together.

If you have any issues please let us know. Hopefully, it just makes your life easier =)

Update Backtest

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Jared Broad

Accepted Answer

Update Backtest

Notebook

Emilio Freire

29k ,

Great news! Big thank you to the QC team for this!

Emilio

Juhwan Kim

890 ,

Do you have any examples of the update?

Jared Broad

Hi Juhwan; the best examples are the unit tests. Alex has written hundreds with every API method in multiple combinations.

You can see each test case the Pandas DF accepts 3 test inputs for the indexer:

[TestCase("'SPY'", true)]

[TestCase("symbol")]

[TestCase("str(symbol.ID)")]

And we confirm they all map to the underlying asset. The str(id) accessor keeps it backward compatible with users currently using that style, and the "symbol" test is of the actual object. The "SPY" test is for people who've manually added securities to their algorithms and can access them by ticker.

Tristan F

6.4k ,

Hi Jared,

It's not clear to me what this change does. Work arounds are still needed to filter dataframes by a list of tickers. See example below.

# Set up problem:

tickers = ['SPY','BND','GLD']

no_gld = ['SPY','BND']

h_close = self.History(tickers,252,Resolution.Daily)['close'].unstack(level=0)



# This works:

gld_close = h_close.loc[:,'GLD']



# This doesn't work:

spy_bnd = h_close.loc[:,no_gld]



# This works:

symb = [str(self.Symbol(x).ID) for x in no_gld]

spy_bnd = h_close.loc[:,symb]

Alexandre Catarino

Hi Tristan F ,

All of those ways to slice the data frame should work.
We are aware of the issue and have opened a GitHub issue to address it:
Selecting Multiple Columns From History With List of Symbols #4547

Jared Broad

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!