QuantConnect Community

Pandas Symbol Remapper - Easier Access to Data Frames

Back

Hello All,

We merged an update to LEAN to map symbol objects to the underlying row of a Pandas DataFrame. This should allow you to use the symbol object instead of hacks like "str(symbol.ID)". The underlying DataFrame is still indexed by the raw symbol hash which is guaranteed to be truly unique. 

We tried to do this last week but had to roll back the implementation as we had too many bugs. This new merge represents a lot of effort by the LEAN team and we have complete coverage of the Pandas Unit Tests so we're pretty sure its backward compatible and also supports symbols on all indexes even when you chain calls together.

If you have any issues please let us know. Hopefully, it just makes your life easier =) 

Update Backtest



The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

5 Comments
 
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Great news! Big thank you to the QC team for this!

Emilio

2

Do you have any examples of the update?

1

Hi Juhwan; the best examples are the unit tests. Alex has written hundreds with every API method in multiple combinations.

You can see each test case the Pandas DF accepts 3 test inputs for the indexer: 

[TestCase("'SPY'", true)]
[TestCase("symbol")]
[TestCase("str(symbol.ID)")]

And we confirm they all map to the underlying asset. The str(id) accessor keeps it backward compatible with users currently using that style, and the "symbol" test is of the actual object. The "SPY" test is for people who've manually added securities to their algorithms and can access them by ticker.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Hi Jared,

It's not clear to me what this change does.  Work arounds are still needed to filter dataframes by a list of tickers.  See example below.

# Set up problem:
tickers = ['SPY','BND','GLD']
no_gld = ['SPY','BND']
h_close = self.History(tickers,252,Resolution.Daily)['close'].unstack(level=0)

# This works:
gld_close = h_close.loc[:,'GLD']

# This doesn't work:
spy_bnd = h_close.loc[:,no_gld]

# This works:
symb = [str(self.Symbol(x).ID) for x in no_gld]
spy_bnd = h_close.loc[:,symb]

 

1

Hi Tristan F ,

All of those ways to slice the data frame should work.
We are aware of the issue and have opened a GitHub issue to address it:
Selecting Multiple Columns From History With List of Symbols #4547

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Update Backtest



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Loading...

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Discussion Tags