Back

How to get history from a security symbol

Hi everyone!

Thanks for the great platform to backtest and trade with!

I'm working on a new strategy that uses a different universe of stocks everyday pulled from a dropbox file.

My algo needs to calculate the 90 day rolling moving standard deviation for the close to open returns. Therefore I have to set the start date of the algo earlier than my inended start date because I can't use warm ups with universes. That's fine and working.

However, I can't figure out how to access the close prices from the multi-index dataframe I get back. I'm sure it's something silly but I'm stuck. 

 

Here's a snippet from a function called by the `AfterMarketOpen` time rule:

self.Debug('security.Symbol: {}'.format(security.Symbol))

# Get a multi index dataframe
history_midf = self.History(security.Symbol, 90, Resolution.Daily)
self.Debug('{}'.format(history_midf))

# Select the security close and get a single index dataframe
close_df = history_midf.loc[security.Symbol]['close']
self.Debug('{}'.format(close_df))

I get this error though:

Runtime Error: In Scheduled Event 'SPY: EveryDay: SPY: 1 min after MarketOpen', TypeError : object is not callable TypeError : object is not callable (Open Stacktrace)

Here's the complete log:

1639 | 15:20:53:
Launching analysis for ef718efee8a2329c161567f513e46cfb with LEAN Engine v2.4.0.0.4601
1640 | 15:20:53:
2017-07-07 00:00:00 Universe Changes: SecurityChanges: Added: SPY R735QTJ8XC9X
1641 | 15:20:53:
Backtest deployed in 16.057 seconds
1642 | 15:20:54:
2018-01-04 00:00:00 Universe Changes: SecurityChanges: Added: CMC R735QTJ8XC9X,NEOG R735QTJ8XC9X,UNF R735QTJ8XC9X
1643 | 15:20:55:
Runtime Error: In Scheduled Event 'SPY: EveryDay: SPY: 1 min after MarketOpen', TypeError : object is not callable TypeError : object is not callable (Open Stacktrace)
1644 | 15:21:05:
Algorithm Id:(ef718efee8a2329c161567f513e46cfb) completed in 12.19 seconds at 4k data points per second. Processing total of 48,668 data points.
1645 | 15:21:05:
security.Symbol: CMC R735QTJ8XC9X
1647 | 15:21:05:
open ... volume
symbol time ...
CMC R735QTJ8XC9X 2017-08-26 17.341478 ... 507806.0
...

(The security isn't actually SPY, but I did add SPY via AddEquity in order to get access to the AfterMarketOpen and BeforeMarketClose calls.)

The security I'm looking at in this example is CMC but security.Symbol is this weird thing: 

CMC R735QTJ8XC9X

The error is definitely generated by this line:

close_df = history_midf.loc[security.Symbol]['close']

Any ideas?

Update Backtest







I'm far from an expert on this, but have several backtests that load symbols from Dropbox using AddUniverse in a Python environment.  If I understand it correctly symbol strings and symbol objects should be treated the same, but this doesn't seem to be the case when symbols are added with AddUniverse.  I would try calling history with security.Symbol.Value and see if that works. 

No guarantees.

0

Hi Hugh. Thanks for the reply. I also  found references to `security.Symbol.Value`. But when I used that I get this error:

 

Cannot create history for the given ticker. Either explicitly use a symbol object to make the history request or ensure the symbol has been added using the AddSecurity() method before making the history request.

Now, it does suggest to use the symbol object (which I did and got the above error) or call AddSecurity before the call to history. Doing that gets me another error message:

 

TypeError : unindexable object TypeError : unindexable object (Open Stacktrace)

So, i'm SOL either way.

0

Hi Brett; you should use security.Symbol not .Value -- the value is the ticker string but you need to use the actual object.

Note the missing brackets on your API call:

self.History( [ security.Symbol ] , 90, Resolution.Daily)
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Jared. Thanks for the reply. I still get the error. Let me share the code. You have to uncomment the lines below so that the error occurs because I can't share a backtest with a runtime error:

 

### Uncomment these lines to see the error:

# Select the security close and get a single index dataframe
# close_df = history_midf[security.Symbol]['close']
# self.Debug('{}'.format(close_df))
0


There are some great docs here on history along with many examples. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Brett,

Sorry about sending you in the wrong direction with my comment yesterday. 

I'm not sure if you've solved your latest error but this one works.  I spent way too much time on this, but like I said yesterday I have several projects using similar structures and wanted to figure it out.  As Jared said, you must call self.History using security.Symbol, but it turns out that to reference the data in the df you must use str(security.Symbol).   This makes no sense to me, but it works.  The pertinent code is this:

sym = str(security.Symbol)
close_df = history_df.loc[sym]['close'] # select date & closing prices for one sym

security.Symbol does not work, and security.Symbol.Value does not work, but str(security.Symbol) does work.  See lines 127 - 133 of the attached backtest.

This may be a bug, because it is not consistent with any example code I've seen.

Hope it helps you out.

0


Hugh Todd 

Thanks! That works! I really appreciate it!

0

In Lean/QuantConnect, string is implictly converted into Symbol object (and vice-versa). It means that in any method or dictionary, we can use string where Symbol object is required. For instance in a History call:

spy_symbol = self.AddEquity('SPY').Symbol

# Last 10 SPY bars with minute resolution
df = self.History([spy_symbol], 10)
# or
df = self.History(['SPY'], 10)

Unfortunatelly implicit conversion is not supported by pandas DataFrame:

spy_df = df['SPY'] # works
spy_df = df[spy_symbol] # type error
spy_df = df[str(spy_symbol)] # works
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed