Back

Defining list of securities QuantConnect (QC) vs. Quantopian (Q)

Quantopian 

context.security_list = [sid(4922), sid(679), sid(24), sid(3149), sid(698), sid(1267), sid(23112), sid(1900), sid(4283), sid(2190), sid(2119), sid(8347), sid(20088), sid(3496), sid(3766), sid(3951), sid(4151), sid(25006), sid(4707), sid(5029), sid(5061), sid(5328), sid(5923), sid(5938), sid(7041), sid(7792), sid(21839), sid(35920), sid(8229), sid(7883)]

QuantConnect

self.aapl = self.AddEquity("AAPL")

self.other_symbol = self.AddEquity("other_symbol")

....

self. last_symbol = self.AddEquity("last_symbol") 

Question

Is there a better was to do this in QuantConnect with Python ? 

Update Backtest








# Something like this?

securities = ["AAPL", "IBM", "CAT"]
for stock in securities:
this.AddEquity(stock)
1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Does this mean QuantConnect function don't really interface with Pandas? 

0

We do connect with pandas. It just means we use a class and object OO approach instead of static globals.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


There's really two issues here. 1) adding securities to the QC "Securities" object, and 2) defining a list of securities.

In order for a QC algorithm to get data or trade a security it must be in the "Securities" Dictionary (take note of the capital D). This is a C# 'Dictionary' and NOT a Python 'dict'. One cannot use the familiar Python methods (even something as simple as 'len(self.Securities)' isn't understood). The docs describe this object:

"Algorithms have a securities property which stores a security object for each asset in your algorithm. Security objects hold the models (backtesting behaviors) and properties of an asset. Each security can be completely customized to behave as you'd like. Securities is  a Dictionary so you can access your Security objects with their ticker"

So, in order for an algorithm to work, one must add any and all securities one wishes to trade, or even access data, to the 'self.Securities' Dictionary. This is typically accomplished, as Jared showed above, with the the 'AddEquity' method (however, this works a little different when using dynamic universes). 

Now, one could use 'self.Securities' as a list of securities. Many of the algorithms I see posted here in the forums do just that. However, since this isn't a Python dict (or other standard Python object like a dataframe) this quickly becomes tedious. My preferred approach is to separately create a Python list or, better yet, a Pandas dataframe to define a list of securities. Also, another note. QC generally just uses the ticker symbol (eg the string 'IBM') to refer to securities and not the actual security objects (eg sid(3766)) as Quantopian does. So to define a list of securities one simply needs to define a list of tickers.

 

import pandas as pd

# One can create a list
context.security_list = ['SPY', 'IBM', 'AAPL']

# One can also turn that list into a dataframe and then store all kinds of stuff
# The benefit here is all the built in Pandas methods that come with it
context.security_df = pd.DataFrame(index=security_list)

 

Note, this is separate from adding securities to the 'self.Securities' Dictionary. One still needs to do that. This is just to keep track of securities and to manipulate them in a familiar Python manner.

1

Thanks Dan!

0

Thanks Dan! Great Answer

I think it works,  but I  have not been able to access the amounts of stock owned in the porfolio object.. 

0

Two little problems.... Try this.

# Iterate over the index (that holds the ticker symbols you have)
for stock in self.secs_df.index:
# Use 1.0 rather than 1. Otherwise python uses integer
# math and rounds down the percent to zero.
self.SetHoldings(stock, 1.0/len_sec)

Just need to iterate over the dataframe index. Also change 1 to 1.0. 

Good luck. See attached backtest.

0


Sorry. Didn't really address your issue "have not been able to access the amounts of stock owned in the porfolio object". Use the 'self.Portfolio[stock].AbsoluteQuantity' method.

#Assumes 'stock' is a string variable for the ticker you want to check
stock = 'IBM'
shares_held = self.Portfolio[stock].AbsoluteQuantity

See attached backtest. Check the logs.

0


class MyAlgo(QCAlgorithm):
def Initialize(self):
# Reference to AAPL
self.SetStartDate(2017,10,2)
self.SetEndDate(2017,10,6)
self.SetCash(10000)
self.secs = ["AAPL", "IBM", "CAT"]
self.Log("hello")
#self.secs_df = pd.DataFrame(index=self.secs)
self.stock = self.AddEquity(self.secs)
self.portfolio_df = pd.DataFrama([], [self.secs],[['Qty'],['BuyPrcie']])

def OnData(self, data):
''' Runs every tick defined by resolution in initialize under equity'''
# Position 100% of our portfolio to be long in AAPL
len_sec = len(self.secs_df)
self.Log("total securities to trade " + str(len_sec))
self.SetHoldings(stock, 1.0/len_sec)
#self.SetHoldings("IBM", 0.5)
shares_held = self.Portfolio[secs_df.index].AbsoluteQuantity
self.portfolio_df['Qty'] = shares_held
BuyPrice = self.Portfolio[secs_df.index].AveragePrice
self.portfolio_df['BuyPrice'] = self.BuyPrice
self.Log(self.porfolio_df)
#self.Log(str(stock) + ' ' + str(shares_held) + ' ' + str(self.BuyPrice))

This is what I have in mind in regards to connecting with pandas, I am aware it does not work but I think the code communicates what I am trying to accomplish

0

Please help us understand the objective.
If you have Portfolio and Securities that are "enhanced" dictionary keyed by the security symbol, why would you need to convert it to a pandas.DataFrame? 
At the moment, Historical Data Requests can return a pandas.DataFrame because we have learnt that it helps mathematical operations on the time series. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


"Please help us understand the objective."

My objective for using objects based on standard python libraries (rather than 'enhanced' objects -love that euphemism) is fourfold: 

  1. ability to use common/standard python operators and methods
  2. reduced reliance on 'for' loops / increased use of more 'pythonic' vectorized methods 
  3. generally better handling and wider range of input parameters
  4. robust documentation

An example of 1. To currently check if a stock is in a portfolio one needs to reference a special method 

context.Portfolio[stock].HoldStock
or
context.Portfolio[stock].Invested
(also confusing having two methods which are the same).

The common python expectation would be 

stock in context.Portfolio.Keys

(which doesn't work). If 'Portfolio' were a python dict, or list, or some object based upon those, then this approach would work. No need to develop special custom methods.

An example of 2. To currently create a list of stocks one needs to use a for loop or list comprehension. Below is an example from the docs on how to select the top 50 dollar value stocks from the 'course' object. The 'course' object isn't a standard python object. Manipulating it requires either explicit or implicit looping.

# Take the top 50 by dollar volume using coarse
# sort descending by daily dollar volume
sortedByDollarVolume = sorted(coarse, \
key=lambda x: x.DollarVolume, reverse=True)

return [ x.Symbol for x in sortedByDollarVolume[:50] ]

If 'course' were a common pandas dataframe, for example, this could be accomplished without loops in a single (more comprehensible) line. 

return course.nlargest(50, 'DollarVolume')

An example of 3. Most of the current objects require specific data types and are not very forgiving. This works

my_stocks = ["SPY", "IBM"]
context.History(my_stocks, 14)
 

This does not

my_stocks = numpy.array(["SPY", "IBM"])
context.History(my_stocks, 14)

Most python users are accustomed to more robust handling of various data types.

As for number 4, the examples are obvious. Currently, using QC relies upon knowing a large number of custom methods. The documentation for these methods is often cryptic and without any examples (and that's if one can navigate through the documentation to begin with). If the objects in QC were based upon standard python libraries then one could rely upon the many well documented resources on the web. QC would then only need to document a small subset of specialized methods. Take example 1 above. Creating a custom method just to check if a stock is in some list requires creating custom documentation and examples unique to the QC platform. If however, this were a standard python object then QC wouldn't need to focus as much on documentation (simply use the 'in' operator which is well documented outside of QC). Another example would be length. To get the number of stocks in 'Portfolio' one needs to use the 'Portfolio.Count' method which isn't well documented. It would be much simpler to be able to use 'len(Portfolio.Keys') or something similar which uses familiar well documented python methods.

I know some of these examples extend beyond the original 'Portfolio' and 'Securities' objects in question, but I do see basing more objects on standard python libraries would help the python user experience (and functionality) all around.

1

ty Dan interesting read. We'll work on making it closer to a naitive object =)

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Dan,  
I could not had explain it better.  Thanks for taking the time to understand what I meant and express it in the correct vocavulary and logic. 

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed