QuantConnect Community

Does this mean QuantConnect function don't really interface with Pandas? 

We do connect with pandas. It just means we use a class and object OO approach instead of static globals.

There's really two issues here. 1) adding securities to the QC "Securities" object, and 2) defining a list of securities.

In order for a QC algorithm to get data or trade a security it must be in the "Securities" Dictionary (take note of the capital D). This is a C# 'Dictionary' and NOT a Python 'dict'. One cannot use the familiar Python methods (even something as simple as 'len(self.Securities)' isn't understood). The docs describe this object:

"Algorithms have a securities property which stores a security object for each asset in your algorithm. Security objects hold the models (backtesting behaviors) and properties of an asset. Each security can be completely customized to behave as you'd like. Securities is  a Dictionary so you can access your Security objects with their ticker"

So, in order for an algorithm to work, one must add any and all securities one wishes to trade, or even access data, to the 'self.Securities' Dictionary. This is typically accomplished, as Jared showed above, with the the 'AddEquity' method (however, this works a little different when using dynamic universes). 

Now, one could use 'self.Securities' as a list of securities. Many of the algorithms I see posted here in the forums do just that. However, since this isn't a Python dict (or other standard Python object like a dataframe) this quickly becomes tedious. My preferred approach is to separately create a Python list or, better yet, a Pandas dataframe to define a list of securities. Also, another note. QC generally just uses the ticker symbol (eg the string 'IBM') to refer to securities and not the actual security objects (eg sid(3766)) as Quantopian does. So to define a list of securities one simply needs to define a list of tickers.

import pandas as pd

# One can create a list
context.security_list = ['SPY', 'IBM', 'AAPL']

# One can also turn that list into a dataframe and then store all kinds of stuff
# The benefit here is all the built in Pandas methods that come with it
context.security_df = pd.DataFrame(index=security_list)

Note, this is separate from adding securities to the 'self.Securities' Dictionary. One still needs to do that. This is just to keep track of securities and to manipulate them in a familiar Python manner.

Thanks Dan!

Thanks Dan! Great Answer

I think it works,  but I  have not been able to access the amounts of stock owned in the porfolio object.. 

• Notebook

Two little problems.... Try this.

Just need to iterate over the dataframe index. Also change 1 to 1.0. 

Good luck. See attached backtest.

Sorry. Didn't really address your issue "have not been able to access the amounts of stock owned in the porfolio object". Use the 'self.Portfolio[stock].AbsoluteQuantity' method.

See attached backtest. Check the logs.

This is what I have in mind in regards to connecting with pandas, I am aware it does not work but I think the code communicates what I am trying to accomplish

Please help us understand the objective.
If you have Portfolio and Securities that are "enhanced" dictionary keyed by the security symbol, why would you need to convert it to a pandas.DataFrame? 
At the moment, Historical Data Requests can return a pandas.DataFrame because we have learnt that it helps mathematical operations on the time series. 

"Please help us understand the objective."

My objective for using objects based on standard python libraries (rather than 'enhanced' objects -love that euphemism) is fourfold: 

1. ability to use common/standard python operators and methods
2. reduced reliance on 'for' loops / increased use of more 'pythonic' vectorized methods 
3. generally better handling and wider range of input parameters
4. robust documentation

An example of 1. To currently check if a stock is in a portfolio one needs to reference a special method 

The common python expectation would be 

(which doesn't work). If 'Portfolio' were a python dict, or list, or some object based upon those, then this approach would work. No need to develop special custom methods.

An example of 2. To currently create a list of stocks one needs to use a for loop or list comprehension. Below is an example from the docs on how to select the top 50 dollar value stocks from the 'course' object. The 'course' object isn't a standard python object. Manipulating it requires either explicit or implicit looping.

If 'course' were a common pandas dataframe, for example, this could be accomplished without loops in a single (more comprehensible) line. 

An example of 3. Most of the current objects require specific data types and are not very forgiving. This works

This does not

Most python users are accustomed to more robust handling of various data types.

As for number 4, the examples are obvious. Currently, using QC relies upon knowing a large number of custom methods. The documentation for these methods is often cryptic and without any examples (and that's if one can navigate through the documentation to begin with). If the objects in QC were based upon standard python libraries then one could rely upon the many well documented resources on the web. QC would then only need to document a small subset of specialized methods. Take example 1 above. Creating a custom method just to check if a stock is in some list requires creating custom documentation and examples unique to the QC platform. If however, this were a standard python object then QC wouldn't need to focus as much on documentation (simply use the 'in' operator which is well documented outside of QC). Another example would be length. To get the number of stocks in 'Portfolio' one needs to use the 'Portfolio.Count' method which isn't well documented. It would be much simpler to be able to use 'len(Portfolio.Keys') or something similar which uses familiar well documented python methods.

I know some of these examples extend beyond the original 'Portfolio' and 'Securities' objects in question, but I do see basing more objects on standard python libraries would help the python user experience (and functionality) all around.

ty Dan interesting read. We'll work on making it closer to a naitive object =)

Dan,  
I could not had explain it better.  Thanks for taking the time to understand what I meant and express it in the correct vocavulary and logic.