Back

Migrating from Quantopian to QuantConnect

In this thread, we are going to cover the differences between Quantopian and QuantConnect APIs.

Basic Algorithm
In QuantConnect, all algorithms must have an Initialize method to setup your strategy. Once setup most algorithms have OnData event handlers to process market data and make trading decisions:

# Quantopian
def initialize(context):
    # Reference to AAPL
    context.aapl = sid(24)

def handle_data(context, data):
    # Position 100% of our portfolio to be long in AAPL
    order_target_percent(context.aapl, 1.00)
# QuantConnect

class MyAlgo(QCAlgorithm):
    def Initialize(self):
    # Reference to AAPL
        self.aapl = self.AddEquity("AAPL")

    def OnData(self, data):
        # Position 100% of our portfolio to be long in AAPL
        self.SetHoldings("AAPL", 1.00)

Please note that we must define a class with the QuantConnect API and it must inherit from QCAlgorithm.
self refers to the instance attributes of the algorithm class. It is used to access methods and members of QCAlgorithm like AddEquity in the example above.

Update Backtest






The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks for writing this Quantopian transition guide. In Quantopian's Morningstar fundamentals, you can get a company's sector with morningstar.asset_classification.morningstar_sector_code. I don't see an equivalent version in Quant Connect's Morningstar documentation. Am I missing it? Is there some other way to filter by sector?

1

Hi Tyler Bandy 

Check out the doc for Morningstar fundamental data here

I also made a simple demonstration algo based on Alexandre Catarino 's example above. 

1


Vladimir Prelovac - We don't limit account size; but we'd prefer institutional investors contact us and setup a institutional / dedicated account. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Yan Xiaowei,

Thanks for the response. I was already familiar with that Morningstar fundamental doc. It has the other fundamental factors I want, but the Morningstar Sector Code seems to be missing. The IndustryTemplateCode that you used in your demonstration only differentiates between normal, mining, utility, transportation, bank, and insurance. Unfortunately, I need a little more granularity. The Quantopian Morningstar Sector Code is a 3 digit code that represents different sectors. Since Quant Connect seems to have most of the same Morningstar data, I thought it would be available.

1

Thanks for the tutorials, cloning them and testing them out now.

0

Having some trouble with getting data from history. Since its bars I assume that the OHLC data can be called via ['open'] ['close']. Shouldnt this work if I wanted to get a list of close data where close[-1] gives the last bar close.

 

data = self.History(self.qqq, period, Resolution.Daily)
close = data['close']


#So that close[-1] > close[-2] = 1 bar ago close greater than 2 bars ago.

Also in history is volume included in this bar object?

Thanks

1

For example to get something like this working.

import numpy as np
import pandas as pd

class TestAlgo(QCAlgorithm):

def Initialize(self):
# Backetst Info
self.SetCash(25000)
self.SetStartDate(2009,1,1)
self.SetEndDate(2011,1,1)

# Initialize Assets
self.AddEquity('QQQ')
self.qqq = 'QQQ'

#Schedule
self.Schedule.On(self.DateRules.EveryDay("QQQ"),self.TimeRules.AfterMarketOpen("QQQ"),Action(self.trade))

def trade(self):
period = 10

data = self.History(self.qqq, period, Resolution.Daily)
close = data['close']

if (close[-1] > close[-2]):
self.SetHoldings(self.qqq,1.0)
else:
self.SetHoldings(self.qqq,0)

 

1

Thank you very much for posting this guide, Alex
1

Jonathan Gomez, if you want a pandas.DataFrame of the Historical Request, you need to use a list of simbols for the first parameter:

data = self.History([self.qqq], period, Resolution.Daily)
if data.empty:
return

I found out that History Request for Daily QQQ is returning an empty list. We will look into it.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


We create an example to show how to use a simple indicator with universe selection.
A dictionary is created to save object that keeps the state of a exponencial moving average cross:

class SymbolData(object):
def __init__(self, symbol):
self.symbol = symbol
self.tolerance = d.Decimal(1.01)
self.fast = ExponentialMovingAverage(100)
self.slow = ExponentialMovingAverage(300)
self.is_uptrend = False
self.scale = 0

def update(self, time, value):
datapoint = IndicatorDataPoint(time, value)

if self.fast.Update(datapoint) and self.slow.Update(datapoint):
fast = self.fast.Current.Value
slow = self.slow.Current.Value
self.is_uptrend = fast > slow * self.tolerance

if self.is_uptrend:
self.scale = (fast - slow) / ((fast + slow) / 2)

We select the securities that are up trending and get those with the highest "scale":

# In the selector method: CoarseSelectionFunction
# Filter the values of the dict: we only want up-trending securities
values = filter(lambda x: x.is_uptrend, self.averages.values())

# Sorts the values of the dict: we want those with greater difference between the moving averages
values.sort(key=lambda x: x.scale, reverse=False)

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Resolution of Universe Data:

A common misunderstanding is this line of the universe selection: 

self.UniverseSettings.Resolution = Resolution.Daily

This means the assets being added to your universe will be added at Daily resolution. It defaults to Minute resolution (i.e. universe added securities are in Minute resolution).

It is great for doing really fast backtests -- but it also means all market orders (or SetHoldings) will be converted to MarketOnOpen orders as when the daily bar arrives -- the market has already closed. 

As the orders are placed when market is closed it can also make portfolio modelling tricky as we don't know what price the market order will fill for -- so some orders may be rejected due to insufficient margin. Those orders will show up as Invalid in the console.

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Leverage
On Quantopian, there is no limit to the amount of money that can be invested by an algorithm, regardless of its starting capital, so controlling cash and leverage is necessary.
On QuantConnect, each asset has its own pre-defined leverage (2 for Equity, 50 for Forex and CFD, etc). This value are used in margin models to check whether there is margin for an order to be filled. We can change the default values in two different ways: either using the method to subscribe to the security or the SetLeverage mthod that belong to the Security object:

 

AddEquity("SPY", Resolution.Minute, Market.USA, True, 1)

# or
spy = AddEquity("SPY")
spy.SetLeverage(1)
# or
self.Securities["SPY"].SetLeverage(1)

 

If we want to use all the leverage in SetHoldings helper method, we can verify the leverage the secutiry has and multiply it by the target percentage:

import decimal as d

spy_leverage = self.Securities["SPY"].Leverage
qqq_leverage = self.Securities["QQQ"].Leverage
self.SetHoldings("SPY", d.Decimal(0.5) * spy_leverage)
self.SetHoldings("QQQ", d.Decimal(0.5) * qqq_leverage)

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


import numpy as np
import pandas as pd
import talib as tb

class TestAlgo(QCAlgorithm):

def Initialize(self):
# Backetst Info
self.SetCash(25000)
self.SetStartDate(2009,1,1)
self.SetEndDate(2011,1,1)

# Initialize Assets
self.AddEquity('SPY')
self.spy = ['SPY',0]
self.stocks = [self.spy]

#Schedule
self.Schedule.On(self.DateRules.EveryDay("SPY"),self.TimeRules.AfterMarketOpen("SPY"),Action(self.trade))

def trade(self):
period = 30
spy = self.spy

for i in self.stocks:
data = self.History([i[0]], period, Resolution.Daily)
i[1] = data['close']

rsi = tb.RSI(spy[1],timeperiod=14)

for i in self.stocks:
if (i[1][-1] > i[1][-2]) and (rsi[-1] > rsi[-2]):
self.SetHoldings(i[0],1.0)
else:
self.SetHoldings(i[0],0)

Just a note, I write things in loops because later on there would be more than just 1 security. If I can get it working for 1 I can get it working for all.

So in Quantopian when you used talib you would be able to source the data as the close (aka spy[1]) and then it would spit out a list of values I could reference using [-1] for last bar, [-2] for 2 bars ago ect. When I added rsi[-1] > rsi[-2] it breaks. I assume just as in price its not outputting a list?

1

Specifying Assets - Symbols

In Quantopian you specify a security id sid(#). The number is unique from the QP database.

In QuantConnect you should use the string name of the ticker, as of today/when you launch the algorithm (or use universe selection). We use that string to look up the Symbol object.

Under the surface we use Symbol objects to uniquely identify securities. These are class objects which encode all the information required to uniquely identify a symbol. This information includes: original ticker string, list date of the security, security type, contract expiry, contract strike price.

We encode it into a symbol string: e.g. YHOO R735QTJ8XC9X is actually the original list string, YHOO, with the rest of the information encoded into the hash string after it. Over the last few years that company was renamed to AABA. For more information on decoding this see the symbol class. If you specify "AABA" we automatically look up the right Symbol object for you.

Common Issue

Because of delistings, renames etc the encoded symbol isn't neccessarily the same as the ticker, People often do things like this:

Symbol _spy = QuantConnect.Symbol.Create("SPY", SecurityType.Equity, Market.USA);

Which is OK for stocks which don't really change symbol but sometimes with complex symbols like GOOG this can cause trouble. The SPY string is the ticker string *at the listing date*. Depending on the listing date of the ticker you might select the wrong one by accident. The listetd GOOG is today called GOOGL.

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks for writing these short helpers. Regarding history requests for a particular symbol, is it possible to get the history in the form of a panda data frame or Series timeseries object? I can't seem to find a way to get the date part of the TradeBar enumerations returned by self.History(symbol, n, Resolution.Daily).

 

Thanks for your help,

Takis

1

Hi Takis,

I used np.array to convert a (pandas?) series into a (numpy?) array. I imagine you may find a similar code on google. I also wanted to post how I was able to 'resample' the minutely pricing into 30-Minute time bars for anyone interested. I'm certain there is a better way, but it works for now.

for stock in self.stock_list:
_30mprices = []
_30mrsi = []
pricesm = self.History(stock, 8000, Resolution.Minute).close
price_array = np.array(pricesm)

#until code can be resampled, iterating through numbers to pull close of last minute of 30 minute time period.
for i in range (1, 250):
_30mprices.append(price_array[(29-i*30)]) #list[-1], list[-31], list[-61], etc.
_30mprices.reverse() #To correct the order
_30ray = np.array(_30mprices)
_30mrsi = talib.RSI(_30ray, timeperiod=14)

 

1

Also Takis, Alexandre probably provided an answer earlier in this thread.

""

Alexandre Catarino

Staff  Pro , 19 hours ago, ,

Jonathan Gomez, if you want a pandas.DataFrame of the Historical Request, you need to use a list of simbols for the first parameter:


 
  1. data = self.History([self.qqq], period, Resolution.Daily)
  2. if data.empty:
  3. return

I found out that History Request for Daily QQQ is returning an empty list. We will look into it.

""

1

QuantConnect Debug Message Flooding:

Its common for people to put a debug message into the primary flow of data. As we're working with high resolution data that can result in millions of debug packets being streamed to the browser; slowing everything down and often crashing your browser tab. To prevent these self-DDOS attacks we require debug messages to be slightly different each time. An exact duplicate message will be filtered out. You can simply get around this by putting the time onto the message:

self.Debug(str(self.Time) + " Test")

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Addison/Jonathan,  Thanks for the answer -- the pandas dataframe returned when you use a list of symbols in the history request is what I was looking for.

1

Three questions, sorry for the length, I thought this was going to be shorter  when I started typing:

1. When do I need to use the warmup functionality? If I use "data = self.History([self.qqq], period, Resolution.Daily)" do I also need to use warmup? If my course and fine universe selection selects stocks based on a 30 day moving average crossover do I need to warmup?

 

2. Is there a 'best practice' to calculate stuff prior to market open? The equivalent quantopian code would be:

def before_trading_start(context, data):
#Run before trading

Which runs about 45min before trading begins for the day. 

 

3. What order do all the functions run, or is there a specific time that the algo beings executing? If I have a function scheduled to run at 10:00am and a function that is running every minute with OnData(self, data) which one will fire first? What time of day does the algorithm universe selection run?

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed