Back

Converting algos from Quantopian

Hey, all. I have several live algos trading at Quantopian and am porting everything here. The conversion process is going slower than expected, so I am reaching out for help. I do not have a programming background, so these questions may seem naive. Many thanks!

The attached algo is simple. Buy SPY at 10am every day when the 1d MA >= 5d MA and exit if not. Questions:

1. How do I set a variable to equal the current price for SPY at 10am (when the scheduler calls the "Rebalance" function)?

2. How do I write a "for stock in" loop in the "Rebalance" function that (a) generates 1d MA and 5d MA for SPY and QQQ in "self.equities" (using "stock" rather than the hard coded SPY you see) and (b) places orders for SPY and QQQ if logic is met (again using "stock" rather than hard coded SPY)? 

3. How do you get comfortable that your trading logic is working when you cannot print the data like on Quantopian? For example, after someone (hopefully) shows me how to get the current price at 10am, how do I know that code is producing the right output. If I understand correctly, the data cannot be viewed in QC.

For Jared and Team QC, props on the platform! Very sophisticated. Would like to see more algo examples in Python (presumably with time via the community, but right now vast majority in C#) and a de-bugger for the IDE (I know this is on your priority list). Just being able to search for Python algos ONLY would also be a big help. I'm having trouble finding enough examples. Finally, looking forward to more order types on IB, esp the best efforts VWAP, in November.

Update Backtest








Hi Brett Crawford, check the attached algorithm.

1. In order to get the current price of the indicator, you should use  `self.SMA().Current.Value`

2. In order to loop the SMA indicator for different stocks, in Initialize() step, you can store self.SMA() objects for different stocks in a list. Then you can use the indicator in this list to call the current value in Ondata or Rebalance.

3. For checking the result, you can use self.Debug() to print out the value of indicator and the security symbols or the time( use `self.Time`) to validate the result. By subscribing the prime member, you can get more log data for your algorithm validation.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Excellent! Thanks, Jing. What if I want to compare the current prices of SPY and QQQ at 10am, respectively, to each one's 5d MA (instead of 1d MA to 5d MA), replacing this code: 

if self.SMA1[i].Current.Value > self.SMA5[i].Current.Value:
 
0

You can use self.Securities[symbol].Price to get the current price of the stock.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hmm. I used

   if self.equities[i].Price >= self.SMA5[i].Current.Value:

replacing code above in prior post, but it didn't work. I get the following Runtime error

   AttributeError : 'str' object has no attribute 'Price'

Thoughts? Same algo as above.

0

self.equities is the list of tickers you have defined.
You need to use self.Securities that is a member of QCAlgorithm:

if self.Securities[i].Price >= self.SMA5[i].Current.Value:
# trading logic
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hey, Alex. I tried self.Securities[i].Price before and again just now, but get the following error.

   Runtime Error: Python.Runtime.PythonException: TypeError : No method matches given arguments

If I # that code, then algo runs fine. Backtest attached for further inspection. Any idea what the hangup is?

As reminder, the goal here is to compare the current price at 10am to a moving average, once for each security in the list. I got the moving average. Just can't find the right syntax for current price (not yest close). Thank you!

0


Hi Brett Crawford, for self.Securities[symbol].Price, there should be a symbol in the bracket. But in your code

current_price = self.Securities[i].Price

here [i] in the bracket is the order number. self.equities[i] is the symbol for equity i. You should change to 

current_price = self.Securities[self.equities[i]].Price

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Okay, thanks Jing! That was the missing link. Works great now.

I'm almost done. Now I'm setting the position weights, which are based on the inverse of each security's volatility. I've found this to be an effective risk management tool across all strategies. For some reason the self.History code creates a runtime error "unindexable object". Here's the offending code (also in attached algo):

prices = self.History(self.equities[i], vol)['close']

where self.equities is my list of equity securities. The next two lines calculate annual standard deviation based on the last 100 days and then scale weights to assign to a portfolio of securities.

The attached algo has line 48 above commented out so you can see it running. Removing # causes the runtime error above.

Also, can you check line 55. It should update the indicator at 10am, when the function is scheduled.

Thanks in advance!


 
0


After a couple days work, I got closer, but still get unindexable object error. My research indicates floats in a list are not iterable. However, I am not advanced enough to know how to fix this error. Remaining steps are (1) convert to log normal of 1-day ROCP, (2) compute .stdev and (3) annualize for trading days. Any help with the correct syntax would be greatly appreciated. Thanks in advance for a quick look!

0


Hi Brett, 

annl_stdev = np.log(roc[i]).std() * (252 ** 0.5)

here you already assigned roc to be a float, a float is not indexable. It should be

annl_stdev = np.log(roc).std() * (252 ** 0.5)

But since roc is just a number, your calculation of std for a constant should be 0. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thank you. I see the problem now. How do I modify to get the stdev of the 1-day rate of change for the last 20 days, instead of just 1 day?



Brett Crawford, CIO

Flatiron Partners







This communication may contain privileged or confidential information. If you have received this communication in error, please notify us immediately by email, and delete the original message. This is not an offer to sell, or a solicitation of an offer to buy any financial product or an official confirmation of any transaction.
0

The easier way I can think of is first getting the history 

history = self.History("SPY", 21, Resolution.Daily)
close = history['close'] # close is a list containing the 21 days closing price

Then calculate the rate of change manually 

roc = [(close[i]-close[i-1])/close[i] for i in range(1,len(a))]

Then roc is a list, you can calculate the standard deviation using roc.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jing, thank you! That's how exactly I did it on Quantopian -- X days of historical prices, compute change, then log, then stdev, then annualize. I copied your code into the algo verbatim. Two questions: (1) why does the log for history show this message below instead of prices?

QuantConnect.Util.MemoizingEnumerable`1[QuantConnect.Data.Market.TradeBar]

And (2) why do close and roc return an error "unindexable object"?

Thanks for taking a look!

0


"SPY" continued to generate "unindexable object" errors, so I modified the code. Now I get data. Historical prices ("prices") are separated by ticker in the log. However, rate of change ("roc") runs both tickers together in one list How do I separate 20 days closing prices for SPY and 20 days closing prices for IWM? 

Also, how do I iterate through each ticker separately to compute volatility (then I know how to set target weights and enter trades) before moving on the the next ticker in the list? Thanks for your help!

0


Hi Brett, hope this is what you need. Price data on QC are all decimal. Each time when there is problem with decimal and float can be multiplied, you need to change the decimal to float or change the float to decimal to make them match

from decimal import decimal
# if a is a float, b is a decimal
decimal(a)*b
#or you can do this
a*float(b)
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks, Jing. Any thoughts on the second post about iterating through each ticker in the list separately? The goal is to assign a target weight then enter trades for each ticker in the list one at a time. Thanks!

0

One can also get the standard deviation of the historical prices by using dataframe methods in just two lines of code. This probably runs faster than interating over a for loop.

history = self.History(self.symbols, 21, Resolution.Daily)

# Extract just the close prices and use the 'unstack' method to make a column for each equity
close_prices = history.close.unstack(level=0)

# Use the built in 'pct_change' and 'std' to get the volatility
annl_stdev = (close_prices.
pct_change(axis=0).
std(axis=0, ddof=0) * (252.0 ** 0.5))

Attached is a backtest. Look at the logs and see both the numpy and the pandas methods produce the same results. One difference however, between the two is that numpy defaults to the 'population standard deviation' and the pandas dataframe method defaults to the 'sample standard deviation'. I'm not sure exactly which is applicable in equity volatility. Use the parameter 'ddof = 0' to get the 'population standard deviation' in the dataframe method if that's the way one wants to go.

0


Is the 252 the default trading days in a year?

0

Many thanks, Dan, for posting. Nice solution.

0

Thomas - yes

0

Dan, excellent work - crystal clear code. Thank you for the post. A follow up if I may...

I believe QC excludes the current price from self.History, unlike Quantopian. How would one call current prices for the symbol list, then merge with historical prices, and then calculate stdev including the current price (at the time of day when the function is scheduled)? The idea is to reflect current volatility conditions, rather than those of the prior day.

Second, I have only submitted orders by iterating through a preset security list or through pipeline on Quantopian. How would one submit orders from the pandas dataframe in QC? 

I made several attempts, but all ended in runtime errors, which I have struggled to comprehend on QC. Thanks in advance for your time.

0

Brett

You are correct that the QC 'History' method doesn't tack on the current data when requesting daily data. You can do that manually though. Just get the last minute data using 'History' with a minute resolution and then append that dataframe to the first.

# Get 21 previous days data plus the last minute data
history = self.History(self.symbols, 21, Resolution.Daily)
last_minute_data = self.History(self.symbols, 1, Resolution.Minute)

# Append the last minute data
history = history.append(last_minute_data)

Once the current data is appended, all the following logic remains the same.

There's a number of ways to submit orders on QC. I've mostly used the 'SetHoldings' method which is the same as the Quantopian 'order_target_percent'. Iterate through a list of symbols like this.

# Iterate through the self.symbols list to order
for stock in self.symbols:
# do whatever calculation to find weight
# here as an example it's just the ratio of std dev
weight = annl_stdev_series[stock] / annl_stdev_series.sum()
self.SetHoldings(stock, weight)

You could also iterate through the index of a dataframe (if you indexed it by symbols). Just remember to convert it to a list first (QC is particular about that).

# Iterate through a dataframe index
for stock in my_dataframe.index.tolist():
# do whatever you want here

Attached is your code with some of these changes. Take a look at the logs and you'll see the last minute data was appended correctly. Hope that helps...

Good luck.

1


Actually, the following DOES work to iterate through dataframe index. Just ensure the index is the 'symbol' (ie a string ticker symbol') and not a 'Security' object.

# Iterate through a dataframe index
for stock in my_dataframe.index:
# do whatever you want here like place an order
context.SetHoldings(stock, weight)

I like to use the name 'context' when referring to the algorithm object and 'self' if it's within some other class/situation. Just personal preference. Just a name. Didn't want to cause any confusion though.

0

Super helpful. Can't thank you enough.

0

Hello,  This algorithm is really helpful in getting started. Thanks 

One question: 

On the last line self.Log("{} {}".format(stock, weight))  the algorithm is logging what you have ordered. If we wanted to know how many stocks we actually bought how can we access the porfolio and get the log of securities owned ? 

Thanks 

0

Klaus, I run portfolio statistics on the close with the following function. It logs positions and overall portfolio leverage for a set list of securites in self.stocks. It doesn't list trades though.

    def PortfolioStats(self):
        
        port_val = self.Portfolio.TotalPortfolioValue
        
        for s in self.stocks:
            actual_weight = self.Portfolio[s].Quantity * self.Securities[s].Price / port_val 
            self.Log("{} {}".format(s, round(actual_weight, 3)))
        account_leverage = self.Portfolio.TotalAbsoluteHoldingsCost / port_val
        self.Log("{}%".format(round(account_leverage,3) * 100))

0

Hi Brett,

Would you mind sharing your justification for migrating from Quantopian to QuantConnect? My son-in-law is using Quantopian whereas I have briefly been exposed to QuantConnect. We would like to standardize on one of these two platforms before we get too far on developing algorithms. Any input from you that would help us reach a decision would be very appreciated.

All the best.

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed