ETF Trend Following Algorithm

nicely done,....thanks for posting.

Jing,

This is awesome! If you were to modify the algorithm to consume output from CourseSelection/FineSelection as a starting universe, how would you handle the uptake of the Universe symbols instead? I tinkered around with it a bit this morning, but need an extra nudge! Thanks in advance.

Aaron

nicely done,....thanks for posting.

Hi Aaron,

This is my attempt to add the universe selection for symbol picking. The universe selection is rebalanced every month and it picks the symbol based on PE ratio. It's a nice suggestion to improve the algorithm. If you have some great ideas about how to choose the universe, welcome to post it here.

As always, Thanks Jing! I will post a new version if I find anything worthwhile.

Hi Jing,

thanks you share the strategy !
what wrong of the following message that when I backtest the universe selection for symbol picking version strategy and backtest start date from 2015 

38 | 09:44:30:

Runtime Error: QuantConnect.Scheduling.ScheduledEventException: Python.Runtime.PythonException: ValueError : arrays must all be same length
at QuantConnect.Scheduling.ScheduleManager+<>c__DisplayClass16_0.<On>b__0 (System.String name, System.DateTime time) [0x00000] in <e60e6336223b4153a44c7c1bc57e55f5>:0
at QuantConnect.Scheduling.ScheduledEvent.OnEventFired (System.DateTime triggerTime) [0x00036] in <e60e6336223b4153a44c7c1bc57e55f5>:0
at QuantConnect.Scheduling.ScheduledEvent.OnEventFired (System.DateTime triggerTime) [0x00086] in <e60e6336223b4153a44c7c1bc57e55f5>:0
at QuantConnect.Scheduling.ScheduledEvent.Scan (System.DateTime utcTime) [0x0010f] in <e60e6336223b4153a44c7c1bc57e55f5>:0
at QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler.SetTime (System.DateTime time) [0x00019] in <edb82c23e8dc4c13a9514cc4c313efae>:0
at QuantConnect.Lean.Engine.AlgorithmManager.Run (QuantConnect.Packets.AlgorithmNodePacket job, QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.IDataFeed feed, QuantConnect.Lean.Engine.TransactionHandlers.ITransactionHandler transactions, QuantConnect.Lean.Engine.Results.IResultHandler results, QuantConnect.Lean.Engine.RealTime.IRealTimeHandler realtime, QuantConnect.Lean.Engine.Server.ILeanManager leanManager, QuantConnect.Lean.Engine.Alpha.IAlphaHandler alphas, System.Threading.CancellationToken token) [0x007d5] in <edb82c23e8dc4c13a9514cc4c313efae>:0
at QuantConnect.Lean.Engine.Engine+<>c__DisplayClass8_1.<Run>b__3 () [0x000ac] in <edb82c23e8dc4c13a9514cc4c313efae>:0 (Open Stacktrace)

Hi Devon, this error happens in calc_vol_scalar(self) function when trying to convert the dict into dataframe.

Converting the array value in dict into series will solve the problem.

Hi Jing Wu, Thanks for your work.You normalized the slope as below, Can you explain about this?

# Normalized slope
slope = a / b *252.0

Is your intention to make it 'annualized slope'? (or else?) If you want to get annualized slopes, should'nt it be 'sqrt(252)'? Or am I missing something here? Thanks.

Hi Jing Wu, I got same error as below.

Hi Jing Wu, One more quick question. Is there way to get all the slopes of entire symbols that are filtered? You got candidates from filtering 'P/E ratio' in 'FineSelectionFunction' and then you got the slope of each stock by using linear regression. Instead, can we get the series (or in the form of dataframe) of all the slopes of entire stocks that we filetered with the P/E ratio as below?

# Slopes of each stock (I used just dummy numbers, not real slopes.)

AAPL 2.0
MSFT 1.5
AMZN 2.5
XOM 1.5
......

 So that we can play with this data (e.g. ranking, etc.). Thank you.

Hi Hanbyul, the normalized slope here is intended to show the percentage change at each bar. The formula should be 

Slope = Change in Price
Normalized Slope = Change in Price / Bar 

The bar here is 1/252. Thus Normalized Slope = a/b *252

This normalization would allow for comparison of the Regression Slope of different stocks trading in different price ranges. Thus here we divide the slope by the intercept b.

QuantTrader, HanByul_p, the error message comes from the history request. The number of days parameter in self.History(symbols, number of days, resolution) must be int. In Python2, int/int will be automatically converted to int but in Python3, the result of int/int is a float.

# instead of self.lookback = 252/2
self.lookback = int(252/2)

The updated algorithm is attached.

Jing Wu, Thanks!

How to avoid the backtest issue:

When running the backtest I see a bunch of order warnings.  I am pretty sure this means that your algo was trying to enter orders and your portfolio didn't have enough available margin to enter them.  So the backtest isn't accurately representing the algorithm.

In a related matter I think there may be some bug with how the Lean engine is calculating margin requirements on orders, especially when going far back in time.  

I believe there is an error in rule 4. although it is not hit if you change sign:

What would it take to get someone to explain this algorithm in plain language?

NOOB, and can't make any sense of it...

Can someone explain how self.weigth is determined in this algo? As is initialized with 0, but I don't see any part in which is assigned a new value for self.weigth. 

Thanks

Nicolas

Hi Nicolas, the weight is assigned in regression() method.
# Trend is up
if slope > slope_min:
# price crosses the regression line
if delta[-1] > 0 and delta[-2] < 0 and symbol.weight == 0:
symbol.stopprice = None
symbol.weight = slope

# Trend is down
if slope < -slope_min:
# price crosses the regression line
if delta[-1] < 0 and delta[-2] > 0 and symbol.weight == 0:
symbol.stopprice = None
symbol.weight = slope

This is a really interesting algo -- but I'm having trouble getting it working, even with the int(), and even explicitly looking for the key values prior to running dict assignment (on self.price). I either get a version where it skips everything, or it throws a key error on hyg, or other symbols within regression function.

Any ideas?