Help me improve this retiree friendly low volatility stock algo with +93% PSR

Personally, I would use a more robust estimator for volatility than the sample standard deviation.

For one, we do have the arch package and thus the following volatility processes are available:

https://arch.readthedocs.io/en/latest/univariate/volatility.html

In terms of shrinkage/robust sample estimation (i.e., it does not produce a "volatility process", just an estimate for variance altogether), one can employ methods from:

https://scikit-learn.org/stable/modules/covariance.html

There is also the simple method of using EWM https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.ewm.html

Thanks Aaron.

I tried getting started with the EWM, but can't get it working. This is my volatility section at the end:

def Volatility(self):
        data = [float(x.Value) for x in self.Price]
        return pd.ewm(data, span = self.lookback)

what do I have wrong?

Mark, there are problem with the data types, you need to cast the data, a list, as a pandas DataFrame, then call the EWM and the function you will apply it to, variance I guess in this case, and then return the number you obtain:

This is very inefficient due to dataframe abuse, but there seems not to be a numpy implementation for exponential weighted functions.

In any case, I would change the volatility calculation to (ewm) std of daily returns for an absolute price independant measure:

Thanks Hector.

I put this in and the returns have dropped a lot, seems this particular calculation does not help my strategy. I'll try using the covariance next.

Hi Mark,

Thanks for sharing this algorithm with the community!

The original algorithm is currently trading at stale prices. To resolve this, we need to subscribe to a data resolution higher than daily (hour, minutes, second) since we're trading 30 minutes after the open.

To continue the development of this strategy, consider replacing the volatility calculation with the StandardDeviation indicator. Once initialized, setup a consolidator to have it updated at each time step.

Best,
Derek Melchin

Thanks Derek. Yes i do have the resolution set to daily, it makes the backtest run much faster than minute.  But I'll change the scheduled time for 60 minutes after the open, and use Hour resolutions for everything. That should be ok.

I'll put in the SD indicator, although I think that will give similar results from my original. But will the indicator   implementation help it run faster? My original version takes hours to run.

Hi Mark,

With hourly resolution, the first trading bar arrives at 10AM. Since RTH start at 9:30, if we want to trade 60 minutes after the open, we'll need to use minute data (or a higher resolution) to avoid stale fills. To stick with the hourly resolution, we'd need to rebalance at a full hour (10AM, 11AM, 12PM, etc) instead of at 10:30AM.

Yes, using the SD indicator should speed up the execution.

Best,
Derek Melchin

Thanks Derek. For the resolution I have always thought about market open time being the start, not the actual clock time. Newbie mistake :)

Does anyone know why this is not buying the right amount of stocks? My filtering gets down to a list of 11 stocks, but only buys 10. When i lower the fine results to 10, it only buys 9. Anyone know why or is this a bug?

Hi Mark,

I was unable to reproduce the issue from the backtest above.

Best,
Shile Wen

Hi Shile, I'm not seeing what you are. Here is what I'm seeing.

In this backtest I changed the self.stocks number to 10. So if you look at orders it should buy 10 stocks plus the UST, but it doesn't. It only buys 9 stocks plus UST. If I comment out the UST SetHoldings line, it buys 10 stocks like it should.

What am I missing?

Hi Mark,

This is because UST is part of the SymbolData due to the code in OnSecuritiesChanged, and the list of stocks is generated from the SymbolData. To address this, we can add a conditional to make UST and SPY be excluded from the SymbolData dict. I've shown this in the attached backtest.

Best,
Shile Wen

Thats great, thanks Shile!

I added some more fundamental filters, and tweaked the coarse and fine filtering counts. This improved return and PSR, but drawdown increased. But if you remove the UST hedge and go back to 2000 the returns are almost 5x from my original, so I think that is worth it.

I still plan on trying some other volatility variations to see what comes up.