Universe Based RSI Selection

When Quantconnect grabbed the backtest, it grabbed the code used for that backtest, which does not grant very much in the way of copyright. I am in fact granting the code to the public domain. Here is the same algo with a better copyright notice.

Hey Warren! Thanks for sharing! By sharing public algorithms on QuantConnect you're releasing any copyright ownership and putting them into public domain -- this is detailed here: quantconnect.com/terms

The key to understanding the downturn is really understanding what your algorithm is doing; in this case it seems like you're using moving average and RSI on a portfolio of algorithms. They are both trending indicators so its likely the tickers you were following were not trending well during the downtime. This could also be high volatility causing excess trading and eating up any gains. 

Hi Lucas. The code is much the same but it's been updated here and there. I've added a lot rounder to deal with the odd lot problem and cleaned up things. The strategy is fundamentally different than the original 'Bounce' algo. This algo is basically mean reversion with some short term RSI thrown in.  The RSI will make sure the stock has finished falling and has turned around some instead of buying as soon as it hits 10% under the EMA.. In the same way it lets the stock continue rising some when it hits the 10% over EMA point until the short term RSI shows that it has started to fall. You can change the resolution to Minute and the dollar volume based trade sizing should be ok. If you go with a resolution other than minute or daily you'll have to update the trade sizing code to deal with that. The move to limit orders is complicated but gets rid of slippage problems accurately.

I received a detailed question by email concerning the operation of the algo, I'm providing most of the answer here for those interested.

From Investopedia.com, "Mean reversion is the theory suggesting that prices and returns eventually move back toward the mean or average.". The algo is effectively a mean reversion algo at heart. 

EMA = exponential moving average.

RSI = relative strength index.

This dictionary:
Dictionary<string,ExponentialMovingAverage> movingAverages=new Dictionary<string,ExponentialMovingAverage>();
is central to the strategy and holds one exponential moving average indicator for every stock, 500 altogether in the algo posted, one for every stock in the S&P 500.  Each Key holds a ticker symbol and the Value of each dictionary entry holds an exponential moving average indicator.

decimal lowerBandRatio=0.9m; indicates that you want to buy when a stock falls below 90% of the EMA for that stock.

decimal upperBandRatio = 1.1m; indicates that you want to sell when a stock hits 110% of the EMA for that stock.

"int maPeriod = 60;" indicates that the algo is to consider the last 60 days when calculating the EMA of each stock.

The rsisBuy and rsisSell dictionaries hold a short term RSI indicator for each stock, in a manner similar to the main moving average indicator. These RSI indicators aren't strictly necessary in a mean reversion algo but I included them to ensure that the stock has stopped falling to some extent when you buy and stopped rising to an extent when you sell. The default value of "int rsiPeriod = 3;" indicates that the RSI indicators are to consider momentum during the last 3 days.

RSI values range from 0 to 100 with 100 being strong positive momentum and 0 being strong negative momentum.

So, a value of 50 for the "decimal rsiBuyCutoff=50;" just indicates that you want to buy when the stock has stopped falling in the short term and looks like it might be turning around. A mean reversion strategy without this short term check will tend to buy stocks that are falling through the floor with no regards for negative momentum.

The "decimal rsiSellCutoff=50;" works just like the "decimal rsiBuyCutoff=50;" except that it lets the stock continue rising if the short term momentum is positive before selling.

The other dictionaries contain copies of the limit orders made, the durations that the limit orders have been in effect, and the prices of the buy orders. You shouldn't have to modify these or adjust them at all to change the strategy of the algo, and in fact you can just leave them as is and change the entire strategy of the algo by changing the dictionaries containing indicators, the code in the buy and sell functions, and the variables concerning strategy. The main purpose of these dictionaries is to hold copies of the orders until they expire, so a count for how long an order has been in existence needs to be stored for each and every order. "barsToHoldBuyOrdersFor" and "barsToHoldSellOrdersFor" indicate how long you want to keep your orders in effect for and are the only things you are likely to want to modify concerning orders.

"ratioOfDollarVolumeForMaxTrade " prevents the algo from attempting to invest too much in low dollar volume stocks.  The default value of ".25m / 6.5m / 60m;" for daily resolution indicates that you don't want to invest more than 15 seconds worth of daily dollar volume. The .25 indicates a quarter of a minute, the 6.5 divides the daily dollar volume by the number of trading hours in a day, and the 60 by the number of minutes in an hour. So the average amount traded in 15 seconds of the day becomes the maximum trade size allowed by the algo.

The RoundLot function can be copied and pasted out of the algo into your own algos if you want it. It prevents 'odd lots' from being ordered by rounding off the quantity of stocks being ordered to an acceptable number. Google 'odd lot' if you are unfamiliar with this concept.
 

@Warren, I suggest you to use a class to hold the symbol data instead of those dictionaries:

And save an object for each security in a list:

Your code will be more reusable and you will be able to use Linq extensions, for example:

Looking at your code, I also noticed that you have two RSI indicators with the same period. Do you intend to use a RSI with a different period for buy and sell signal in the another algorithm?
 

You're right about the extra RSI indicator. The code will run a bit faster without the extra RSI indicator. Moving to a class will also get rid of some of the parallel dictionaries. I'm not sure if I want to move to lists over dictionaries altogether though, I'll have to be careful if I do, I chose dictionaries to make sure I get the right data back  by ticker instead of indexes, even though the dictionaries are slower. The algo relies on this for stability. In a similar manner the TradeBars can't just be accessed by numeric index because they won't always have every ticker in them. If I'm careful I could get a List to match the tickers by index and iterate through that to speed up the algo a fair bit. The orders would need their own lists/dictionaries due to how the algo works as best I can tell. It kind of evolved to where it's at starting from one of the basic examples on QuantConnect so ya, it could be cleaned up a fair bit at this point.

Nice work (especially the explanation), always interesting to see other people's reasoning. Just be careful of S&P500 membership bias, in case you used the current S&P500 to select symbols.

@Warren, if you do not want to move away from all dictionaries, you just keep one with the symbol data class:

I agree with Petter.
Maybe you could try to integrate universe selection in the algorithm to avoid membership/survivorship bias.

Hi Alexandre, thanks for the suggestions. I usually go with a List of classes over parallel arrays but got lazy evolving the code. I moved to a List of SymbolData classes. This will no doubt speed up the Buy routine as the algo now accesses the List of SymbolData classes by index. The Sell routine needs to access the SymbolData by ticker, so there's now a Dictionary of SymbolData classes as well. Ugly, but fast with regards to processor time. The long lists of tickers I've been using really suggest index access with a List where possible. The SymbolData classes are stored byref in both the List and Dictionary so there shouldn't be too much memory loss. I'm testing with minute data now, on fairly long lists of tickers. The order Dictionaries I left as is for the time being as the code seems stable and the pertaining Dictionaries probably don't eat up too much time given the short lists of orders.

I tried the universe selection before but couldn't iterate through the stocks in the universe from within the Initialize routine if I recall the problem correctly. Anyone know if this has been fixed or if there is some way of enumerating the stocks in the universe that I missed?

I'll post an update after I play with the new code for a while.

Yes Jared, I need to set the FeeModels and I need ticker symbols to create indicators for each stock. It would be nice if I could just get a list of tickers corresponding to the securities in the universe from anywhere in the algo. Or a list of Security objects with the ticker symbols on them, or something like that.

Being able to use code like this:

            foreach (Security s in Securities.Values)
            {
                s.FeeModel=new CustomFeeModel();
            }

...would be nice, it's what I'm doing now. Maybe it works now it's been a while since I tried the universe stuff. Thanks.

Nevermind about the universes Jared, I found this:

https://github.com/QuantConnect/Lean/blob/master/Algorithm.CSharp/EmaCrossUniverseSelectionAlgorithm.cs

It uses indicators and universe selection and provides a nice example.

About the SecurityInitializer class Jared has mentioned, there is this example: 
- CustomSecurityInitializerAlgorithm.cs

Applying it to your algorithm:

Your CustomSecurityInitializer woulb be:

Thanks Alexandre, that'll come in handy if I move to universe selection.

It's coming along. I moved to universe selection and got rid of the Dictionaries in favor of one ConcurrentDictionary and two lists. I just have to clean up an update as I have a bunch of excess code in there where I was experimenting. Sorry for the delay.

Here's an update, for what it's worth. The profit comes in spikes so I don't trust it. Last time I investigated a 'too good to be true' spike it was from a reverse split that wasn't accounted for. You'ld expect a nice slowly ascending graph. I am going to keep the updated code though, the changes make the code base more appealing. The code needs to be checked for bugs still, I wouldn't trust it, unless you want to work the glitches out of it and use it as a platform.

Here's another update. Let me know if you find any problems.