Runtime Error and Code optimization - Help Wanted

Hi Ryan,

Welcome to QC!

(1) I was not able to reproduce this error when backtesting from Jan 1 2020 to October 30 2020. Can you attach a backtest which demonstrates it?

(2) We can improve the execution speed by making the universe smaller or running with a faster backtesting node. Additionally, we can remove the `GetTopPerformers` method and the `performance` property, since they don't affect the backtest. Lastly, we could reduce the number of History calls by moving the History call before the

loop in OnSecuritiesChanged.

(3) In the OnSecuritiesChanged method, increase the History call period to 40, manually warm-up a Maximum indicator, and register a consolidator to have the indicator updated. Then `Rebalance` can narrow down the selected symbols with

Best,
Derek Melchin

Hello Derek,

Thank you so much for responding to my questions.  This has helped smooth my transition to Quantopian. I was worried when I ran into a wall with these multiple errors.

1) Unfortunately it is not possible to share backtests with the community if they do not complete.  All that was different from the one I posted above and those that would not complete were the start and end dates.  I actually have a backtest saved in my "See Results" tab that did not complete and the start date was on Oct 26 2020 and end date Oct 30 2020.  I can not share for the reason above.  Everything else was exactly the same.  Since you responded, I have only had the error happen once.  I am wondering if it has something to do with my end.  Perhaps because I have several tabs open of the same algo because i'm looking at the code and copying and pasting into a doc so I can compare my changes and results.  I'll report back if I can find what is causing the problem.

2) How much faster are the B2-8, B4-12 and B16-16 than the community B-Micro.  Can I make the assumption that since the B-Micro is on 1 core that the B2-8 would complete the exact same backtest 2x as fast (half time)?  Then the B4 would complete the same backtest 12 times as fast (1/12 time)?  And lastly the B16 would be 16x faster (1/16 time)?  If my assumption is incorrect, how much time would be saved with each option?

3) I will try your suggestion.

4) The ticker "HUGE" does not appear to have traded from 9:18am to 11:am on June 3 2020 according to Tradingview.  Yet if you ask an algo to purchase shares at 9:32 an algo will purchase the shares.  The message "Warning: fill at stale price (06/02/2020 16:00:00 America....".  The share price from 9 to 11am appreas on Tradingview not to trade belwo $8, yet a fill price at 9:32am is for $3.10.  Can you please provide an explanation for why this would fill when the stock may have been halted/not trading and why the fill price does not appear to trade anywhere near what was taking place?

Derek,

1) I conducted several backtests this weekend without issue.  Still no idea why I had those problems early last week.

2) I'll need this info before I can choose a tier.

3) Haven't got that far yet.

4) I have found that market orders will fill even though a stock may be halted.  I had to add "self.UniverseSettings.FillForward = False" in def Initialize so that the order will when the stock is no longer halted.  Problem solved.

5) I have realized when using a gap up filter that I am receiving false gappers due to reverse stock splits.  So a stock at $1 on Wednesday with a 10:1 split becomes $10 on Thursday.  If I'm looking for gap ups Thursday morning this false gapper will be at the top of the list even though I'm using "DataNormalizationMode.SplitAdjusted".  Thus I've been trying to find a way to remove reverse stock split stocks the day of the split.  I was hoping I could use 

if self.HasSplitEventOnNextTradingDay() == True:
                self.Spliteventsec.append(symbol)

then remove the stock from my universe in "OnSecuritiesChanged".  This gives me an error: "AttributeError :  object has no attribute 'HasSplitEventOnNextTradingDay' at OnMarketClose in main.py:linee"

I've now tried to make two dictionaries with the

self.Spliteventbefore[symbol] = self.Value.SplitFactor[symbol] in "def OnSecuritiesChanged" with

self.Spliteventafter[symbol] = self.Value.SplitFactor[symbol] in my "Rebalance(self)" scheduled at 2 min after open with 

if self.Spliteventbefore[symbol] == self.Spliteventafter[symbol]: restricting the proceed if the SplitFactor has changed overnight which it should for a reverse split.

This also give me an error.

Do you have any suggestions how I could track the split event and remove those securities just for the day after the split event?

Hi Ryan,

(1) To share a backtest that won't complete, simply copy the code and paste it into the forum instead of attaching a backtest or just run the algorithm on a single day.

(2) The specs for the backtesting nodes are available in the documentation. The performance improvement is not perfectly linear as described above.

(4) Please attach a backtest which showcases this and we will investigate.

(5) It shouldn't be necessary to remove these stocks from the universe. Instead, update the lookback window when a split occurs. For further assistance with this, attach a backtest.

Best,
Derek Melchin

Hello Derek,

Thank you again for your comments.

4) I've attached a backtest that shows what I described for the symbol "HUGE" on June 3 2020.  With "self.UniverseSettings.FillForward = True" shares are purchased at 9:32 even though Tradingview does not show any volume at that time.  With "self.UniverseSettings.FillForward = False" no shares are purchased.  I think that is the intent of "self.UniverseSettings.FillForward = False", to have no transactions in the backtest if there were no transactions in real life.  I think I read that this is only required for market orders.

5) I don't know how updating the lookback window would solve the issue,  Isn't that the point of using "DataNormalizationMode.SplitAdjusted" so the algo does that for us?  Using the algo I shared with you, by changing the backtest dates I will provide some example of strange price/volume action that I was able to find.  What I am comparing against the backtest volume and price day before/day after is the charts from StockCharts, Yahoo Finance and Tradingview.  MDLY on November 3 and 4 2020 shows that it gapped up but I don't see that it did.  IMAC on March 25 2020 backtest says gapped up but I don't see that it did.  PRTO on Jan 10 has a actually had a reverse split and the backtest says it fills at 9:32am at $17.25 entry and 9:32am at $45.00 exit which is very weird to have such a large range in a single minute of trading (exit will happen if you add a SetRiskManagement(MaximumDrawdownPercentPerSecurity(0.2) to the algo).  ENT on April 16 2020 has a reverse split, does not gap, but shows as a gapper in the algo.  I'm hoping now that I have shared the backtest as per your "quid pro quo", and if these price/volume errors are confirmed, that you will share your knowledge on how I can implement a split event detection/log so that I can more easily single out only split adjusted stocks for inspection instead of inspecting every single transaction manually like I have been doing.  It is rather time consuming to evaluate each and every single transaction given my current suspicion of split adjusted data.

Thanks again for your advice and time.

Hi Ryan,

(4) LEAN is not currently equipped to know when trading is halted. Refer to this related thread. HUGE is not purchased in the algorithm above when using `self.UniverseSettings.FillForward = False` because HUGE isn't added to the `selected` list in `Rebalance`.

(5) Thanks for the thorough reporting. Yes, when using split adjusted normalization, we infact don't need to update any rolling windows when splits occur. However, to ensure we are getting split adjusted data when using a universe selection model, we need to use a security initializer. In the attached algorithm, I've tested each of the tickers listed above. It seems that we don't have data showing splits for those securities at those times. Please submit a data issue to have this resolved.

Best,
Derek Melchin

Derek,

Thank you very much for your help, everything seems to be running fine now.

Have a good day.