Seeing faulty backtest patterns in a LOT of strategies posted

I'm noticing a lot of people posting strategies on QuantConnect using the following anti-pattern:

class SomeAlgo(QCAlgorithm):
  def initialize(self):
     
     self.entity = self.add_equity("SOME_SYMBOL", Resolution.DAILY).symbol
     
     self.schedule.on(
         self.date_rules.every_day(self.entity), 
         self.time_rules.before_market_close(self.entity, 5), 
         self.rebalance
     )
     ...
     
  def rebalance(self):
  
    self.set_holdings(self.entity, some_fraction)
    

The backtest reports strong values, but there's a severe backtest flaw in the above pattern.

Because self.entity  is set to Resolution.DAILY,  self.set_holdings  will fill the price to the last closing bar price.  In the above, because we are 5 minutes away from market closing, it will fill the price at yesterday's close price. What does this mean? We can cheat by doing something like:

class FakeAmazingAlgo(QCAlgorithm):
  def initialize(self):
     
     self.entity  = self.add_equity("SPY", Resolution.DAILY).symbol
     
     # Usually, this is SMA or something derived from minute data, but I'll just 
     # set it directly to price to prove a point
     self.entity_minute = self.add_equity("SPY", Resolution.MINUTE).symbol

     self.schedule.on(
         self.date_rules.every_day(self.entity), 
         self.time_rules.before_market_close(self.entity, 5), 
         self.rebalance
     )
     ...
     
  def rebalance(self):
    # Note, this is usually hidden with some indicator like SMA, EMA, etc.
    indicator_value = self.securities[self.entity_minute].price 
    indicator_threshold = self.securities[self.entity].price
    
    if indicator_value > indicator_threshold:
       self.set_holdings(self.entity, 1.0)
    else:
       self.liquidate(self.entity)
       

In the above, we are literally doing the following: we will buy SPY at yesterday's close price if the price right now is greater than yesterday's close price. Otherwise, liquidate at yesterday's close price.

Too many people are using the above anti-pattern. The correct pattern is either:

1. Change the resolution of the entity you are trading to MINUTE

class SomeAlgo(QCAlgorithm):
  def initialize(self):
     
     ## Change the resolution of entity we are trading to MINUTE so that it's price is updated by minute
     self.entity = self.add_equity("SOME_SYMBOL", Resolution.MINUTE).symbol 
     
     
     self.schedule.on(
         self.date_rules.every_day(self.entity), 
         self.time_rules.before_market_close(self.entity, 5), 
         self.rebalance
     )
     ...
     
  def rebalance(self):
  
    self.set_holdings(self.entity, some_fraction)
    

OR 2.  Use self.market_on_close_order with at least 16 minutes before closing (need to check, but IIRC, it was minimum of 15 minutes before closing to establish a MOC order)

class SomeAlgo(QCAlgorithm):
  def initialize(self):
     
     self.entity = self.add_equity("SOME_SYMBOL", Resolution.DAILY).symbol
     
     self.schedule.on(
         self.date_rules.every_day(self.entity), 
         self.time_rules.before_market_close(self.entity, 5), 
         self.rebalance
     )
     ...
     
  def rebalance(self):
  
    self.market_on_close_order(self.entity, ... ) 

Here's an example of a flawed algorithm (was scratching my head why this was performing so well in backtest but performance differed in live).