Linear filter used in a custom indicator gives incorrect results

Hi Guys,

I am trying to implement a custom indicator based on Butterworth filter from scipy.signal. It is possible to use scipy.signal.lfliter on an entire series as well as on a stream values (see the example here: https://stackoverflow.com/questions/40483518), so this should work well inside of the Update method of a PythonIndicator class. However I am getting slightly different results from a custom indicator:

from scipy.signal import butter, lfilter, lfilter_zi
import numpy as np
from QuantConnect.Indicators import PythonIndicator
class ButterworthFilterBasic(PythonIndicator):
   def __init__(self, period=5):
       self.Period = period
       self.b, self.a = butter(3, 1/self.Period, btype='lowpass')
       self.zi = lfilter_zi(self.b, self.a)
       self.Value = np.nan
       self.barNum = 0
   
   def Update(self, input):
       y, self.zi = lfilter(self.b, self.a, [input.Close], zi=self.zi)
       self.Value = y[0]
       self.barNum += 1
       return self.barNum > self.Period

 when I use it in a backtest:

from butterworth import ButterworthFilterBasic
class ButterworthTest(QCAlgorithm):
   
   def Initialize(self):
       self.SetStartDate(2019, 1, 1)
       self.SetEndDate(2021, 1, 1)
       self.spy = self.AddEquity('SPY', Resolution.Daily)
       self.bf5 = ButterworthFilterBasic(5)
       self.RegisterIndicator(self.spy.Symbol, self.bf5, Resolution.Daily)

       
   def OnData(self, data):
       self.Plot('Trade Plot', 'Price', data.Bars[self.spy.Symbol].Close)
       if self.bf5.IsReady:
           self.Plot('Trade Plot', 'BF5', self.bf5.Value)

the resulted plot is not smooth:

This is what I expected to see:

This second image was captured from QuantBook where it was produced by the following code:

# QuantBook Analysis Tool 
# For more information see [https://www.quantconnect.com/docs/research/overview]
qb = QuantBook()
spy = qb.AddEquity("SPY")
history = qb.History(qb.Securities.Keys, 500, Resolution.Daily)
import numpy as np
import pandas as pd
from scipy.signal import lfilter, lfilter_zi, butter
b, a = butter(3, 1/5, btype='lowpass')
y1 = lfilter(b, a, history.close.values)
y2 = np.zeros(y1.shape[0])
z = lfilter_zi(b, a)
for i, x in np.ndenumerate(history.close.values):
   y_, z = lfilter(b, a, [x], zi=z)
   y2[i] = y_[0]
   
bf = pd.DataFrame(index=history.index)
bf['BF1'] = y1
bf['BF2'] = y2
pd.concat([history.close, bf], axis=1).iloc[150:220].plot(figsize=(36, 24), style=['b.-', 'r.-', 'ro-'])

In the research code there are 2 filtered series:

• y1 calculated from the entire close price series at once,
• y2 calculated from individual samples one by one in a for loop.

On the plot both series are matching perfectly: the red line with big dot markers (y2) covers the red line without markers (y1). This proves that both filtering methods produce exactly the same results.

Why then the custom indicator (ButterworthFilterBasic) gives incorrect results? With the 3rd order Butterworth filter the resulting line should be smooth. It seems like the lag from the custom indicator is the same as on the second plot, but the line is not as smooth. I hope this can be fixed and someone helps me spot the error.