QuantConnect Community

Always cool to learn something new. First time I've heard about this Filter! 
Quite simple, but very elegant and probably useful. I'll play with it :) 

Thanks mate for bringing it up! :) 

By the way: to followup on the “Filter Series” - do you know if the Kalman Filter has been already coded somewhere on quantconnect?

Simone Pantaleoni, 
There are thousands of ways to apply the Kalman filter to trading algorithms - from estimating 
the moving average and standard deviation of prices using Kalman filtering 
to generating signals based on forecast errors and dynamically calculating hedge ratios. 
A couple of years ago, I created more than a hundred Kalman Filters applications 
on Quantopian that I had not yet ported to QuantConnect. 
Here is one you can try converting to the QC API.

# Trading QQQ-XLP-TLT using Kalman Filter 
 
import numpy as np
import pandas as pd
from pykalman import KalmanFilter
import statsmodels.api as sm

# ---------------------------------------------------------------------------------------------------------------
STOCK_A, STOCK_D, BOND, PERIOD, MA, LEV, H, M = symbol('QQQ'), symbol('XLP'), symbol('TLT'), 60, 90, 1.0, 0.2, 65
# ---------------------------------------------------------------------------------------------------------------
ASSETS = [STOCK_A, BOND]

def initialize(context):
   context.X = KalmanMovingAverage(STOCK_A)
   context.Y = KalmanMovingAverage(BOND)
   context.kf = None  
   schedule_function(trade,date_rules.week_start(1),time_rules.market_open(minutes = M))
 

def trade(context, data): 
   uptrend = data.current(symbol('SPY'), 'price') > data.history(symbol('SPY'), 'price', MA , '1d').mean()     
   stock = STOCK_A if uptrend else STOCK_D
   
   if context.kf is None:
       initialize_filters(context, data)
       return
   if get_open_orders(): return

   prices = np.log(data.history(ASSETS, 'price', PERIOD, '1d'))
   context.X.update(prices)
   context.Y.update(prices)
   
   mu_Y = context.Y.state_means
   mu_X = context.X.state_means
   
   frame = pd.DataFrame([mu_Y, mu_X]).T    
   context.kf.update(frame.iloc[-1])    
   beta, alpha = context.kf.state_mean    
   spreads = (mu_Y - (beta * mu_X + alpha)).tail(PERIOD)    
   zscore = (spreads[-1] - spreads.mean()) / spreads.std()   
   
   record(beta = beta, alpha = alpha, mean_spread = spreads[-1], zscore = zscore)   
   
   wt_stk = LEV*(1.0 - H) if zscore > 0 else LEV*H
   wt_bnd = LEV - wt_stk
   
   for asset in context.portfolio.positions.keys():        
       if asset is not stock and asset is not BOND:
           if data.can_trade(asset):
               order_target_percent(asset, 0)
   
   if all(data.can_trade([stock, BOND])):
       order_target_percent(stock, wt_stk)
       order_target_percent(BOND, wt_bnd)

       
def before_trading_start(context,data):        
   record(leverage = context.account.leverage) 
       
   
def initialize_filters(context, data):
   prices = np.log(data.history(ASSETS, 'price', PERIOD, '1d'))
   context.X.update(prices)
   context.Y.update(prices)
   
   context.X.state_means = context.X.state_means.iloc[-PERIOD:]
   context.Y.state_means = context.Y.state_means.iloc[-PERIOD:]    
   context.kf = KalmanRegression(context.Y.state_means, context.X.state_means)

class KalmanMovingAverage(object):
   # Estimates the moving average of a price process via Kalman Filtering.    
   # See http://pykalman.github.io/ for docs on the filtering process. 
  
   def __init__(self, asset, observation_covariance=1.0, initial_value=0,
                initial_state_covariance=1.0, transition_covariance=0.05, 
                initial_window=20, maxlen=300, freq='1d'):
       
       self.asset = asset
       self.freq = freq
       self.initial_window = initial_window
       
       self.kf = KalmanFilter(transition_matrices=[1],
                              observation_matrices=[1],
                              initial_state_mean=initial_value,
                              initial_state_covariance=initial_state_covariance,
                              observation_covariance=observation_covariance,
                              transition_covariance=transition_covariance)
       self.state_means = pd.Series([self.kf.initial_state_mean], name=self.asset)
       self.state_covs = pd.Series([self.kf.initial_state_covariance], name=self.asset)
       
       
   def update(self, observations):
       for dt, observation in observations[self.asset].iterkv():
           self._update(dt, observation)

       
   def _update(self, dt, observation):
       mu, cov = self.kf.filter_update(self.state_means.iloc[-1],
                                       self.state_covs.iloc[-1],
                                       observation)
       self.state_means[dt] = mu.flatten()[0]
       self.state_covs[dt] = cov.flatten()[0]  
     
       
class KalmanRegression(object):
   # Uses a Kalman Filter to estimate regression parameters in an online fashion.    
   # Estimated model: y ~ beta * x + alpha
   
   def __init__(self, initial_y, initial_x, delta=1e-5):
       self._x = initial_x.name
       self._y = initial_y.name
       trans_cov = delta / (1 - delta) * np.eye(2)
       obs_mat = np.expand_dims(
           np.vstack([[initial_x], [np.ones(initial_x.shape[0])]]).T, axis=1)
       
       self.kf = KalmanFilter(n_dim_obs=1, n_dim_state=2,
                              initial_state_mean=np.zeros(2),
                              initial_state_covariance=np.ones((2, 2)),
                              transition_matrices=np.eye(2),
                              observation_matrices=obs_mat,
                              observation_covariance=1.0,
                              transition_covariance=trans_cov)
       state_means, state_covs = self.kf.filter(initial_y.values)
       self.means = pd.DataFrame(state_means, 
                                 index=initial_y.index, 
                                 columns=['beta', 'alpha'])
       self.state_cov = state_covs[-1]

       
   def update(self, observations):
       x = observations[self._x]
       y = observations[self._y]
       mu, self.state_cov = self.kf.filter_update(self.state_mean, self.state_cov, y, 
                                                  observation_matrix=np.array([[x, 1.0]]))
       mu = pd.Series(mu, index=['beta', 'alpha'], 
                      name=observations.name)
       self.means = self.means.append(mu)
       
   def get_spread(self, observations):
       x = observations[self._x]
       y = observations[self._y]
       return y - (self.means.beta * x + self.means.alpha)
       
   @property
   def state_mean(self):
       return self.means.iloc[-1]
   
   
'''
100000

START
06/01/2007
END 
01/17/2018

STOCK_A, STOCK_D, BOND, PERIOD, MA, LEV, H, M = symbol('QQQ'), symbol('XLP'), symbol('TLT'), 60, 90, 1.0, 0.4, 65

Total Returns
224.8%
Benchmark Returns
126.36%
Alpha
0.09
Beta
0.23
Sharpe
1.22
Sortino
1.77
Volatility
0.09
Max Drawdown
-14.3%

'''

You may also search QuantConnect forum for "Kalman Filter".
Here are some of them

From Research To Production: Kalman Filters and Pairs Trading
https://www.quantconnect.com/forum/discussion/6826/from-research-to-production-kalman-filters-and-pairs-trading

Ernie Chan's EWA/EWC Pair Trade with Kalman Filter
https://www.quantconnect.com/forum/discussion/2331/ernie-chan-039-s-ewa-ewc-pair-trade-with-kalman-filter/p1

http://pykalman.github.io/

Also try this

Thanks Vladimir  :)  no need to search for me, really :P 

But very much thanks for your feedback - I'll work on traslating your code and I'll post it here. I'll also experiment a bit with this Laguerre Filter, since I see a lot of possible potential there

ps. I had already seen the example you've posted from the community - but wanted to have some more feedback from you

Interesting thread and another interesting filter. Thanks for sharing Vladimir.  I still need to learn the possible applications. 

On that note, Simone Pantaleoni, what applications of this come to mind? how might you consider using this filter? It's not immediately apparent to me. eg: would you use it as you would a moving average cross over? Or would you apply it to some other series other than price? If so, which? Etc etc.

Simone Pantaleoni,

A while ago you asked me: 

do you know if the Kalman Filter has been already coded somewhere on quantconnect?

Today I posted my Kalman Filter on quantconnect forum.

Vladimir Mate, you read my mind. Application on BTC momentum trading strategies was exactly the purpose why I was interested on recreating the Kalman filter ❤️

This let me think that we should talk more often! :P 

Simone Pantaleoni,

Try this one

Vladimir : for the Kalman filter, how come you have such a long warmup period? IE: Why 5 * Period? What is the significance of the constant '5'?

Also, Vladimir  : I noticed that in your original Kalman version you were calculating based on the Close of the Tradebar, but here you are calculating based on the Low of the Tradebar. Why is that? was it to make it a little less responsive / give it more room?  In what circumstances would you calculate based on Close, Low, or perhaps even High?

@vladimir_2 I told you that you read my mind! :P 

That was the exact use of the Kalman filter I was working on - just adding a triple momentum will however reduce DD and improve overall performance :) 

Check this out mate 

Simone Pantaleoni, .ekz.,

It would be better if we moved our talk about Kalman filters to “Share: Kalman Filter Crossovers for Crypto and ‘Smart’ RollingWindows”.