Back

Pairs Trading - Copula Method vs Cointegration

I finished a pairs trading strategy based on copula method. Please look into tutorial page for details. In order to compare the performance of copula pairs trading technique, I also implemented a simple cointegration methods for comparison. Leave a comment if anyone has questions or suggestion about my implementation.

Update Backtest






The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



Hi Jing, this is really great. Thanks for sharing your work. I am looking forward to exploring the math further. Your explanation seemed to really stream-line the difficult/important concepts into a more understandable framework, which I appreciated. Is there anything you were able to identify for why the strategy's activity slowed down from October 2014 to December 2015?

0

Hi FX trader,  For this pairs trading method, the chosen pair is “GLD” & “DGL”. In comparison to the profitable period from October 2012 to Feb 2014, if we take a close look at the historical log returns from October 2014 to December 2015,  the returns are less volatile and the 2 curves almost overlap with each other. Not much price divergence for pairs traing profits. Although we have trades during this period, the profits offset from longing and shorting this pair simultaneously 

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


It would be a good project to try copula pairs trading on not just ETF but stock pairs because of the volatility of stock market. It is promising to try high-frequency pairs trading with copula method. See Xiaowei's great example for High Frenquency Pairs Trading for cointegration Method Based on Stock Market 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


 -- sorry for my late reply. Thanks for taking the time to look in to this and get back to me. This makes me interested in having a sort of dormant tag, that attempts to measure and perceive how similar current market conditions are with historical conditions that have been identified to place ideal trades. More for a reference framework. And thanks for sharing this strategy too

0

This is the new version of pairs trading algorithm with copula method. I fixed the bug of the unsupported decimal and float multiplication.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Just a question: I cloned it and ran it. I got bunch of errors with no output. Is that normal?
0

Hi Manoj Gupta, try to clone the new version algorithm in my last comment or check tutorial page here https://www.quantconnect.com/tutorials/pairs-trading-strategy/ for the updated algorithm

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jing for posting this strategy. This is great work! I have been taking a look and it appears to trade just one pair at a time by design. Would it be easy to amend the code to have it trade the top 3 pairs based on the Kendall rank rather than only the max?
0

Hi Hugh Donnelly, it can be amended to trade multiple pairs at the same time, but the performance based on the candidate pairs you choose. It's kind of difficult because the copula parameter estimation process has to be adjusted for multiple pairs in Algorithm initialization and then the Ondata trading part accepts different input and trigger the different signals. Almost all the parts have to be amended. It might be more efficient to just change the candidate pairs in algorithm if you want to see the performance of copula method for other pairs.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jing Wu -- Thanks, I will take a look.

0

I am very new here, and not sure yet how to do many even simple tasks, so apologize for not doing the work!  In this instance, visually speaking, it appears the algo is correlated with SPY, but I am not sure how to numerically prove this claim: i see beta=0.57, and given that volatility of your portfolio is most certainly lower than that of SPY itself, it means the correlation with SPY is definitely above that number. That should not be the case for a pair trade between SPY and DIA - but it seems there is a long market bias in the strategy right now.  Maybe there is something in portfolio construction that needs to be addressed?

1

Hi,

Thanks for the post and introducing me to this novel technique. I am wondering why you use the "hedge ratio" as below:

self.coef = stats.linregress(x,y).slope

Shouldn't the portfolio be dollar neutral? That is equal amount of both pairs with one long and one short?

 

Best regards,

Pravin

0

Hi Satyapravin, dollar neutral is one of the methods to decide the hedging ratio in pairs trading. Here I use the beta neutral hedge ratio Price(Stock A) - beta * Price(Stock B) (beta is the coefficient in simple linear regression) not the dollar neutral ratio. Then I purchase x shares of stock A and short x*beta shares of stock B.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Karen Chaltikian Thank you for your question! It's a very interesting perspective.

I understand what you are saying, and I test the market volatility within the same period, which is 0.14, and the algorithm's volatility is 0.104. In this case, the correlation coefficient between the market and the algorithm must be a number higher than 0.57. 

I will try to look into the portfolio construction! Hopefully we can figure it out.

0

Hi Jingw, I have some questions:

1). Is this supposed to pick only one pair out of 8 pairs?  2). Can I put 100 pairs in your code 'def _pair_selection()'?  3). I cloned the fixed version but it showed different result from the one you posted. What am I missing here? (See attached backtest.)  4). How does this algo control leverage? In the code, the quantity of leg1 is set up always 0.4. I am curious about total maximum leverage.  5). In your tutorial, you said "...We use the first 3 years of data to choose the best fitting copula and asset pair ("training formation period"). Next, we use a period of 5 years from 2011 to 2017 ("the trading period"), to execute the strategy...." What does this mean? Is this sort of Machine Learning algo? What does exactly mean by 'training formation period' and 'trading period'? 

0


Hi 

I changed a little: trading everyday after 15 min. after market open, and added 'QQQ', 'XLK'. I got en error message as below around November 7~14, 2016:

Runtime Error: System.Exception: BacktestingRealTimeHandler.Run(): There was an error in a scheduled event XLK: EveryDay: XLK: 15 min after MarketOpen. The error was Python.Runtime.PythonException: KeyNotFoundException : 'QQQ' wasn't found in the Slice object, likely because there was no-data at this moment in time and it wasn't possible to fillforward historical data. Please check the data exists before accessing it with data.ContainsKey("QQQ")
at QuantConnect.Data.Slice.get_Item (QuantConnect.Symbol symbol) [0x0002d] in <8f9e699bbc3f46739adc6b359ebcd5b9>:0
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in <dca3b561b8ad4f9fb10141d81b39ff45>:0
at Python.Runtime.Dispatcher.Dispatch (System.Collections.ArrayList args) [0x00018] in <387056c9810b431d9b668f2df5d6c027>:0
at __System_ActionDispatcher.Invoke () [0x00006] in <166e461c70884c28abc297d007aa4fef>:0
at QuantConnect.Scheduling.ScheduleManager+<>c__DisplayClass16_0.<On>b__0 (System.String name, System.DateTime time) [0x00000] in <8f9e699bbc3f46739adc6b359ebcd5b9>:0
at QuantConnect.Scheduling.ScheduledEvent.OnEventFired (System.DateTime triggerTime) [0x00036] in <8f9e699bbc3f46739adc6b359ebcd5b9>:0

To me, it's impossible not to find the symbol 'QQQ'. What's happening here? Should I change the trading schedue? Thanks :)

0

I am not sure why you are having a problem. I have modified the algorithm to only trade 'QQQ' and 'XLK', and set the schedule to 30 minutes after market open on the same month start days. Maybe you can see what is different between this version and yours to help you.

1


Hi JingwKaren Chaltikian, As I said above ("...trading everyday after 15 min. after market open..."), I changed the trading interval from once a month to a daily basis. I tried different schedule but in November 2016, the same error message showed up. I tried with Jingw's version and Karen's version, but both had same error messages.                                           

self.Schedule.On(self.DateRules.EveryDay(self.syl[0]),self.TimeRules.AfterMarketOpen(self.syl[0],15),Action(self._set_signal))
self.Schedule.On(self.DateRules.EveryDay(self.syl[1]),self.TimeRules.AfterMarketOpen(self.syl[1],15),Action(self._set_signal))

I don't know what happened then. I guess there's a bug we missed. I will look into this more in detail. Any suggestions and comments are welcome. Thanks :)

0

Jingw, I have more questions. Please see the attached backtest for the period from 2016,01,01 to 2017,9,20 and its trades logs as below. As you can see below, the quantity for both legs decreased dramatically from 2016-02-19. 

self.Schedule.On(self.DateRules.MonthStart(self.syl[0]), self.TimeRules.AfterMarketOpen(self.syl[0],30), Action(self._set_signal))
self.Schedule.On(self.DateRules.MonthStart(self.syl[1]), self.TimeRules.AfterMarketOpen(self.syl[1],30), Action(self._set_signal))
Time Symbol Price Quantity Type Status Value
2016-01-22T05:00:00Z XLK 38.8754958 -1049 4 Filled -40780.39509
2016-01-22T05:00:00Z QQQ 100.9991229 995 4 Filled 100494.1272
2016-01-23T05:00:00Z QQQ 101.6080246 -995 4 Filled -101099.9845
2016-01-23T05:00:00Z XLK 38.9530916 1049 4 Filled 40861.79309
2016-01-23T05:00:00Z XLK 38.9530916 2076 4 Filled 80866.61816
2016-01-23T05:00:00Z QQQ 0 -1970 4 0 0
2016-01-30T05:00:00Z XLK 39.83574383 -1049 4 Filled -41787.69527
2016-01-30T05:00:00Z QQQ 101.7651606 995 4 Filled 101256.3347
2016-02-19T05:00:00Z XLK 39.2634748 -15 4 Filled -588.952122
2016-02-19T05:00:00Z QQQ 99.10367064 14 4 Filled 1387.451389
2016-02-23T05:00:00Z XLK 39.79694593 10 4 Filled 397.9694593
2016-02-23T05:00:00Z QQQ 100.8321659 -9 4 Filled -907.4894934
2016-03-02T05:00:00Z XLK 40.7765929 10 4 Filled 407.765929
2016-03-02T05:00:00Z QQQ 103.7391807 -9 4 Filled -933.6526267
2016-03-16T04:00:00Z XLK 41.5913488 -4 4 Filled -166.3653952
2016-03-16T04:00:00Z QQQ 104.4659345 3 4 Filled 313.3978034
2016-03-18T04:00:00Z XLK 42.60086073 -2 4 Filled -85.20172146
2016-03-18T04:00:00Z QQQ 105.8493788 1 4 Filled 105.8493788
2016-04-07T04:00:00Z XLK 43.10789775 16 4 Filled 689.726364
2016-04-07T04:00:00Z QQQ 108.3808594 -14 4 Filled -1517.332031
2016-06-08T04:00:00Z XLK 43.08839633 -1 4 Filled -43.08839633
2016-06-30T04:00:00Z XLK 42.1408428 5 4 Filled 210.704214
2016-06-30T04:00:00Z QQQ 105.2026139 -4 4 Filled -420.8104556
2016-07-01T04:00:00Z XLK 42.53285064 -15 4 Filled -637.9927596
2016-07-01T04:00:00Z QQQ 106.1606174 13 4 Filled 1380.088027
2016-09-17T04:00:00Z XLK 46.58008455 34 4 Filled 1583.722875
2016-09-17T04:00:00Z QQQ 116.4017414 -30 4 Filled -3492.052241
2016-09-20T04:00:00Z XLK 46.53087677 -5 4 Filled -232.6543838
2016-09-20T04:00:00Z QQQ 116.0304518 4 4 Filled 464.1218073
2016-10-29T04:00:00Z XLK 46.80644034 -2 4 Filled -93.61288067
2016-10-29T04:00:00Z QQQ 116.3571866 1 4 Filled 116.3571866
2016-11-08T05:00:00Z XLK 46.3438872 2 4 Filled 92.68777441
2016-11-08T05:00:00Z QQQ 115.0601485 -1 4 Filled -115.0601485
2016-11-10T05:00:00Z XLK 46.84580656 3 4 Filled 140.5374197
2016-11-10T05:00:00Z QQQ 116.971052 -2 4 Filled -233.9421039
2016-12-08T05:00:00Z XLK 47.43629992 -4 4 Filled -189.7451997
2016-12-08T05:00:00Z QQQ 117.2284794 3 4 Filled 351.6854381
2016-12-17T05:00:00Z XLK 48.07476211 6 4 Filled 288.4485727
2016-12-17T05:00:00Z QQQ 118.9056876 -5 4 Filled -594.528438
2016-12-20T05:00:00Z XLK 48.53939584 -3 4 Filled -145.6181875
2016-12-20T05:00:00Z QQQ 119.5908799 2 4 Filled 239.1817598
2017-01-06T05:00:00Z XLK 48.4800809 4 4 Filled 193.9203236
2017-01-06T05:00:00Z QQQ 120.146978 -3 4 Filled -360.4409339
2017-01-10T05:00:00Z XLK 48.83597056 7 4 Filled 341.8517939
2017-01-10T05:00:00Z QQQ 121.5173626 -6 4 Filled -729.1041753
2017-01-27T05:00:00Z XLK 50.14089933 10 4 Filled 501.4089933
2017-01-27T05:00:00Z QQQ 124.9284285 -9 4 Filled -1124.355857
2017-03-21T04:00:00Z XLK 53.12743299 -9 4 Filled -478.1468969
2017-03-21T04:00:00Z QQQ 131.6614825 8 4 Filled 1053.29186
2017-03-30T04:00:00Z XLK 52.86938829 22 4 Filled 1163.126542
2017-03-30T04:00:00Z QQQ 131.5719237 -20 4 Filled -2631.438473
2017-06-22T04:00:00Z XLK 55.89936957 26 4 Filled 1453.383609
2017-06-22T04:00:00Z QQQ 140.6317219 -24 4 Filled -3375.161327
2017-07-19T04:00:00Z XLK 57.01536411 1 4 Filled 57.01536411
2017-08-31T04:00:00Z XLK 58.41035729 5 4 Filled 292.0517865
2017-08-31T04:00:00Z QQQ 144.7127759 -4 4 Filled -578.8511035
2017-09-01T04:00:00Z XLK 58.76906983 5 4 Filled 293.8453491
2017-09-01T04:00:00Z QQQ 146.0398668 -4 4 Filled -584.1594671
2017-09-16T04:00:00Z XLK 58.85 10 4 Filled 588.5
2017-09-16T04:00:00Z QQQ 145.5609167 -9 4 Filled -1310.04825
2017-09-19T04:00:00Z XLK 58.88 -25 4 Filled -1472
2017-09-19T04:00:00Z QQQ 145.81 23 4 Filled 3353.63

And I thought that it trades just once a month on the first day of the month but it does not. Is this supposed to do this? What am I missing? Thanks :)

0


Jingw, I attached a backtest for the year 2017. It looks very disappointing and has a lot different outlook from the one above. Do we need some sort of warm-up period? Can you explain about this? Thank you :)

0


Hi HanByul, yes this algorithm needs model selection and training period. For the two parameters here,

self.numdays = 1200 # set the length of formation period which determine the copula we use
self.lookbackdays = 250 # set the length of history data in trading period I use 1200 days to train the

history data to find the best fit copula from Archimedean copulas class, and use 250 days to train the chosen copula function to find the best parameter estimation. For pairs trading, when to trade and how many trades each month really depend on if the criteria is satisfied and can't control manually. But for this reason, when there is a trading opportunity for buying the pair(buy first sell second), it could happen that the last trading opportunity is still buying signal, then since you already hold the long position, you have limited money to buy new ones. That's why the quantity decreased dramatically sometimes. But the decreasing trading quantity can be modified by changing the leverage or you can set your own trading rules when there is a pairs trading opportunity.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jingw, Thank you for your explanation. Regarding the quantity of each leg, I think it's understandable. But I still don't understand why trading from year 2017 (see backtest right above.) looks so disappointing. Compared to the backtest starting 2016, two results are not giving me any level of confidence. If I deployed this algo in Jan. 2017, I would've gotten so bad result. However, if I did in Jan. 2016, it would've been totally a different story. What do you think? Is there any other way to improve regarding this aspect? How will this algo go in live trading? Thanks :) 

0

HanByul, It is reasonable. As in algorithm, we just choose copula once and use this copula in the following days. The only thing that changes is the parameter of that copula. Your backtest starting from 2016 and starting from 2017 use the different history data to choose the copula, they might choose the different copula and the algorithm will use this copula for the following backtesting. You can control the copula you use by ignoring the copula selection process to reach the same result.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jingw, Thanks for all the answers and explanation. I will dig into more about 'copula' method and see if we can implement for multi-pairs instead of picking one pair (If you've already done for multi-pairs, please let us know. I appreciate it.). Again, thank you for your great work and the opportunity to explore the 'copula' method.  :)

0

your universe is: tick_syl =  [["SPY","AGG","XME","TNA","FAS","XLF","EWC","QLD"],
                                         ["DIA","JNK","EWG","TLT","FAZ","XLU","EWA","QID"]]

as a non-programmer, I'm wondering if I can replace names here and run backtest with my names?  I've tried and gotten errors

0

If for a Gumbel copula we get a combination of (u,v) of (1,1) -> the pdf of your algorithm returns an infinity value.

0

Hi Dennis,

When (u, v) -> (1, 1), log(u) and log(v) are zero so the Gumbel pdf will be "inf". u and v are  constructed with the empirical distribution function from the log return series, there must be a 1 in u or v. Therefore, in the original algorithm I try to convert the infinity value to the finite number with

# Replace nan with zero and inf with finite numbers in lpdf list 
lpdf = np.nan_to_num(lpdf)

The finite numbers are still huge so the sum would be inf. A better solution would be converting the 1 in u or v to a number less than 1 for example 0.9999.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed