Discussion Forum - QuantConnect.com

Pavel Paramonov INVESTOR

| Reward Discussion

Now that options data is available at QuantConnect, I was trying to create a log of historical volatility skew for a selected symbol. Here is an example chart of GLD skew based on fixed 90 - 110 moneyness strikes:

http://www.nextedgecapital.com/an-options-market-perspective-of-gold/
http://nextedgecapital.com/wp-content/uploads/2014/11/GLD-vs-90-110-Implied-Volatility-SKEW-Inverted-Scale.png

Attached is my first attempt to collect the data for a portion of the chart. The code interpolates implied volatilities of the nearest expiration contracts daily at a specific time.

This backtest runs out of memory when processing just 2 months of history ("Memory Usage Maxed Out"); also, it runs quite slowly. I suspect contract pre-filtering and ranking could be done more efficiently. How could this code be optimized for a smaller memory use and a more performant option contract selection?

Update Backtest

Pavel Paramonov

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Alexandre Catarino

100.6k Pro ,

We have performed quick sensibililty tests locally where we have changed the value for mnyPrefilt.
As expected the lower its value, the faster the algorithm runs:

mnyPrefilt | Time (min) 0.15 | 10 0.10 | 5 0.05 | 2

for a 10 days simulation with the same results (comparing the "Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2").

In respect to the memory maxed out, we will be working on some improvements to the engine that will address it.

Pavel Paramonov

1.6k Pro ,

Thank you Alexandre for doing local benchmarks; it helps to know that the size of the contract data structure primarily determines the run time. Realistically, at least 15-20% around ATM price is needed for exploring multi-leg structures in data handler functions. Cloud backtests vary in their run time significantly, I guess they execute along with other jobs on shared cloud instances. Still, 0.8 hours per month of a single-underlyer backtest seems to be quite a bottleneck; any improvements here would help.

One issue I can not seem to graps is why a backtest with a wider range in the contract pre-filtering fails to find some contracts which were found with more strict filter settings. Specifically, with this particular algorithm, setting mnyPrefilt to 0.20 (vs. 0.11 or 0.15) results in no contracts found on 2011-08-24 through 2011-09-01. Any insights on that?

Output with mnyPrefilt = 0.15

2011-08-01 00:00:00 Launching analysis for b7a9a5fd8cba3db0036cf1eb8bba4af3 with LEAN Engine v2.4.0.0.1524

2011-08-01 00:00:00 Time, Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2

2011-08-01 10:00:00 , 158.1100, 158.1100, .0162, 0.227961464668090, 0.2117278096094234285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-02 10:00:00 , 159.3700, 159.3700, .0138, 0.2304918844828906071428571429, 0.2167294325617126785714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-03 10:00:00 , 162.6400, 162.6400, .0105, 0.2242071988434935, 0.213727273861809, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-04 10:00:00 , 163.7300, 163.7300, .0081, 0.2332001217899089642857142857, 0.2251072987164092857142857143, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-05 10:00:00 , 161.7600, 161.7600, .0324, 0.2604848937044794285714285714, 0.2280772482935974285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-08 10:00:00 , 164.9300, 164.9300, .0227, 0.2748755672162427857142857143, 0.2521538445933613928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-09 10:00:00 , 168.8000, 168.8000, -.0011, 0.3127602017132037142857142857, 0.3138537790468401428571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-10 10:00:00 , 172.5500, 172.5500, .0027, 0.33165719912357725, 0.32897865112220675, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-11 10:00:00 , 171.6400, 171.6400, .0094, 0.3326001341101495714285714286, 0.3232097632151302142857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-12 10:00:00 , 169.7200, 169.7200, .0115, 0.3030578796538018214285714286, 0.29160785417144625, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-15 10:00:00 , 169.2100, 169.2100, -.0034, 0.2593786558762513571428571429, 0.2627365396871488571428571429, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-16 10:00:00 , 173.0400, 173.0400, -.0402, 0.257292387851548, 0.2975280653005263928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-17 10:00:00 No matching back expiration

2011-08-18 10:00:00 No matching back expiration

2011-08-19 10:00:00 No matching back expiration

2011-08-22 10:00:00 No matching back expiration

2011-08-23 10:00:00 No matching back expiration

2011-08-24 10:00:00 , 175.6900, 175.6900, .0264, 0.349581513743895, 0.3231828760642492, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-08-25 10:00:00 , 168.2200, 168.2200, .0015, 0.3258917008872349714285714286, 0.3243970777968120857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-08-26 10:00:00 , 173.3500, 173.3500, -.0022, 0.3586418769450583428571428571, 0.3608008887988068857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-08-29 10:00:00 , 174.5500, 174.5500, -.0123, 0.3327861598923732, 0.3451126204593124857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-08-30 10:00:00 , 176.9400, 176.9400, .0026, 0.3196602686159474857142857143, 0.3170463362922745714285714286, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-08-31 10:00:00 , 176.8700, 176.8700, -.0068, 0.3030619807674288, 0.3099028156836198, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-09-01 10:00:00 , 177.8600, 177.8600, .0124, 0.3076854986810011428571428571, 0.2952497872527017142857142857, 09/17/2011 00:00:00, 10/22/2011 00:00:00

2011-09-01 19:59:00 Algorithm Id:(b7a9a5fd8cba3db0036cf1eb8bba4af3) completed in 2397.70 seconds at 11k data points per second. Processing total of 27,499,131 data points.

Output with mnyPrefilt = 0.20

2011-08-01 00:00:00 Launching analysis for 611200c84ba632f826e3d541e618050d with LEAN Engine v2.4.0.0.1524

2011-08-01 00:00:00 Time, Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2

2011-08-01 10:00:00 , 158.1100, 158.1100, .0162, 0.227961464668090, 0.2117278096094234285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-02 10:00:00 , 159.3700, 159.3700, .0138, 0.2304918844828906071428571429, 0.2167294325617126785714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-03 10:00:00 , 162.6400, 162.6400, .0105, 0.2242071988434935, 0.213727273861809, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-04 10:00:00 , 163.7300, 163.7300, .0085, 0.2335748271596796785714285714, 0.2251072987164092857142857143, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-05 10:00:00 , 161.7600, 161.7600, .0324, 0.2604848937044794285714285714, 0.2280772482935974285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-08 10:00:00 , 164.9300, 164.9300, .0227, 0.2748755672162427857142857143, 0.2521538445933613928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-09 10:00:00 , 168.8000, 168.8000, -.0011, 0.3127602017132037142857142857, 0.3138537790468401428571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-10 10:00:00 , 172.5500, 172.5500, .0027, 0.33165719912357725, 0.32897865112220675, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-11 10:00:00 , 171.6400, 171.6400, .0094, 0.3326001341101495714285714286, 0.3232097632151302142857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-12 10:00:00 , 169.7200, 169.7200, .0115, 0.3030578796538018214285714286, 0.29160785417144625, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-15 10:00:00 , 169.2100, 169.2100, -.0034, 0.2593786558762513571428571429, 0.2627365396871488571428571429, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-16 10:00:00 , 173.0400, 173.0400, -.0045, 0.2585831315637228571428571429, 0.2630715216197309642857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00

2011-08-17 10:00:00 No matching back expiration

2011-08-18 10:00:00 No matching back expiration

2011-08-19 10:00:00 No matching back expiration

2011-08-22 10:00:00 No matching back expiration

2011-08-23 10:00:00 No matching back expiration

2011-08-24 10:00:00 No matching back contract 1

2011-08-25 10:00:00 No matching back contract 1

2011-08-26 10:00:00 No matching back contract 1

2011-08-29 10:00:00 No matching back contract 1

2011-08-30 10:00:00 No matching back contract 1

2011-08-31 10:00:00 No matching back contract 1

2011-09-01 10:00:00 No matching back contract 1

2011-09-01 19:59:00 Algorithm Id:(611200c84ba632f826e3d541e618050d) completed in 2853.35 seconds at 10k data points per second. Processing total of 27,462,719 data points.

Pavel Paramonov

1.6k Pro ,

Backtest performance improves significantly once option contracts are more strictly filtered around the strikes of interest (please see reworked code in the attached backtest).

However, there seems to be missing data for a number of option contracts. This example uses GLD which has fairly liquid options.

2011-09-01 15:59:00 , 177.7500, 160, 196, 09/17/2011 00:00:00, 10/22/2011 00:00:00, 0.2993, 0.2692, 0.2942, 0.2848, -0.0038

2011-09-02 15:59:00 No matching back contract for mny2

2011-09-02 15:59:00 Could not find strike = 202



2011-09-16 15:59:00 , 175.9600, 158, 194, 10/22/2011 00:00:00, 10/22/2011 00:00:00, 0.2969, 0.2969, 0.2973, 0.2973, -0.0004

2011-09-19 15:59:00 No matching front expiration

2011-09-20 15:59:00 No matching front expiration

2011-09-21 15:59:00 No matching front expiration

2011-09-22 15:59:00 , 169.0600, 152, 186, 10/22/2011 00:00:00, 10/22/2011 00:00:00, 0.3243, 0.3243, 0.3150, 0.3150, 0.0093



2011-10-03 15:59:00 , 160.8500, 145, 177, 10/22/2011 00:00:00, 11/19/2011 00:00:00, 0.4539, 0.4082, 0.3198, 0.3108, 0.1184

2011-10-04 15:59:00 , 157.7600, 142, 174, 10/22/2011 00:00:00, 11/19/2011 00:00:00, 0.3984, 0.3749, 0.3439, 0.3418, 0.0446

2011-10-05 15:59:00 No matching front expiration

2011-10-06 15:59:00 No matching front expiration

2011-10-07 15:59:00 No matching front expiration

2011-10-10 15:59:00 No matching front expiration

2011-10-11 15:59:00 No matching front expiration

2011-10-12 15:59:00 No matching front expiration

2011-10-13 15:59:00 No matching front expiration

2011-10-14 15:59:00 No matching front expiration

2011-10-17 15:59:00 No matching front expiration

2011-10-18 15:59:00 No matching front expiration

2011-10-19 15:59:00 No matching front expiration

2011-10-20 15:59:00 No matching front expiration

2011-10-21 15:59:00 No matching front expiration

2011-10-24 15:59:00 No matching front expiration

2011-10-25 15:59:00 No matching front expiration

2011-10-26 15:59:00 No matching front expiration

2011-10-27 15:59:00 No matching front expiration

2011-10-28 15:59:00 No matching front expiration

2011-10-31 15:59:00 No matching front expiration

Alexandre Catarino

100.6k Pro ,

Pavel Paramonov, those days are actually missing from our database. We have asked out data vendor about it and they sent us a new dataset that is under processing right now.
It is a lot of data. Hopefully, it will be processed by next Monday and we will be able to patch holes in existing data.

Pavel Paramonov

1.6k Pro ,

Suggestion : Alexandre Catarino Could you priotitize data fill for the most liquid options, the ones that CBOE calculates volatility indices for:

http://www.cboe.com/products/vix-index-volatility/volatility-on-etfs

Such as GLD, EWZ etc.

Pavel Paramonov INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

241400 Quants.

Online Now

Stay Connected

Reproducing Volatility Skew: Option History Log

Started By:

Join QuantConnect for Free Sign Up

Platform

SIGN IN

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

241400 Quants.

Online Now

Stay Connected

Reproducing Volatility Skew: Option History Log

Started By:

Join QuantConnect for Free Sign Up