Share New Research 
Now that options data is available at QuantConnect, I was trying to create a log of historical volatility skew for a selected symbol. Here is an example chart of GLD skew based on fixed 90 - 110 moneyness strikes:
http://www.nextedgecapital.com/an-options-market-perspective-of-gold/
http://nextedgecapital.com/wp-content/uploads/2014/11/GLD-vs-90-110-Implied-Volatility-SKEW-Inverted-Scale.png
Attached is my first attempt to collect the data for a portion of the chart. The code interpolates implied volatilities of the nearest expiration contracts daily at a specific time.
This backtest runs out of memory when processing just 2 months of history ("Memory Usage Maxed Out"); also, it runs quite slowly. I suspect contract pre-filtering and ranking could be done more efficiently. How could this code be optimized for a smaller memory use and a more performant option contract selection?
Alexandre Catarino
We have performed quick sensibililty tests locally where we have changed the value for mnyPrefilt.
As expected the lower its value, the faster the algorithm runs:
mnyPrefilt | Time (min)0.15 | 10
0.10 | 5
0.05 | 2
for a 10 days simulation with the same results (comparing the "Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2").
In respect to the memory maxed out, we will be working on some improvements to the engine that will address it.
Pavel Paramonov
Thank you Alexandre for doing local benchmarks; it helps to know that the size of the contract data structure primarily determines the run time. Realistically, at least 15-20% around ATM price is needed for exploring multi-leg structures in data handler functions. Cloud backtests vary in their run time significantly, I guess they execute along with other jobs on shared cloud instances. Still, 0.8 hours per month of a single-underlyer backtest seems to be quite a bottleneck; any improvements here would help.
One issue I can not seem to graps is why a backtest with a wider range in the contract pre-filtering fails to find some contracts which were found with more strict filter settings. Specifically, with this particular algorithm, setting mnyPrefilt to 0.20 (vs. 0.11 or 0.15) results in no contracts found on 2011-08-24 through 2011-09-01. Any insights on that?
Output with mnyPrefilt = 0.15
2011-08-01 00:00:00 Launching analysis for b7a9a5fd8cba3db0036cf1eb8bba4af3 with LEAN Engine v2.4.0.0.1524 2011-08-01 00:00:00 Time, Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2 2011-08-01 10:00:00 , 158.1100, 158.1100, .0162, 0.227961464668090, 0.2117278096094234285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-02 10:00:00 , 159.3700, 159.3700, .0138, 0.2304918844828906071428571429, 0.2167294325617126785714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-03 10:00:00 , 162.6400, 162.6400, .0105, 0.2242071988434935, 0.213727273861809, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-04 10:00:00 , 163.7300, 163.7300, .0081, 0.2332001217899089642857142857, 0.2251072987164092857142857143, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-05 10:00:00 , 161.7600, 161.7600, .0324, 0.2604848937044794285714285714, 0.2280772482935974285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-08 10:00:00 , 164.9300, 164.9300, .0227, 0.2748755672162427857142857143, 0.2521538445933613928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-09 10:00:00 , 168.8000, 168.8000, -.0011, 0.3127602017132037142857142857, 0.3138537790468401428571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-10 10:00:00 , 172.5500, 172.5500, .0027, 0.33165719912357725, 0.32897865112220675, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-11 10:00:00 , 171.6400, 171.6400, .0094, 0.3326001341101495714285714286, 0.3232097632151302142857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-12 10:00:00 , 169.7200, 169.7200, .0115, 0.3030578796538018214285714286, 0.29160785417144625, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-15 10:00:00 , 169.2100, 169.2100, -.0034, 0.2593786558762513571428571429, 0.2627365396871488571428571429, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-16 10:00:00 , 173.0400, 173.0400, -.0402, 0.257292387851548, 0.2975280653005263928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-17 10:00:00 No matching back expiration 2011-08-18 10:00:00 No matching back expiration 2011-08-19 10:00:00 No matching back expiration 2011-08-22 10:00:00 No matching back expiration 2011-08-23 10:00:00 No matching back expiration 2011-08-24 10:00:00 , 175.6900, 175.6900, .0264, 0.349581513743895, 0.3231828760642492, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-08-25 10:00:00 , 168.2200, 168.2200, .0015, 0.3258917008872349714285714286, 0.3243970777968120857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-08-26 10:00:00 , 173.3500, 173.3500, -.0022, 0.3586418769450583428571428571, 0.3608008887988068857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-08-29 10:00:00 , 174.5500, 174.5500, -.0123, 0.3327861598923732, 0.3451126204593124857142857143, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-08-30 10:00:00 , 176.9400, 176.9400, .0026, 0.3196602686159474857142857143, 0.3170463362922745714285714286, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-08-31 10:00:00 , 176.8700, 176.8700, -.0068, 0.3030619807674288, 0.3099028156836198, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-09-01 10:00:00 , 177.8600, 177.8600, .0124, 0.3076854986810011428571428571, 0.2952497872527017142857142857, 09/17/2011 00:00:00, 10/22/2011 00:00:00 2011-09-01 19:59:00 Algorithm Id:(b7a9a5fd8cba3db0036cf1eb8bba4af3) completed in 2397.70 seconds at 11k data points per second. Processing total of 27,499,131 data points.Output with mnyPrefilt = 0.20
2011-08-01 00:00:00 Launching analysis for 611200c84ba632f826e3d541e618050d with LEAN Engine v2.4.0.0.1524 2011-08-01 00:00:00 Time, Underlying1T1, Underlying2T2, Skew, iv1, iv2, exp1, exp2 2011-08-01 10:00:00 , 158.1100, 158.1100, .0162, 0.227961464668090, 0.2117278096094234285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-02 10:00:00 , 159.3700, 159.3700, .0138, 0.2304918844828906071428571429, 0.2167294325617126785714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-03 10:00:00 , 162.6400, 162.6400, .0105, 0.2242071988434935, 0.213727273861809, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-04 10:00:00 , 163.7300, 163.7300, .0085, 0.2335748271596796785714285714, 0.2251072987164092857142857143, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-05 10:00:00 , 161.7600, 161.7600, .0324, 0.2604848937044794285714285714, 0.2280772482935974285714285714, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-08 10:00:00 , 164.9300, 164.9300, .0227, 0.2748755672162427857142857143, 0.2521538445933613928571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-09 10:00:00 , 168.8000, 168.8000, -.0011, 0.3127602017132037142857142857, 0.3138537790468401428571428571, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-10 10:00:00 , 172.5500, 172.5500, .0027, 0.33165719912357725, 0.32897865112220675, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-11 10:00:00 , 171.6400, 171.6400, .0094, 0.3326001341101495714285714286, 0.3232097632151302142857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-12 10:00:00 , 169.7200, 169.7200, .0115, 0.3030578796538018214285714286, 0.29160785417144625, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-15 10:00:00 , 169.2100, 169.2100, -.0034, 0.2593786558762513571428571429, 0.2627365396871488571428571429, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-16 10:00:00 , 173.0400, 173.0400, -.0045, 0.2585831315637228571428571429, 0.2630715216197309642857142857, 08/20/2011 00:00:00, 09/17/2011 00:00:00 2011-08-17 10:00:00 No matching back expiration 2011-08-18 10:00:00 No matching back expiration 2011-08-19 10:00:00 No matching back expiration 2011-08-22 10:00:00 No matching back expiration 2011-08-23 10:00:00 No matching back expiration 2011-08-24 10:00:00 No matching back contract 1 2011-08-25 10:00:00 No matching back contract 1 2011-08-26 10:00:00 No matching back contract 1 2011-08-29 10:00:00 No matching back contract 1 2011-08-30 10:00:00 No matching back contract 1 2011-08-31 10:00:00 No matching back contract 1 2011-09-01 10:00:00 No matching back contract 1 2011-09-01 19:59:00 Algorithm Id:(611200c84ba632f826e3d541e618050d) completed in 2853.35 seconds at 10k data points per second. Processing total of 27,462,719 data points.Pavel Paramonov
Backtest performance improves significantly once option contracts are more strictly filtered around the strikes of interest (please see reworked code in the attached backtest).
However, there seems to be missing data for a number of option contracts. This example uses GLD which has fairly liquid options.
2011-09-01 15:59:00 , 177.7500, 160, 196, 09/17/2011 00:00:00, 10/22/2011 00:00:00, 0.2993, 0.2692, 0.2942, 0.2848, -0.0038 2011-09-02 15:59:00 No matching back contract for mny2 2011-09-02 15:59:00 Could not find strike = 202 2011-09-16 15:59:00 , 175.9600, 158, 194, 10/22/2011 00:00:00, 10/22/2011 00:00:00, 0.2969, 0.2969, 0.2973, 0.2973, -0.0004 2011-09-19 15:59:00 No matching front expiration 2011-09-20 15:59:00 No matching front expiration 2011-09-21 15:59:00 No matching front expiration 2011-09-22 15:59:00 , 169.0600, 152, 186, 10/22/2011 00:00:00, 10/22/2011 00:00:00, 0.3243, 0.3243, 0.3150, 0.3150, 0.0093 2011-10-03 15:59:00 , 160.8500, 145, 177, 10/22/2011 00:00:00, 11/19/2011 00:00:00, 0.4539, 0.4082, 0.3198, 0.3108, 0.1184 2011-10-04 15:59:00 , 157.7600, 142, 174, 10/22/2011 00:00:00, 11/19/2011 00:00:00, 0.3984, 0.3749, 0.3439, 0.3418, 0.0446 2011-10-05 15:59:00 No matching front expiration 2011-10-06 15:59:00 No matching front expiration 2011-10-07 15:59:00 No matching front expiration 2011-10-10 15:59:00 No matching front expiration 2011-10-11 15:59:00 No matching front expiration 2011-10-12 15:59:00 No matching front expiration 2011-10-13 15:59:00 No matching front expiration 2011-10-14 15:59:00 No matching front expiration 2011-10-17 15:59:00 No matching front expiration 2011-10-18 15:59:00 No matching front expiration 2011-10-19 15:59:00 No matching front expiration 2011-10-20 15:59:00 No matching front expiration 2011-10-21 15:59:00 No matching front expiration 2011-10-24 15:59:00 No matching front expiration 2011-10-25 15:59:00 No matching front expiration 2011-10-26 15:59:00 No matching front expiration 2011-10-27 15:59:00 No matching front expiration 2011-10-28 15:59:00 No matching front expiration 2011-10-31 15:59:00 No matching front expirationAlexandre Catarino
Pavel Paramonov, those days are actually missing from our database. We have asked out data vendor about it and they sent us a new dataset that is under processing right now.
It is a lot of data. Hopefully, it will be processed by next Monday and we will be able to patch holes in existing data.
Pavel Paramonov
Suggestion : Alexandre Catarino Could you priotitize data fill for the most liquid options, the ones that CBOE calculates volatility indices for:
http://www.cboe.com/products/vix-index-volatility/volatility-on-etfs
Such as GLD, EWZ etc.
Pavel Paramonov
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!