Category: Quant Interviews

7 Tips For Fixing Your Strategy Backtesting a Q&A With Top Quants

Strategy backtesting is a mix of art and science. Quants who rely too much on science will fall victim to the infamous Curve Fitting phenomenon. While some quants who overcompensate on their artistic balance will create disillusioned theories that back their models. We created this post to compile leading quants’ perspectives on strategy backtesting covering tools, tips and how to avoid common mistakes made while strategy backtesting.

1. Have A Common Sense Idea of Your Model

“If you can’t come up with a pretty common sense explanation that you can describe to your 12-year-old niece or nephew. Then chances are, you’re simply data mining. Coming up with something that won’t exist in the future.”said Mebane Faber (@mebfaber), Portfolio Manager at Cambria Investments.

When Mebane first started in quant trading he thought he had found the “holy grail” model by analyzing its historical returns. Mebane says extrapolating historical returns to the present is a common mistake made by new quants which can be fixed if you know why your model works. “Most traders that are older have a lot of battle scars either from real money or paper trading portfolios that performed differently than the historical model showed.” says the experienced Mr. Faber.

So what is Mebane’s advice to new quants looking to perfect their backtesting strategy: “come up with a system, or multiple systems (which I think is more important) that fits your personality but is also robust over time“. Wise words from a King of quantitative trading.

2. Use Blind Data to Improve Your Strategy

Optimizing a strategy is for some a process that is ongoing and for some a process that produces such low results that it is abandoned. Deepak Shenoy(@deepakshenoy), Co-Founder of Capital Mind a big data analysis company, says it is a combination of both groups “I don’t like curve fitting so I tend to avoid over-optimization“. Instead he adds unrelated factors “like volume, open interest or options price sensitivity (vega) – or simply by using the knowledge of near term events to augment a system“.

Additionally, an important step for Deepak after he’s optimized, is to test on a blind data set for validation. “I test an optimized strategy on either different sets of stock data or different timeframe data… or both. To see how the system does in other timeframes.” This is before he moves the system to a lengthy paper trading schedule to further validate the model. Deepak’s perspective seems to be a focus on perfection, validation from multiple sources and comprehensive market regime testing. A worthy lesson for quants, be a perfectionist!

3. Decide Your Most Important Metrics Before

When you get to the performance evaluation segment of your strategy backtesting process, historical data is plentiful. “While it is pretty standard to look at popular performance statistics like the Calmar Ratio, Sharpe Ratio, CAGR, or MaxDD, they only reveal a small picture of how the model performed” says Michael Guan, an Associate with Macquarie Group. Michael says he focuses on additional measures like Positive Rolling 12 month periods, average drawdown, Annual Returns, average profit per trade which all provide a “more multi-dimensional view through time“.

Michael is a true believer in a multi-dimensional backtesting process. He explained his viewpoint eloquently: “backtesting is an integral part of trading model development. Making as few assumptions as possible and testing on a wide variety of assets ensures statistical robustness“. Many quants run a backtest with no preconceived notion of the data they want and this creates an issue in determining what data is valuable. Follow Michael’s tip and make a list of the variables that matter to you. And data has more value to you, is the Sharpe Ratio worth more to you than the P:L ratio? You should know this before you start backtesting.

4. Make Sure You Are Looking At The Right Data

Continue reading

Designing Sentiment Trading Strategies with Stefan Nann

Stefan Nann is the Co-Founder and CEO of StockPulse, a provider of social media sentiment data for individual and index securities. Stefan studied Business Administration and Information Systems before performing graduate studies with a focus on financial markets, analysis of unstructured text and online communities. While a visiting scholar at the Massachusetts Institute of Technology (MIT) Center for Collective Intelligence, Jonas Krauss (Co-Founder) and Stefan Nann build the semantic compilation algorithm first used for Oscar predictions that is the backbone to StockPulse’s sentiment scores today. Stefan is analytical, eloquent and truly visionary individual and it was a pleasure speaking with him about building his algorithm, valuing input data by source value and signal management for modern financial markets. Enjoy our interview with Stefan Nann:

Simon Burns: Stefan Nann, thank you for joining us. Could you start by taking us through the your story and how you got into compiling sentiment scores for stocks?

Stefan Nann: Basically, it started when I was with my colleague Jonas Krauss. We met in University and started working together for a study we were doing predicting Oscar winners with algorithms. We collected vast amounts of communication data on people’s opinion of movies and actors. This information was found online from sources like the the internet movie database (IMDB) forum where we collected a year’s worth of data.

Using our algorithm, we got 9 out of 10 of our predictions right in 2007. This got us a fair amount of press and validated our work. So we wrote an academic paper which was also well received. Then we thought, “Okay if this works for Oscar predictions, then it probably also work for stocks and other financial instruments”. This was around the time Twitter/StockTwits invented the dollar sign hashtag which organized financial market communications about stocks and made it readable by our algorithm. Continue reading

Talking Regime Change Quantitative Strategies with Blake Woodard of RLF Capital Management

Blake Woodard had a chat with our Growth Hacker Simon Burns on his start in algorithmic trading with Excel following an injury that left him motionless for days, his crowd psychology based market strategy and opinion on HFT. Blake is the Managing Director and Portfolio Manager at RLF Capital Management. Blake will be teaching at the Center for Entrepreneurship and Technology at UC Berkeley in the Fall. Blake is an insightful, candid and well spoken commentator and we thoroughly enjoyed having Blake Woodard at our San Francisco office. Here’s our interview with Blake Woodard:

Simon Burns: Well thank you for joining us Blake. Blake Woodard is the portfolio manage for R.L.F. Capital Management. Blake, tell us the story how long you’ve been running the fund, how did you get involved?

Blake Woodard: I’ve been running the fund for two years. I’ve been trading for a long time before that, basically throughout college then after I graduated, I went traveling sort of a series of post graduation trips. During that time I was trading and also writing down theory. So I started working on quantifying my trading during those trips since a lot of my strategies were math based, based on technical analysis. I put more work into these quant trading models after I got back again and it’s something where I had a little bit of success and it’s really interesting how having a little bit of success can really make you more willing to put more work into it. As it happens two weeks after that first breakthrough, I got assaulted and I actually broke my eye socket and my cheekbone so I got reconstructive surgery . After that I was recovering on my dad’s house, I couldn’t eat. I couldn’t talk. I couldn’t move around that much. For a period of time I was really just glued to my computer working on models and it just started to work. I was essentially working on a strategy that would switch and adapt to regime changes that I was already trading manually.

Continue reading

Social Media Analytics in Finance with Oli Wilkinson of Knowsis

Oli Wilkinson had an interview with our Growth Hacker Simon Burns to discuss the inception of his financial technology startup Knowsis following the 2008 crash, the growing influence of social media in financial markets and the process of building a machine learning tool to dissect social media “noise”. Oli co-founded and is the CEO of London based financial instrument based social media called Knowsis. Knowsis has built a service directed at institutional clients to help them navigate the noisy, inflammatory and unstructured data flows from social media. Oli’s commentary was compelling, candid and comprehensive. Here’s our interview with Oli Wilkinson:

Simon Burns: Oli, you’re an inspiring innovator in the financial technology space. Knowsis has built itself into a pioneer in social media analytics for financial market securities. Can you take through us through how you got started in the space and maybe evolve that into what you’re doing now with Knowsis.

Oli Wilkinson: Yeah, absolutely. Knowsis started 3 or so years ago. At the time, I was an investment bank trader on a trading floor in London. It was around the time when news stories were breaking over Twitter before they were hitting the traditional news wires. Continue reading

Stellar Interview with Spencer Connaughton: 21 Year Old Quant Fund Manager at Archivolt Partners

Spencer Connaughton had a chat with our Growth Hacker Simon Burns on the learning curve in becoming an algorithmic/quant trader using his school’s Bloomberg terminals (and crashing them), his firm’s use of the Residual Income Valuation model in algorithmic trading and the potential of markets with 20-somethings running them. Spencer started and runs a quant, grey box algorithmic trading fund called Archivolt Partners alongside Andrew DeTrempe, a Finance student. Archivolt Partners has built its strategy through close collaboration with Residual Income Valuation model pioneer and Southern Methodist University professor Dr. Greg Sommers.  We had a great time talking to Spencer. His insightful, captivating and at times hilarious stories and analysis on the state of algorithmic trading, starting an ATS fund and democratizing model creation was enjoyable. Here’s our interview with Spencer Connaughton:

Simon Burns: Thank you for joining us today Spencer Connaughton. Spencer you are notable and have acquired some fame as a young quantitative fund manager. Your fund is called Archivolt Partners and was started, as many funds have started, from academic grounds. Can you take us through the history of the fund and how it came to be?

Spencer Connaughton: Sure. My former college residential advisor, Andrew DeTrempe, approached me one day and said: “Spencer, my professor Dr. Greg Sommers just told me about this highly accurate equity valuation methodology that he has studied for decades. But it takes an army of grad students to gather all the data for it and crunch all the numbers. It sounds like the kind of problem that you could write a computer program to solve. Want to give it a shot?”  So we sat down with his professor and talked through the issues involved. I was thrown off the deep-end into the world of earnings-based valuation methodologies. We decided to set up this program on the Bloomberg platform to look at every stock in the universe, and run our valuation formula on each of them. Once we got into backtesting, we realized there was something there. We spent about 6 months developing and studying the algorithm, then we formed a company and raised money from our friends and family.  So that’s how Archivolt Partners started. Continue reading

History of Non-Market Data Correlations

Over the course of the history of the stock market, quantitative observers have built up a comprehensive database of non-market data correlations. From solar flares to hurricane cycles, biota growth and New York City temperature. Figures like Consumer Sentiment data and other broadly distributed survey data have been used by market participants when deciding market purchases as well as by economists to predict metrics. Investor interest has grown massively around Twitter Sentiment data, crowd sourced estimates data like Estimize, which integrates its data with QuantConnect, and other unconventional, seemingly unrelated market data. All of which are being used to build stock trading algorithms based on correlations. We decided to write a post with an overview of different types of non-market data used to predict security prices.

Calendar/Time Data’s Influence on Trading

For hundreds of years speculators have placed seasons, months, days and hours as either to blame or be praises for their seemingly superstitious distribution of market returns. A well known and often repeated line is the classic Wall Street saying, “Sell in May and Go Away”. Surprisingly, the strategy would have performed mightily since 1950 in the US as the chart below indicates.1 Other models have based trading on avoiding the recurring October declines or  “October Surprise”, as well as tax loss selling based models which happen at the end of the fiscal year (last week of December). 1 Continue reading

Interview with Mebane Faber of Cambria Investment Management

Mebane Faber had a chat with our Growth Hacker Simon Burns on the learning curve in becoming an algorithmic/quant trader, correlating non-market data and the move towards democratization of algorithmic model creation among. Mebane runs a long form qualitative and quantitative analysis blog at Mebane Faber, is the author of Shareholder Yield: A Better Approach to Dividend Investing and is the portfolio manager at Cambria Investment Management. He is a frequent speaker and writer on investment strategies and has been featured in Barron’s, The New York Times, and The New Yorker. We enjoyed the opportunity to listen to Mebane’s insightul, empirically tested and eloquently stated views and analysis on the state of algorithmic trading, HFT and markets. Enjoy our interview with Mebane Faber:

Simon Burns: Thank you so much for joining us Mr. Faber. One of the things that caught our attention recently from your blog is when your post titled, “Sell in May or November Timber.” And your line here, which I think is really pertinent in the world of big of data is “if you cannot explain why an inefficiency exist or the fundamentals behind a technical strategy, then you’re likely just data mining.” Now the majority of trading on markets is algorithmic, could you please elaborate  for us how much of the trading is really in true correlations and how much is data mining?
Mebane Faber: It goes back to any approach, not just technical but also fundamental, behavioral, or the example we used, calendar based. And the challenge from those investors is being able to come up with a fundamental reason why something works and why it should work in the format of capitalism or in markets. One of the difficulties about being human is that we so often learn by telling stories. In many cases, stories make a lot of sense and they sound great in the investing world, but don’t necessarily fit either history or common sense. So, we try to spend as much time with the historical data possible but also take a step back and say, does this make sense from either statistical foundation or fundamental backdrop and in many cases likely if you can’t come up with a pretty common sense explanation, you can probably describe to your 12-year-old niece or nephew then good chances are, you’re simply data mining, coming up with something that won’t exist in the future. Continue reading

Interview with Tadas Viskanta of Abnormal Returns

Tadas Viskanta sat down for an interview with our Growth Hacker Simon Burns on what he sees in the future for financial blogging, the effect of Twitter data on markets and the move towards democratization of algorithmic model creation among other things. Tadas is a prolific blogger at Abnormal Returns, author of Abnormal Returns: Winning Strategies from the Frontlines of the Investment Blogosphere and long time market commentator. His views, analysis and projections for algorithmic trading, HFT and markets in general are insightful and blended from experience, extensive market reading and his own research. Enjoy our interview with Tadas Viskanta:

Simon Burns: Thank you for joining us. So before we get started, I’ll give you a little introduction on QuantConnect. We have a software solution that is all C# and allows your average investor with basic coding knowledge to build an algorithmic trading model. Very much unique and in a sentence, we like to say we are democratizing algorithmic trading or algorithmic model creation.

I’ve been reading a few a few of your pieces recently, broadly on the financial blogosphere and how its “death”, maturity or some form of development has occured. I’d be interested in hearing your thoughts, or maybe contrasting this development with the rise of twitter as a market force and for crowdsourcing investment advice.

Tadas Viskanta Interview with Quant Connect

Tadas Viskanta had a conversation with QC’s Simon Burns on the future of HFT, the effect of Twitter on markets and what his optimism on the democratization of markets.

Tadas Viskanta:  I think that Twitter and Stocktwits are definitely taking share from the blogosphere. I don’t think there is any doubt about that. But you know, I do think they are big market segments and some of what used to happen in the blogosphere has shifted over to Twitter. For widgets they are the appropriate form. Also short form, quick messaging, one to many sort of messaging..has all been taken over from the blogosphere. But I think anything that gets beyond 140 characters, the blog is still the proper format. If you think about anything quantitative you need to go to the longer form, there is really just not the space in that short form communication to make that sort of argument. Certainly just from my blogging perspective, I’ve been on Twitter and Stocktwits for a long time and I still see people join that weren’t there before joining. I think it’s all part of growing communication, it’s not better or worse just strengthening the entire pie.

Simon: In the second stage of development in financial blogging, you talked about quantitative analysis. Do you think that long form quantitative analysis is ideal for blogs and Twitter is taking the place for other forms of investment content, is there a dichotomy there?

Tadas: Sure, I mean anything quantitive with linear equations or regressions or anything with that sort of frame, you have to go to that long form. I don’t know if blogs will take more share or move more in that direction. I think there is certainly a vibrant sub-culture of quantitative blogs. Like there is anything else there is kind of a wide variety of interests, skill levels…different markets being analyzed on different time frames. So I don’t think I say anything one thing, but a variety of things are spreading out. Continue reading

Intersection Between Quant and Technology

Deutsche Bank Mention

Deutsche Bank had a few nice words to say about QuantConnect in their recent news letter:

“Recently we came across an interesting company called QuantConnect that perfectly embodies the intersection between quant and technology. QuantConnect is a startup that is trying to “democratize” quant investing. Most of us, who have spent too many late nights trying to fix a recalcitrant server or debug a crashed SQL script, know that the biggest barrier to entry in quant is the IT infrastructure required to maintain a robust back-testing and production environment. QuantConnect overcomes this by offering a cloud-based platform that is pre-loaded with the data and back-testing capability that is needed to test strategies. However, the really cool feature is that once you are happy with your back-tested strategy, you can actually turn it on live, and even connect it directly to your brokerage account to trade it in real time.” - Deutsche Bank Quantitative Strategy Team

Thanks guys!

 Scheduled Upgrade

We’re rolling out a scheduled upgrade tonight, hopefully there will be no disruption and services will resume as normal 9am 4th April. If you have any questions please feel free to reach out to Jared at jared@quantconnect.com. There will be breaking API changes! These will be detailed in a following post.

 

© 2017 QuantConnect Blog

Democratizing algorithmic investments Up ↑