trophy

Long Only, ETF Alpha Streams Competition with $27,500 Prize Pool

Back

Alpha Stream Scoring Criteria

Hello All! 

We're looking for feedback on a proposed new scoring function for algorithms. This "fitness function" would allow algorithms to be compared so we can provide a cut off for acceptance to Alpha Streams -- we've laid out the research in the notebook attached and would love your feedback!

One note other than what's laid out below; the reason Sortino and RoMaD were selected for the research is that Sortino covers the average volatility of the strategy, while RoMaD covers the absolute single worst case drawdown+return. They're both risk-adjusted but using the combination we can filter martingale-like strategies which will have bad single day drawdowns while having a great Sharpe Ratio. 

If you have suggestions please clone the notebook and attach your own suggestion! 

Thanks,

Jared

Update Backtest





The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Here are other thoughts to gauge the value of trading and screen out "Buy APPL in 1998" strategies. This will take a few posts, my apologies in advance.

The two primary ways that can make active trading more profitable than buy-and-hold are:

  1. Choice of assets

  2. Trading pattern (buy and sell decisions)

Choice of assets

Here you would want to compare the assets you selected to an appropriate benchmark. The traditional CAPM model could provide an alpha that shows the value of the chosen assets in excess of the benchmark and a risk free rate. The difficulty with this is that you would need to automate the selection of the benchmark, which is somewhat subjective and therefore difficult to do.

Since we are mainly interested in backtesting and have the benefit of 20/20 hindsight; it's very easy to select the best assets in the rear view mirror. The choice of assets is therefore not as valuable as a backtest screen, and probably doesn't follow the intent of the volume factor anyways. For now, let's put this one aside and focus on of the trading pattern.

Trading Pattern

A simple approach to look at the value of trading is to compare desired metrics between an active trading strategy and a buy-and-hold strategy with the same assets held at the average allocation from the trading model.

A simple example: we trade SPY using the traditional 200 day moving average, if SPY is above its 200MA, we buy; if it's below; we sell and buy IEF (10 year treasuries). The following backtest shows results from 1/1/2009 to 12/31/2018. From the overview, and with rf = 0; the RoMaD = 10.757%/17.3% = 62%. From the results report, the average allocation is 79% SPY and 21% IEF. I can't find the Sortino anywhere on the output, but let's assume the negative deviation is 9%; and the Sortino ratio is therefore 10.757/9 = 1.2.

0


Now we take the average allocation from the active trading strategy (79% SPY and 21% IEF) and see how things look with a buy-and-hold strategy. We get a RoMaD of 11.454%/22.1% = 52%; and an estimated Sortino of 11.454/10.5 = 1.09.

0


We now need to determine the value of active trading. We'll ignore the tax impact of turning over the portfolio since our target customers are hedge funds and not individual investors.

Active Strategy:

  • Annual Return: 10.76%
  • Negative Std Dev: 9%
  • Max Draw Down: 17.3%
Passive Strategy:
  • Annual Return: 11.45%
  • Negative Std Dev: 10.5%
  • Max Draw Down: 22.1%

The idea is to understand what risk-adjusted value is produced by active trading. The Modigliani risk-adjusted performance looks at the expected return of oe investment if it is scaled to the same risk as another investment. For example, if we look at the expected return of the active trading for the same negative standard deviation of the passive approach, we get 10.76% * 10.5% / 9% = 12.5%. In other words, if we took the same level of risk as the passive approach, we would get an expected return of 12.5% on the active approach, or a 1.05% higher return.

This represents the change in the expected return generated by the trading strategy for the same level of risk as the buy-and-hold strategy. We can do the same thing for the RoMaD; the only difference is the risk metric (max DD instead of neg SD). In the example above, we would get an expected return of 10.76% * 22.1%/17.3% = 13.74%, or a 2.29% higher expected return from the active trading strategy for the same maximum draw down.

The fitness score could conceivably be equal to the average of these two changes in expectation, in the example this would be (1.05%+2.29%)/2 = 1.67%. If this is greater than a threshold (1%?) then the algorithm makes the alpha stream, otherwise it is rejected.

Pros:

  • Measures the risk-adjusted value of the trading activity, and not the underlying assets

    • Think of this as a “trading alpha” – the return that is generated by trading

    • The growth of the underlying assets is independent from the trading alpha, so buy AAPL in 1998 would have a trading alpha of 0

  • Compares risks at the average level (Sortino) and the extreme (RoMaD), like the original metric

  • Works for longer (say monthly) or shorter term (HFT) trading strategies

    • May favor more frequent trading because there are more opportunities for the trading alpha to develop if there are more trades

Cons:

  • Requires post-run calculations (buy-and-hold backtest)

  • More difficult to test systematically and therefore to know where a good threshold might be

    • However, since this measures an alpha (annual return), a threshold based on judgment from QC, such as 1%, is justifiable

  • Doesn't look at absolute returns but rather the additional returns generated by trading

0

Hey Tristan,

Couple of thoughts regarding your proposed approach:

  • Unless I'm mistaken, this is just comparing the Sortino of alpa with Sortino of benchmark (where benchmark is average holding weight of all assets), and saying the Alpha's has to be at least some ratio of the benchmark's (>1.01)?
  • I see this working with a very specific set of strategies - namely long-only equity strategies. What about long/short strategies or even market-neutral ones, where your benchmark might very well end up having negative returns or zero weights, meaning your score would be wildly inflated compared to a traditional long-only score? What about other asset classes?
0

Douglas,

Good points.  Here are some thoughts:

  • The ratio of Sortinos is a bit different because the result would be dimensionless.  From the prior example, the Sortinos are 1.2 for the active strategy and 1.09 for the passive.  If we compared the two we would get 1.2/1.09 = 1.1.  I prefer calculating a trading alpha instead because it gives you a percent return, something that is easy to interpret
  • I'm not sure why this couldn't be used with long-short strategies or alternative assets.  For a long-short, the benchmark would have negative allocation for the short assets, i.e. you go short and stay short for the duration of the backtest.  Also, I don't know how this would get wildly inflated.  For this to go wild, the risk measure (e.g. max draw down) of the passive strategy would have to be much larger than that of the active strategy.  The formula is simply:

trading alpha = ra * sp/sa - rp

where:

  • ra = annual return of active strategy
  • rp = annual return of passive strategy
  • sa = risk measure, e.g. negative standard deviation or max draw down, of active strategy
  • sp = risk measure of passive strategy 
0

Hi Tristan - I see now that my approximation to ratios of Sortinos isn't quite what you meant, my apologies. I can't quite see a case where the measure would get wildly inflated.

Again, however, I get the impression you're thinking in terms of equities, where indeed a long/short portfolio would typically long some companies and short others, thus ending up with a meaningful measure.

Let's say instead I'm trading a basket of G10 currencies and my active strategy is such that the average weights over the period is 0% across all currencies. rp = 0, sp = 0 - giving a measure of 0 even though the strategy might be profitable.

My point is that the concept of a benchmark is complicated when you start to look outside of equities, where assets often don't have a positive drift and instead act in a more mean-reverting fashion. That's before we even start to talk about options, which brings a wholly different set of problems.

0

Just throwing this into the discussion: our aim is to remove our human bias so ideally, the fitness function should be automated. If we need to manually construct representative passive portfolios it will induce delays and costs in the evaluation process. If "the community" on a whole has factors they believe should be considered please also sanity check it can be automated!

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jared,

The intent was to automate the benchmark, which could've been done relatively easily, so that an alpha that measures the value of the trading activity could be determined.  However, as Douglas points out (thanks Douglas), this method is not universally applicable, so it's value may be limited.

0

Jared,

Would you consider lowering the threshold on the trade factor like you mentioned before?  Maybe requiring full portfolio turnover every month to get a score of 100%?  That would mean average turnover of 10% per trading day under the current metric.

Thanks.

0

I have a suggestion. For leading us to improve Alpha Ranking, the system should present base performance correlating with Alpha Ranking in Overview page in backtest. Such as backtest year, daily portfolio turnover, annual return, max drawdown, annual std of negative return. Now  backtest year, daily portfolio turnover, annual std of negative return are absent.

0

Hi Li,

Thanks for your suggestion, we will take it into consideration.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed