Back

Probabilistic Sharpe Ratio

Hello All! 

We're looking for feedback on a proposed new Alpha selection filter. The Probabilistic Sharpe Ratio would give us another way to measure the results of backtests submitted and provide funds looking to license Alphas with more information about algorithm performance beyond our current metrics. If you have suggestions please clone the notebook and attach your own suggestion! 

Thanks,

Jack

Update Backtest





The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


This would lead to higher performing Alpha Streams.  I would add it to the arsenal.

0

Is this being included in the backtest metrics?

1

We haven't included it yet into the backtest metrics, but it's something that we're considering as well.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


I would like to see this in the backtest metrics. This would be a quick "sanity test" when backtesting.

 

0

Glad to see that Quantconnect seems to be agile and willing to update methods for their algo selection.  I also find it comforting that Lean is opensource, so I can check the implementation and their test cases. :)  

For those that want to do some quick and dirty backtest or optimization/investigation in research, please do keep in mind the inevitable computation differences between LEAN and simple PSR computation in notebook.  I think this is one of those little but important things most engineers/data scientists have to deal with when working among different environments.  Anyways... per Alexandra's response, "rolling PSR" is computed by LEAN (I can't seem to figure out where the rolling part is though in github), while Jack's PSR is a single value derived from one stream of returns.  Below shows you the difference for Sharpe Ratio (SR) and PSR for a few tickers if you buy-and-hold in the same timeframe between notebook (using annualized SR) and Lean.  I would expect PSR for SPY to be a bit closer to 50% from LEAN. Avid notebook users... beware!
 

SPY
notebook SR: 0.836, PSR: 50.000%
Lean SR: 0.900, PSR: 41.185%
----
SHY
notebook SR: 1.237, PSR: 68.901%
Lean SR: 1.224, PSR: 69.799%
----
QQQ
notebook SR: 0.910, PSR: 52.476%
Lean SR: 0.990, PSR: 49.063%
----
AMZN
notebook SR: 1.218, PSR: 83.669%
Lean SR: 1.339, PSR: 78.661%
----
GE
notebook SR: -0.458, PSR: 10.897%
Lean SR: -0.306, PSR: 0.159%
----
1


I would be more than glad if someone can also share how to calculate rolling PSR.

0

I also have a question, why you haven't utilized Deflated Sharpe Ratio? That seems to be an extension over PSR.

0

DSR was not deterministic, and required analysis of many backtests. PSR was chosen because it could be self contained with a single analysis.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hello Jared, thanks for letting me know.!I have a small objection about PSR. Since that it uses a long-only SPY benchmark, it does not make perfect sense for long-short portfolios or a long-only ETF portfolio that includes some inverse ETF(s), such as the current universe. I sometimes obtain better results in many of the metrics but the PSR is lower. To sum up, it is not a very good metric in terms of taking the market-neutrality into account (except for a long-only portfolio that does not include any Inverse ETFs - which is not valid for the current competition universe).

0

We chose not to use SPY as the benchmark but a fixed Sharpe-ratio of 1.0 to make the measurement cross-asset / cross-strategy type; so the PSR readings in LEAN's case are the probability the real algorithm returns are greater than 1.0 Sharpe ratio. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Just to let you know, I encountered another implementation of PSR in some other source codes that I have checked. The first one below is that other implementation and the second one below is the implementation in this thread. I did not yet compare the output of those but they seemed to be different to me. I would welcome an advise on this.

    std = np.sqrt(1 - (sps.skew(ret)*sharpe) + sharpe**2 * (sps.kurtosis(ret)-1)/4)
    return sps.norm.cdf(sharpe * np.sqrt(len(ret)-1) / std)

    std = 1 - (sps.skew(ret) * sharpe) + ((sps.kurtosis(ret)-1)/4) * sharpe  
    return sps.norm.cdf(sharpe / np.sqrt( std / (len(ret) - 1)))

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed