Back

How to backtest market-wide fundamental data

Hi everyone. I'm wondering if anyone has a good solution for how to backtest historical fundamental data in QC including historical analyst upgrades/downgrades and historical short interest if possible. Related to this, I'm wondering how one might go about setting up a market-wide analysis of some historical value (say PE ratio) and organizing that value into deciles as a filter.
Update Backtest








Hey AndresDiana - Working on it.. we talked with MorningStar this morning. Finding a large high quality data source is really the hardest part. For now you can pull in Quandl's data but you need to hand pick each stock
1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jared. According to Quandl there is unlimited access to their data so in theory one could set up a universe of symbols and just download all the relevant information stock by stock. I think the only thing they don't have are historical upgrades & downgrades. As you evaluate data sources, the analyst history might be something to keep in mind.
Thanks!
0

Thanks @Andres - we tested Quandl but its full of holes when you try and use it for all 16,000 symbols. If you're running a stock screener/selection strategy you can't risk having holes or it will self select based on data available..:\
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


What if the data were more simple? IE Finding the X stocks that had the biggest gain/loss in a day/week/month? Is this data available?
0

Sorry Chris at the moment not really - unless you build it manually yourself. We'll address it properly after live trading. I'm painfully aware its a gaping hole in QC as it exposes algorithms to selection bias.

The plan for after live trading is to build a new "daily" dataset from our ticks and allow a generic filter to be applied. This daily source would also allow fetching history for each asset. But trying to provide that filtering currently using tick data would be impossible (terabytes of information to process)...

To do it manually: 1. Get a list of the S&P100 stocks over the last 5 years. 2. Add all those symbols to QC. 3. In your OnData function create your filter. 4. Use your top 10 performers.. It would be painfully slow though. Since you're using high resolution data to create your filter.
2

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jared!
0

Hi Jared,

Sorry to resurrect this thread from the dead. Do you happen to know where we stand with fundamental data? Did the Morningstar interface work out?

Thanks!
1

Hey! We're working through the contract details with Morning star now! We've already laid the technical foundation (Universe selection) and just need the data. ETA 1-2 months.

You can use universe selection now to select based on price/volume and then add in fundamentals once they are ready.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Any update to the Morningstar integration.  Backtesting on fundamental data is a prerequisite for my algorithmns.  

0

Actually I just got a really cool email from Quandl yesterday saying that they've added a new, incredibly massive fundamentals database.  It's located here:

https://www.quandl.com/databases/MF1

0

@Brian, @Stephen -- Coming very soon. Stefano is hard at work and we're just figuring out how to use the Fundamental data in LEAN :) It will be available for backtesting this month. There will be another week or two delay after that for live trading.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Did we gte this part done in Lean yet ?

0

Yes, we did.
Please checkout documentation under Universes section.

I have attached a working example.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Alex,

I think Andres original question was: " I'm wondering how one might go about setting up a market-wide analysis of some historical value (say PE ratio)"

Right now with how the universe selection is done (coarse vs fine), this is not possible.  The PE/EPS/other fundamental datapoints are only available in the fine universe function and that dies with an out of memory execption if you have more than 100 equities you are looking at.  

 

I am dying for this functionality...  I think I've been looking for an affordable platform to run backtests and automated training with this type of analysis for the last 5 years with no success.  (Quantopia didn't work either.)  Please tell me if I've misunderstood the capabilities.  Thanks...

0

This is absolutely possible; though perhaps not enough for your specific use case. For 99% of people its sufficient to do some rough filtering first and then do fine universe filtering on a smaller subset of the data. With a quantitative model most people need liquidity and the first coarse filter is on the liquidity and average volumes.

We'll add an upgrade to increase the memory allocation limits further for people with your use case. In general though it is not common. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jared, thanks for your response....  I'm sure the platform will mature over time to cover my use case. You guys seem to be savvy and passionate.  

Your statement regarding liquidity... it kind of surprises me.  I might be more naiive than i realized. I thought I would be able to get into a position fairly easily with 50k-100k, excluding penny stocks.  Is the platform meant for people that invest in a larger volume in a single trade where liquidity is an issue?   I'm making this up, but I assumed I could get into 80% of the market with a 50k buy without a liquidity issue.  

Thanks Jared, I do appreciate yours and your teams support in answering my questions.  

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed