Share New Research 
I am playing around with QC fundamental universe selection. I am having issues with the backtest taking way too long.
I was trying to approximate the Quantopian Q500US universe. I've added debug statements to try and figure out where it's spending all it's time but they haven't helped identify root cause yet.
Backtest is still running right now, been going for 15+ mins and processed a few days. Seeing ~2min gap between calls to filter functions, filtering itself doesn't seem to take that long. Filtering seems to always return 0 which is another issue but it's taking too long to try and debug that right now.
Start of log:
34 | 11:15:47: Launching analysis for 4b6f8b1ba2b3fc85491c179b03ae63f0 with LEAN Engine v2.4.0.1.2978
35 | 11:15:47: OnData: 1/1/2015 12:00:00 AM, bars: 2
36 | 11:15:47: Adding security: SPY
37 | 11:15:47: Adding security: TLT
38 | 11:17:11: 1/2/2015 12:00:00 AM coarse selection count: 1000
39 | 11:17:11: 1/2/2015 12:00:00 AM fine universe selection: 1000
40 | 11:17:11: filtered: 0
41 | 11:17:11: OnData: 1/3/2015 12:00:00 AM, bars: 2
42 | 11:19:13: 1/5/2015 12:00:00 AM coarse selection count: 1000
43 | 11:19:13: 1/5/2015 12:00:00 AM fine universe selection: 1000
44 | 11:19:13: filtered: 0
45 | 11:19:13: OnData: 1/6/2015 12:00:00 AM, bars: 2
46 | 11:21:16: Backtest deployed in 591.496 seconds
47 | 11:21:16: 1/6/2015 12:00:00 AM coarse selection count: 1000
48 | 11:21:16: 1/6/2015 12:00:00 AM fine universe selection: 1000
49 | 11:21:16: filtered: 0
50 | 11:21:16: OnData: 1/7/2015 12:00:00 AM, bars: 2
51 | 11:23:19: 1/7/2015 12:00:00 AM coarse selection count: 1000
52 | 11:23:19: 1/7/2015 12:00:00 AM fine universe selection: 1000
53 | 11:23:19: filtered: 0
54 | 11:23:19: OnData: 1/8/2015 12:00:00 AM, bars: 2
Thought I'd post it to community and see if anyone else has had similar issues or sees something I'm not.
Any help is appreciated.
Thanks!
Jared Broad
A while back we made the decision to remove code limits and let people abuse the system within memory limits. You're requesting 1000 fine symbols of data -- this will take a long time to open and process (roughly 1GB/day of data).
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Aaron Todd
I see, that makes sense. Thanks for the explanation Jared.
The filtering is trying to approximate the Quantopian Q500 universe. Is there a better way to achieve that goal? Would be great to see a few more parameters in coarse selection such as market cap.
Somehow Quantopian is allowing filtering at this sort of fine grained level with pretty reasonable performance, curious to know how they are doing it (not a negative comment just curious, still love what you guys are doing).
Aaron Todd
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!