Share New Research 
Sometimes when you're downloading a years worth of historic stock data on a ticker, say the example following command:
dotnet QuantConnect.ToolBox.dll --app=PDL --tickers=XLP --resolution=Hour --from-date=20190101-00:00:00 --to-d
ate=20200101-00:00:00 --security-type=Equity --market=usa --api=xxxxxxxxxx
you'll run into an SSL Connection dropped error and have to reissue the command. I've been checking where the downloader left off and will adjust my --from-date accordingly. The issue I am seeing however when I examine my historic stock data is data will be stored in the csv (which is zipped and found in the
~/Lean/Data/equity/usa/hour/qqq.zip) for example like below:
20190102 11:00,502800,505200,502300,503600,2658141
20190102 12:00,503700,505450,502900,505200,1806334
20190102 13:00,505200,506400,504900,506300,1762108
20190102 14:00,506200,506200,503900,504300,1580498
20190102 15:00,504300,505700,501700,505200,0,504400,505800,501800,505300,0
20190102 16:00,505000,505000,489200,490000,0,505300,520200,504800,504800,0
20190102 17:00,490000,502000,490000,500100,0,504800,504800,502000,504000,0
20190102 18:00,500100,500600,498100,498200,0,504000,504000,500000,503800,0
20190102 19:00,498700,498700,498700,498700,0,503800,503800,503800,503800,0Has anyone run into this behavior when downloading historic stock data via the Quantconnect Toolbox? My question is, is this concerning or will this still work when conducting local backtests? Does the downloader know not to double up the data if it re-runs on a particular date when stored in the csv file within the Data directory?
Louis Szeto
Hi Axist
The default behavior would be aggregating the old entry OHLV with the newly acquired bar in every call for hour/daily data, while overwriting the old entry if the resolution is minute/second/tick. (source)
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Axist
So in theory, I should be able to delete after the first 0 in the example?
If all I did was re-run the dotnet QuantConnect.ToolBox.dll line above on the same dates, why would the aggerated portion after the first zero be any different than the first collected?
Louis Szeto
Hi Axist
They are quotebar, so in default they have no volume. 😊
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Axist
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!