I am having issues with my research notebook stability when pulling in data for model training. While the code for pulling the data is well tested and usually works fine, especially early in the morning, it often crashes after 60-70% of the data has been loaded. This is extremely frustrating as it often takes 40-50 minutes to load the data. While the dataset is not large (25MB when stored as parquet), I already have to chunk the data to keep the notebook from crashing. Also, I have noticed that it seems to crash more often in the afternoons. It also seems to run much more slowly in the afternoons, but I haven't timed performance to say for certain. I am assuming because the servers have a higher load? Is there anyone else having these issues or found a solution?
Russell Hughes
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!