Algorithm never completes when using external data

I have created custom data sets for Lean based on the same forex data format, but using data from an external source. Data is 1min and each file has been broken into 1 day from 00:00 through 23:59, and zipped. File formatting and naming is the same (other than the inclusion of volume in my data).

Sample data files are available here.

I have found that UTC+2 works best for this data (FXCM sourced via IQFeed) as I don't get any Sat/Sun data files.

There are sometimes data files that don't a complete 24 hour period, and certain days that don't exist (ie. Sat/Sun). There will be other times when the first trading day of the week doesn't kick off right at 00:00. My assumption is that using Fill Forward sorts out any errors relating to these issues, and this seems to be correct based on the limited understanding of QC.

My issue is when I run a backtest against my data, there are times when the algorithm never finishes, eg: running the MovingAverageCross algorithm with default settings other than:
private const string Symbol = "AUDUSD";
AddSecurity(SecurityType.Forex, Symbol, Resolution.Minute, true);

In this example, the End Date is 1 Jan 2015. When running this against my data (which has not file for this date), the following output is received:

What looks to be occuring is that the algorithm is waiting for data from the FileSystemDataFeed that never arrives (and never will since it doesn't exist). The Isolator.ExecuteWithTimeLimit() message occurs regularly (every 60 seconds).

If I change the End Data to a day when there is a data file available, the algorithm seems to execute (and end) correctly.

Am I missing something here? Is there a way to Fill Forward across days to resolve this issue?

I don't really want to add a check for file existence into the algo as this means nothing other than in backtesting.


Update Backtest

Update Backtest


The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


This discussion is closed