I run a local algo which has a future added through AddFuture method call, and I simply prints the close price of each timestamp. But whatever the value of fillDataForward is, the missing data are filled with the previous one. This is what I did:
- The underlying for the test is “ES”, and the test duration is from 6 Oct 2013 till 7 Oct 2013. The minute data is used by LEAN's unit test project so it is there in the source repo of LEAN.
- I find out the concrete future contract symbol in the mapping file, which says it is 2013-12 for open interest data mapping mode.
- The data file should then be 20131006_trade.zip, and the entry should be 20131006_es_minute_trade_201312_20131220.csv. Open it and remove some lines.
- Run the algo pasted in the post and take a look at the output. There is no “gaps” between data points even I set fillDataForward to false.
namespace Lean.Play
{
public class Algo: QCAlgorithm
{
private Future future;
private Symbol symbol;
public override void Initialize()
{
SetStartDate(2013, 10, 6);
SetEndDate(2013, 10, 7);
SetCash(100000);
future = AddFuture("es", Resolution.Minute, Market.CME, dataNormalizationMode: DataNormalizationMode.Raw, fillDataForward: false);
symbol = future.Symbol;
}
public override void OnData(Slice slice)
{
if(slice.Bars.ContainsKey(symbol))
{
var contractYearMonth = future.Mapped.ID.Date.ToString("yyyyMMdd");
var contractTicker = $"{symbol.ToString()}{contractYearMonth}";
Log($"{contractTicker}@{Time} ({UtcTime}UTC): price = {slice.Bars[symbol].Price}, oi = {Securities[symbol].OpenInterest}");
}
else
{
Log($"{Time}: no data");
}
}
}
}
Louis Szeto
Hi Laye
OnData is an event handler that get called whenever there is data received. In your example algorithm, only 1 asset data was subscribed so OnData get called when there was data received for ES. If you check the `Time` in your logs, you should be able to spot there are missing points.
Best
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laye
Thank you for your response, Louis.
But my test result doesn't support your statement. Let me show you the data I have:
The time series above is 22:00, 22:01, 22:02, 22:03, 22:10, 22:11. So there are missing data from 22:04 to 22:09. When I have fillForwardData set to false, I guess the slice object passed to OnData() has no data at all during that period. But what I got is:
Clearly the data from 22:05 to 22:10 are filled with the previous one, which is the bug I said, fillDataForward flag is ignore. The data I used is from LEAN repo's data/future/cme/minute/es/20131006_trade.zip. To duplicate the issue, please follow the steps and run the code in my initial post. Thank you.
Louis Szeto
Hi Laye
We cannot reproduce your issue. Using the exact same snippets you've provided, changing only the fillDataForward argument, we yield the below results:
- fillDataForward: false
fillDataForward: false- fillDataForward: true
fillDataForward: trueBest
Louis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laye
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!