Hello,
First I apologize if this has been answered before but I did my best to search and could not find it.
Basically I want to develop and debug locally my strategy that trades ES Mini futures contracts. I generated data as per documentation `lean data generate --start 20210101 --security-type Future --resolution Minute --symbol-count 1 --market cme` and I can see that it added an e7 folder, so I thought I would use that for testing and switch to ES when backtesting via cloud. The problem is that it doesn't seem to be loading the data because it's not calling the OnData method.
My questions are:
- Do I need to generate a specific contract? If so how do I do this?
- Can I generate data for ES Mini locally? Or should I just rename the e7 folder?
Any help will be highly appreciated!
Fred Painchaud
Hi Alexandru,
When you generate random data locally, it is to use only locally. You cannot then use that data from the cloud (well, not easily).
Random data is also associated with random ticker. The ticker generated are in the folders of the data. So when you test locally, you use those names in the Add*() methods. If your configuration points to the proper folder, the random data will be found and used. By default, it is in the data folder of the project.
So, re your questions:
So when you generate for Futures, you generate random data (following brownian movements) for random Futures contracts, in the format of Futures data so the Algo can recognize it and read it.
To take another example, let's say I generate equity data. There is no such thing as generating for, say, SPY. I generate random data for fake ticker. Yes you can rename one such fake ticker into SPY but that would only confuse everything since that data is NOT really SPY data, it's data for a non-existing ticker…
Hope this makes it clearer.
Fred
Alex Bejan
Hey Fred, thanks for the reply.
Maybe I didn't explain correctly. I understand that the generated data is random, I'm just trying to load it in my strategy which is what is not working for me.
The generated folder is under …/data/future/cme/minute/e7 how do I load this in my algo? I tried with self.AddFuture('e7', resolution = Resolution.Minute, market = 'CME') or a few combinations with or without parameters but I'm not getting any data. It's initializing the algo but it's never calling OnData, probably because it's not loading the data.
If I generate an equity and load it with AddEquity it's working fine.
Fred Painchaud
The way you use to add the data should be ok. Hmmm., are you sure that the dates of the data and the dates of the algo are ok? Like, say. the data is for 2017 but the dates of your algo are not?
Or it may be a problem elsewhere or a problem with AddFuture.
So, as usual, if you could include a backtest (not working) but basically to include your code, it should help us.
Fred
Alex Bejan
Sure, I've included the code, it's from another post on the forum. The data I've generated is for all 2021 and as you can see in the code I'm also trying to backtest for the same period.
Can you try to generate a future and see if you can load it with my code?
Fred Painchaud
Hi Alex,
So I did generate future data locally and did some quick test to load the data the way you are.
And I also got no call for OnData.
The only reason I can think of is that for some reason, there is something with the generated data which makes the algo not select it as time goes by. Like the algo goes “Do I have data for that minute? No,”, “Do I have data for that minute? No,”, “Do I have data for that minute? No,”, etc. while it should find data.
Looking at the generated data, I noticed it does not have Open Interest data that goes with it, while other future data included with Lean CLI does. Maybe that makes the difference, I don't know enough about Future data in LEAN to say. And I'm pretty sure I noticed that lean data generate was supposed to generate open interest data (when it was logging its progress).
I would report to support@quantconnect.com as this looks like a problem, as far as I can tell.
Fred
Alex Bejan
Hi Fred, thank you so much for spending the time to replicate it. I've emailed support and will update this thread once I know more.
Alexandre Catarino
Hi Alex Bejan ,
Thank you for the report. And thank you, Fred Painchaud for helping to debug: it was a tricky one!.
We have created a GitHub issue to address the problem:
Random Data Generator Date Mismatch for Futures #6115
Please subscribe for updates.
Dayou Jiang
Hi, Alex Bejan and Alexandre Catarino, was this issue solved? I am also seeing this issue in my local test, data generation for equity works fine but not for futures.
Alex Bejan
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!