Back

Data processing and consolidate issue

Hi,

I started used quant library but I found problem with data processing. So, I used Data section on web page to download the Oanda data for symbol DE30EUR on my PC. After that I developed easy project to check how this work but the result is supprising.

First issue how I see that price on Oanda Web PLatform show difference price then I got 

For 01.12.2017 I see:

Open : 13062.4 High : 13181.35 Low : 13023.3 Close : 13052.15

But on Oanda:

Open : 13052.1 High : 13076.1 Low : 12810.8 Close : 12944.3

 

Second problem is with day 04.12.2017. The only one price is avaiable for whole day is 13052.15. I unzipped file "20171204_de30eur_minute_quote.csv" and inside it quote are various.

 

Please take look on my code and log file and advice how resolve those issues.

Thanks,

Grzegorz

 

 

This is my code

private const string _symbol = "DE30EUR"; // "UK100GBP"
private const decimal _cash = 100 * 1000;

private readonly DateTime _startDate = new DateTime(2017, 12, 01);
private readonly DateTime _endDate = new DateTime(2017, 12, 06);

public override void Initialize()
{
AddSecurity(SecurityType.Cfd, "DE30EUR", Resolution.Minute, Market.Oanda, true, 10.0m, false);
SetTimeZone("UTC");
SetStartDate(_startDate);
SetEndDate(_endDate);
SetCash(_cash);

var daylyConsolidator = new QuoteBarConsolidator(TimeSpan.FromDays(1));
SubscriptionManager.AddConsolidator(_symbol, daylyConsolidator);
daylyConsolidator.DataConsolidated += OnDataDay;


Debug("");
Debug("Start");
Debug("---------------------------------------------------------------------------------");
SetBenchmark(_symbol);
}

private void OnDataDay(object sender, QuoteBar consolidated)
{
QuoteBar _quoteBar;
_quoteBar = consolidated;
Debug("------- Day Consolidate: " + _quoteBar.Time.ToShortDateString());
Debug(" Open : " + _quoteBar.Open.ToString());
Debug(" High : " + _quoteBar.High.ToString());
Debug(" Low : " + _quoteBar.Low.ToString());
Debug(" Close : " + _quoteBar.Close.ToString());
}

public override void OnData(Slice data)
{
QuoteBar q = data.QuoteBars[_symbol];
Debug("------- New Data: " +
data.Time.ToShortDateString() + " " +
data.Time.ToShortTimeString() + " " +
q.Open.ToString() + " " + q.Close.ToString());
}

 

And this is snapshot from my log: 

2017-12-14T21:24:43.0892792Z TRACE:: Config.Get(): Configuration key not found. Key: version-id - Using default value: 
2017-12-14T21:24:43.0892792Z TRACE:: Config.Get(): Configuration key not found. Key: cache-location - Using default value: C:\Projects\QData
2017-12-14T21:24:43.0892792Z TRACE:: Engine.Main(): LEAN ALGORITHMIC TRADING ENGINE v2.4.0.1 Mode: DEBUG (64bit)
2017-12-14T21:24:43.0892792Z TRACE:: Engine.Main(): Started 21:24
2017-12-14T21:24:43.6742545Z TRACE:: Engine.Main(): Memory 62Mb-App  3Mb-Used  9515Mb-Total
2017-12-14T21:24:43.6898798Z TRACE:: Config.GetValue(): job-project-id - Using default value: 0
2017-12-14T21:24:43.6898798Z TRACE:: Config.Get(): Configuration key not found. Key: algorithm-path-python - Using default value: ../../../Algorithm.Python/
2017-12-14T21:24:43.6898798Z TRACE:: Config.Get(): Configuration key not found. Key: lean-manager-type - Using default value: LocalLeanManager
2017-12-14T21:24:44.0751768Z TRACE:: JobQueue.NextJob(): Selected QTrader.dll
2017-12-14T21:24:44.1376873Z TRACE:: Config.GetValue(): ignore-version-checks - Using default value: False
2017-12-14T21:24:44.1376873Z TRACE:: Config.GetValue(): algorithm-manager-time-loop-maximum - Using default value: 20
2017-12-14T21:24:44.1598252Z TRACE:: Loader.TryCreateILAlgorithm(): Loading only the algorithm assembly
2017-12-14T21:24:44.2067201Z TRACE:: Loader.TryCreateILAlgorithm(): Loaded QTraderAlgorithm
2017-12-14T21:24:45.2081471Z TRACE:: Config.Get(): Configuration key not found. Key: security-data-feeds - Using default value: 
2017-12-14T21:24:45.3239093Z ERROR:: DefaultDataProvider.Fetch(): The specified file was not found: C:\Projects\QData\cfd\oanda\minute\de30eur\19971230_quote.zip
2017-12-14T21:24:45.3239093Z ERROR:: DefaultDataProvider.Fetch(): The specified file was not found: C:\Projects\QData\cfd\oanda\minute\de30eur\19971231_quote.zip
2017-12-14T21:24:45.3239093Z TRACE:: BrokerageModelSecurityInitializer.Initialize(): Unable to seed security: DE30EUR
2017-12-14T21:24:45.3616803Z TRACE:: BacktestingResultHandler(): Sample Period Set: 04.00
2017-12-14T21:24:45.3616803Z TRACE:: Time.TradeableDates(): Security Count: 1
2017-12-14T21:24:45.3616803Z TRACE:: Config.GetValue(): forward-console-messages - Using default value: True
2017-12-14T21:24:45.3616803Z TRACE:: JOB HANDLERS: 
2017-12-14T21:24:45.3616803Z TRACE::          DataFeed:     QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed
2017-12-14T21:24:45.3616803Z TRACE::          Setup:        QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler
2017-12-14T21:24:45.3616803Z TRACE::          RealTime:     QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler
2017-12-14T21:24:45.3616803Z TRACE::          Results:      QuantConnect.Lean.Engine.Results.BacktestingResultHandler
2017-12-14T21:24:45.3616803Z TRACE::          Transactions: QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler
2017-12-14T21:24:45.3616803Z TRACE::          History Provider:     QuantConnect.Lean.Engine.HistoricalData.SubscriptionDataReaderHistoryProvider
2017-12-14T21:24:45.3929324Z TRACE:: Config.GetValue(): send-via-api - Using default value: False
2017-12-14T21:24:45.4085571Z TRACE:: Debug: Launching analysis for QTraderAlgorithm with LEAN Engine v2.4.0.1
2017-12-14T21:24:45.4241826Z TRACE:: AlgorithmManager.Run(): Begin DataStream - Start: 01/12/2017 00:00:00 Stop: 06/12/2017 23:59:59
2017-12-14T21:24:45.4609400Z TRACE:: FileSystemDataFeed.GetEnumerator(): Begin: 01/12/2017 00:00:00 UTC
2017-12-14T21:24:46.5961918Z TRACE:: Cash.EnsureCurrencyDataFeed(): Adding EURUSD for cash EUR currency feed
2017-12-14T21:24:46.7638382Z TRACE:: Debug: Start
2017-12-14T21:24:46.7643384Z TRACE:: Debug: ---------------------------------------------------------------------------------
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:00 13062.4 13067
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:01 13067.0 13069.55
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:02 13069.6 13072.25
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:03 13072.4 13070.25
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:04 13070.1 13063.05
2017-12-14T21:24:46.7648386Z TRACE:: Debug: ------- New Data: 01/12/2017 08:05 13063.1 13063.9

[...]

017-12-14T21:24:46.9810629Z TRACE:: Debug: -------  Day Consolidate: 01/12/2017
2017-12-14T21:24:46.9810629Z TRACE:: Debug:    Open : 13062.4
2017-12-14T21:24:46.9810629Z TRACE:: Debug:    High : 13181.35
2017-12-14T21:24:46.9810629Z TRACE:: Debug:    Low : 13023.3
2017-12-14T21:24:46.9810629Z TRACE:: Debug:    Close : 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:00 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- Nowy Dzien: 04/12/2017 07:01
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:01 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:02 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:03 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:04 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:05 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:06 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:07 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:08 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:09 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:10 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:11 13052.15 13052.15
2017-12-14T21:24:46.9810629Z TRACE:: Debug: ------- New Data: 04/12/2017 08:12 13052.15 13052.15

[..]

2017-12-14T21:24:47.2126371Z TRACE:: Debug: ------- New Data: 04/12/2017 21:57 13052.15 13052.15
2017-12-14T21:24:47.2126371Z TRACE:: Debug: ------- New Data: 04/12/2017 21:58 13052.15 13052.15
2017-12-14T21:24:47.2126371Z TRACE:: Debug: ------- New Data: 04/12/2017 21:59 13052.15 13052.15
2017-12-14T21:24:47.2126371Z TRACE:: Debug: ---------------------------------------------------------------------------------
2017-12-14T21:24:47.2126371Z TRACE:: Debug: -------  Day Consolidate: 04/12/2017
2017-12-14T21:24:47.2126371Z TRACE:: Debug:    Open : 13052.15
2017-12-14T21:24:47.2282621Z TRACE:: Debug:    High : 13052.15
2017-12-14T21:24:47.2282621Z TRACE:: Debug:    Low : 13052.15
2017-12-14T21:24:47.2282621Z TRACE:: Debug:    Close : 13052.15

Update Backtest








Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed