Discussion Forum - QuantConnect.com

Bronx verzio INVESTOR

| Reward Discussion

Hi everyone, i just stumbled in a trouble that i can't figure out: if i use hourly data for the pair EURUSD (FXMP) all the candles cooresponds to the ones that are showed in the tradingview charts, all the values (Open, High, Low, Close) corresponds in a perfect way; the problem is that when i consolidate the data into daily, there is no more correspondence with the data from the same tradingview chart of before (with daily timeframe).

The following is the code that i use to get daily candles from hourly ones, maybe i am missing something

# in Initialize() i build a 10 days rollingwindow consolidating from hourly data

self.dailywindow = RollingWindow[QuoteBar](10)

        self.Consolidate("EURUSD", Resolution.Daily, lambda x: self.dailywindow.Add(x))





# in OnData() i use this code for printing the OHLC values to check them

if self.dailywindow.IsReady:

        self.Debug(" daily OHLC {0},{1},{2},{3}, ".format(self.dailywindow[0].Open,self.dailywindow[0].High,self.dailywindow[0].Low,self.dailywindow[0].Close))

since i am implementing a two-timeframe strategy it is important that the both hourly and daily data are consistent with FXMP official data (that i look at from tradingview)

Any kind of help or feedback would be appreciate ;)

Update Backtest

Bronx verzio

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Shile Wen

63.5k ,

Hi Bronx,

If you could attach a backtest, as well as a couple of days of daily data from TradingView, it would make it easier for us to assess this issue. However, for now, one idea is that because the values aren’t printed with self.Debug until the RollingWindow is full, the candlestick/bar data is print on a 10-day delay, which can be fixed by adding self.SetWarmup(10) inside the Initialize method.

Best,
Shile Wen

Bronx verzio

260 ,

Thank you for the answer Shile Wen,

i attach the project and the backtest. It contains the way in which i consolidate the candles and print them with debug, so that is easy to retrieve from the log file the OHLC values for each day to compare with tradingview in no time.

I tried self.SetWarmup(10) but the values don't match yet.

could it be a matter of timezone?

Regards,

Bronx

Randall Cooper

1.4k ,

Hi Bronx,

I've been working with FX data a lot and have the same problem between QuantConnect (QC) and TradingView (TV) daily data. Below is what I've found and how I've worked around the problem so far.

To my knowledge, the following are true, which I think is consistent with your experience:

TV and Oanda data (i.e., data taken directly from an Oanda account or the Oanda API) always agree for all time frames.
QC and TV data agree for hourly bars and below
QC and TV data disagree for daily bars, whether the daily bars are taken directly from QC or consolidated from, e.g., hourly QC bars

I think, but am not sure, that (3) may be a result of how QC constructs FX data, but if I am wrong, someone please correct me. Regardless, I think the overarching reason is that the FX is a 24/5 market, but the generally accepted "close" happens at 5pm NYC time instead of midnight NYC time, and consolidators don't take this into account. Therefore, this is basically what I do in the research and backtest environments.

Research Environment: When I want daily bars, I construct them from hourly bars instead of pull daily bars directly from qb.History (I would show you an example, but right now no one can open research notebooks). The trick is to add 7 hours to each hourly bar's timestamp before consolidating so the code "thinks" the day ends at midnight, but in reality ends at 5pm NYC time. As a side note, when you do this, you'll get 5 daily bars per week instead of 6, as desired.
Backtest Environment: Again, I construct daily data from hourly/minute bars, but here I will use scheduled events. For example, if you want to know the previous day's high and low, you can do something like the code snippet below.

I hope this helps. If anyone has better solutions, please let us know!

self.Schedule.On(self.DateRules.EveryDay(), self.TimeRules.At(17, 0), self.SetPrevDayHighLow)

self.Schedule.On(self.DateRules.EveryDay(), self.TimeRules.Every(timedelta(hours=1)), self.CurrDayHighLow)





    def SetPrevDayHighLow(self):

        """At the trading day's close, set previous extrema to the current day's extrema."""

        

        # Don't update on weekend days

        if self.Time.weekday() > 4:

            return

        

        for symbol in self.symbols:

            self.prevHigh[symbol] = self.currHigh[symbol]

            self.prevLow[symbol] = self.currLow[symbol]

            self.currHigh[symbol] = None

            self.currLow[symbol] = None





    def TradeAndCurrDayHighLow(self):

        """For the trading day, update that day's high and low prices."""

                

        for symbol in self.symbols:

            # Check if data exist for that symbol.

            if not self.CurrentSlice.ContainsKey(symbol):

                continue

            

            high = self.CurrentSlice[symbol].High

            low = self.CurrentSlice[symbol].Low

            

            if self.currHigh[symbol] is None or high > self.currHigh[symbol]:

                self.currHigh[symbol] = high

            if self.currLow[symbol] is None or low < self.currLow[symbol]:

                self.currLow[symbol] = low

Jared Broad

STAFF ,

Randall Cooper's reply is spot on -- we use GMT days, not eastern-time days which makes the daily data slightly different to some other platforms. At this point, it would break a lot of people's algorithms to switch the aggregation period so it's difficult to migrate it across.

Bronx verzio INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

240500 Quants.

Online Now

Stay Connected

Discrepancy in daily data when consolidating from hourly

Started By:

Join QuantConnect for Free Sign Up

Platform

SIGN IN

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

240500 Quants.

Online Now

Stay Connected

Discrepancy in daily data when consolidating from hourly

Started By:

Join QuantConnect for Free Sign Up