Back

Inaccurate Market Data History

Folks,

I am trying to get open,close,high, and low data for certain market day. I have put together python script to what I think it should give me what I need. However, I am having 2 concerns here:

1. I requested only 1 day data (1/3/2019), however, onData methind is triggered twice and ended up with 2 sets of data instead of one day.

2. Data desplayed in 2nd set, seem to be close to what I see in thinkorswim charting software but there are not exact!

Also, how can I go to previous day data if I had requested data for one week?

Thanks for your help

Script

class getDailyCandleStick(QCAlgorithm):

    def Initialize(self):
        self.SetStartDate(2019, 1, 3)  # Set Start Date
        self.SetEndDate(2019,1,3)
        self.SetCash(100000)  # Set Strategy Cash
        self.symbol = "SPY"
        self.equty = self.AddEquity(self.symbol, Resolution.Daily)
        self.equty.SetDataNormalizationMode(DataNormalizationMode.Raw)


    def OnData(self, data):
        '''OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
            Arguments:
                data: Slice object keyed by symbol containing the stock data
        '''

        self.Debug("OPen Price:" + str(data[self.symbol].Open))
        self.Debug("Close Price:" + str(data[self.symbol].Close))
        self.Debug("High Price:" + str(data[self.symbol].High))
        self.Debug("Low Price:" + str(data[self.symbol].Low))

Output:

155 | 15:19:12:

Launching analysis for 8d79c65545fea28f8ef696fad964863a with LEAN Engine v2.4.0.0.5954

156 | 15:19:13: OPen Price: 245.98

157 | 15:19:13: Close Price: 250.26

158 | 15:19:13: High Price: 251.21

159 | 15:19:13: Low Price: 245.95

60 | 15:19:13:

Algorithm (8d79c65545fea28f8ef696fad964863a) Completed.

161 | 15:19:13: OPen Price: 248.26

162 | 15:19:13: Close Price: 244.13

163 | 15:19:13: High Price: 248.57

164 | 15:19:13: Low Price: 243.66

165 | 15:19:13: Algorithm Id:(8d79c65545fea28f8ef696fad964863a) completed in 0.51 seconds at 0k data points per second. Processing total of 3 data points.

√√

Update Backtest





The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Data naturally differs between providers, there have been a lot of posts about this in the forums in the past.

Setting the same starting and ending date is not advisable and can yield interesting results. I suspect it's setting the start date to the day before, but I could be wrong. Add the datetime of the incoming data piece to your logs to see what dates the prices correspond to.

0

Thanks Douglas for your help. Not sure what you mean by data differs between providers! I searched the intenet for historic market data and they all show same open,close,high, and low for same day.

Not having accurate data is fundemental issue, If my algorithm is set to place order few cents above market close price, it will not work since the market close price my algorithm is getting is not accuarete in first place.

I tried to set end date 1 day after start (start date is 1/3 and end date is 1/4) and now I ma getting 3 data points!

There is an additional onData trigger that nobody knows where is it coming from. I searched all document and there is no mention of that. I wasn't sure hopw to add datetime to my log.

 

0

data[self.symbol].EndTime gives you the datetime that the incoming bar is ending on. data[self.symbol].Time is the starting time of the bar. This should tell you what days the three log outputs are from. If you don't know what day the prices are from, how can you know they are inaccurate?

1

Thanks Douglas for following up .. I was corellating the OHLC reported from websites and charting software to the data set prints.

I added the log you suggested and you are right! Data was starting from day eerlier. So If my start date is 1/3 and end date is 1/4, onData is called with 1/2, 1/3, and 1/4 daily bars. Not sure why 1/2 was reported since I didn't ask for it.

Anyhow, reported daily data again didn't match reported data by other sources. I guess my last hope here is to define my own data class and find an accurate data feed to interface with my custom data class.

Wish me luck!

0

This is called the EndTime of the bar and is covered in BootCamp. Please

see the documentation on "Understanding Time" in "Key Concepts"
1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Jared. Thanks for your clarification. According to the documentaion here

https://www.quantconnect.com/docs/algorithm-reference/handling-data#Handling-Data-Data-Formats

If I set resolution to Daily, I should be expecting candlestick data similar to what finencial feeds and charting software provide. What you seem to be trying to imply is, the open price of 1/3 is not known until end of 2/1. If I understand this correctly, you are also inluding after hour stock price of all 1/2!. When I look at the function prototype od AddEquity, 

// Complete Add Equity API - Including Default Parameters:
AddEquity(string ticker, Resolution resolution = Resolution.Minute, string market = Market.USA, bool fillDataForward = true, decimal leverage = 0m, bool extendedMarketHours = false)

ExtendedMarketHours is set to false by default and sisnce I don't see it in my code as shown above, it should not include any after hours pricing.

 

0

If you believe there is an issue with the data, you should report a data issue.

However, before you do this, please see the following FAQ on why prices sometimes differ from other providers:

"QuantConnect builds our TradeBars from tick data. While doing this we filter out ticks which our vendor believes are "suspicious". These suspicious trades include ones which are rolled back, reported late, or traded via OTC markets. This makes the bars a more realistic representation of the asset price but it might vary slightly from popular web-portals like Yahoo etc. Although this results in slightly different prices we believe its more important to have realistic backtests than match Yahoo."

Long story short - QC places a heavier focus on making sure the data represents actual, tradable prices at the time of receipt than other data providers might do.

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed