Back

Simulation price is different with price on Google and Yahoo finance

2010-07-23 TLT price is day high 100.97 and day low 99.67 on Google or Yahoo.
How can system give 90.45 price to trade in?
2010-09-30 09:31:00 TLT $94.55092451 Market -2,403 Short Filled
2010-07-23 09:31:00 TLT $90.4501584 Market 2,403 Long Filled
Update Backtest








Welcome @Qingpeng, we use split and dividend adjusted data by default. Yahoo and Google apply the dividends separately. You can control this with the SetDataNormalization() controls -- see more here: https://www.quantconnect.com/forum/discussion/comment/2382
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jared!
If i want the price is the same with the one at Google or Yahoo, i should use which enum in DataNormalizationMode? DataNormalizationMode.TotalReturn?
Where can i find more documentation about this API? I am new to this platform.
If i use the same price with Google or Yahoo, the dividend is still paid in simulation or not?

AddSecurity(SecurityType.Equity, symbol, Resolution.Minute);
Securities[symbol].SetDataNormalizationMode(DataNormalizationMode.TotalReturn);
0

Thanks Jared. I find you post some times ago and it is very helpful.
https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization
0

I'm also seeing a difference between the closing prices I'm getting while backtesting vs the data I pulled from Tradier API & Google API. I've tried all the normalization modes and they don't seem to have an affect on the closing price. Also tried switching between FillDataForward with no affect. It's usually very small differences, about 1-4/100th difference but it's enough to throw off the outcome of an algorithm compared to my local environment.

For example:

TVIX 12/3/2015

Time......BackTest | Tradier (5 Minute)
1:51pm | 6.4100 |
1:52pm | 6.4200 |
1:53pm | 6.4000 |
1:54pm | 6.4000 |
1:55pm | 6.4100 | 6.44

Most of the time it's differences in rounding, such as 6.11 vs 6.10, but on occasion they get a few digits off like above.

Any thoughts?
0

Sorry its impossible to say - we don't know how Google or Tradier process their data, or where it comes from. Some public facing websites only display BATS exchange data which is much cheaper but is only 1/10th the volume of the primary exchanges.

For backtesting we build our bars from ticks, taken from all the exchanges. We apply a filter on those ticks for the bars. Advanced users who request ticks can do their own filtering.


public override bool PassesFilter(Tick tick)
{
// filter out bad ticks for equities
if (tick.Suspicious) return false;
if (tick.SaleCondition == "W") return false;
switch (tick.Exchange)
{
case "W": //Chicago Options
return false;
case "u": //OTC
return false;
case "U": //OTC Bulletin
return false;
case "D": //FINRA ADF
return false;
}
return true;
}


In live trading our live data source has the same high fidelity tick stream, and we build those bars on the fly.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Interesting.. I tried using the TickConsolidator->5min tradebars and applying your filtering. I get the exact sames results as before, which is good, I reached parity. I then varied combinations of exchanges where I started to see the prices start to change a little. I wasn't able to find the right combination to yield the results I'm getting from Tradier so i'm working with them to see if I can find out how they're building their data. Hopefully I can find the right combo. It's very important I get this. I've local scripts that spend hours calibrating combinations of configurations for each security that I plug into QC algo. When my local data differs from QC data, it's just a fail.

In the meantime, could any suggest an API I could use that would yield the EXACT same results I get while back testing in QC? And I mean always precisely the exact same down to the 1/10000 of a cent. I know QuantQuote is recommended, but they took a week+ to get purchased data dump last year and I'm not that patient. I'd rather have access to an API where I can get what I want when I want and feed it into a local db. Any suggestions? Paid APIs are cool so long as it's reasonably priced.
0

@Jared, thanks for the insight into how you build your data. That was very very helpful. I ended up abandoning Tradier API in favor of ActiveTick (AT) API which already seems to be much more inline with QC data. After closely comparing AT's tick data against QC's minute data, I've concluded you are indeed filtering your tick data as described in your filtering method.

So I have two options to synchronize my local db with QC.

1. Rebuild my local db for 1 minute resolution from AT's consolidated tick data follow your filtering rules (probably have two fields for close, Close and QCClose so I could use either)

2. Or rebuild my local db with AT's provided 1 minute resolution & use the TickConsolidator in my QC Algo and use data from all exchanges (including W u U D) because it seems AT doesn't do any filtering.

Both will yield me identical results local vs backtest. But I really have to wonder. Why are you filtering out those exchanges? Is there a good reason not to include ticks from them? It seems D was the last Tick of a minute more often than others, so I just don't know which strategy is better.
0

Hmm tricky questions @Levi. More fundamentally rather than what data to use; I think it comes down to what you can execute on and I'm not sure its practical to build a strategy which is sensitive to tick by tick variations without having co-located infrastructure and a brokerage which will let you execute on a tick by tick basis. These are available (Equinox Colo + Lime Brokerage) but will cost you roughly $1500/mo for equinox + $3000/mo for lime. Our data-feed is tick by tick, but the execution is through your brokerage (IB/Tradier/Oanda/FXCM) -- so if your brokerage is slow or a market maker there is no guarantee you'll fill at the quote price. Tradier for example patches their trades through to Knight which has to match the NBBO (national best bid or offer) more than 90% of the time. This has very little to do with the trades/quotes you'll receive from any datafeed.

What are your holding periods on average? Is it ballpark 1 second or 1 minute? Typically we say QC.com is suitable for holdings periods down to 20-30 seconds on volatile tickers. If you write your own brokerage connections and self-host LEAN in NY4/Equinox you could probably get it down to milliseconds.

We filter out those ticks as the majority of the time they have bad or delayed ticks, which cause spikes and are unrealistic price you can never fill at. The majority of "TradeBar" users don't care about those ticks but care about prices they can achieve. We provide the tick resolution data stream if you want to do something more advanced.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hello,

I am new here and was testing QC backtest data and have noticed the following. I will not post long list of prices here but point out couple.

TSLA

Trade Date: 03/28/2017

5 Minute TradeBar: Time 8:55 has a High of 278.35 on IB, TDAmeritrade etc.

When the Same Bar, consolidated via code using Tick Resolution on Security and using TickConsolidator, to form 5 MinuteBar in QC

I am getting a price of 278.38.

I have noticed so many of these differences to loose my confidence at this point.

I have changed the resolution from minutes, seconds to Tick. This helped a little bit to more accurate.

How do we handle this in live? Use broker feed fo back testing to be more accurate? Please suggest for best way to move forward.

 

0

Time 9:55 EST sorry. I am in CST.

0

Sai; please read the thread above. Its laid out clearly there.

Ultimately we build our data from ticks which is much more accurate than Yahoo or Google's daily sources. In live trading you can't filter out ticks after the fact - you don't know what will be marked as invalid. 

Ticks arrive late, or are invalid prices far from market price. When you request tick data we expect you to understand that. If you use second or minute bars we pre-filter the common causes of price error as shown in above discussion.

Also note the terms and conditions prohibit logging price data due to data vendor restrictions. If you'd like to download the raw data you can purchase it from QuantQuote.com

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed