QUANTCONNECT COMMUNITY

No Results

Join Our Discord Channel

Join QuantConnect's Discord server for real-time support, where a vibrant community of traders and developers awaits to help you with any of your QuantConnect needs.

pending review This research is under review. To publish this research attract three community upvotes.

Draft Discussions

Bookmarked Discussions

Share New Research

Start New Discussion Sign up

SEARCH DISCUSSIONS

TOP 5 Research PUblications

About Quant League

The Open-Quant League is a quarterly competition between universities and investment clubs for the best performing strategy. Previous quarter's code is open-sourced, and competitors must adapt to survive.

competition rules

See the competition code of conduct and rules for participation in prizes.

Read Rules

previous competitions

Browse strategies and organization entries from previous quarter's competitions.

STRATEGY

284,500 Quants.

Become a Quant

VOTE FOR UPCOMING FEATURES

Share your input and vote on our future direction.

LEAN Roadmap

Create an account on QuantConnect for the latest delivered to your inbox.

Simulation price is different with price on Google and Yahoo finance

2010-07-23 TLT price is day high 100.97 and day low 99.67 on Google or Yahoo.

How can system give 90.45 price to trade in?

2010-09-30 09:31:00 TLT $94.55092451 Market -2,403 Short Filled

2010-07-23 09:31:00 TLT $90.4501584 Market 2,403 Long Filled

Update Backtest

person upvoted this people upvoted this

Qingpeng Niu

| |

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Jared Broad

STAFF ,

Welcome @Qingpeng, we use split and dividend adjusted data by default. Yahoo and Google apply the dividends separately. You can control this with the SetDataNormalization() controls -- see more here: https://www.quantconnect.com/forum/discussion/comment/2382

Qingpeng Niu

1k ,

Thanks Jared! If i want the price is the same with the one at Google or Yahoo, i should use which enum in DataNormalizationMode? DataNormalizationMode.TotalReturn? Where can i find more documentation about this API? I am new to this platform. If i use the same price with Google or Yahoo, the dividend is still paid in simulation or not? AddSecurity(SecurityType.Equity, symbol, Resolution.Minute); Securities[symbol].SetDataNormalizationMode(DataNormalizationMode.TotalReturn);

Qingpeng Niu

1k ,

Thanks Jared. I find you post some times ago and it is very helpful. https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization

Levitikon

4.2k ,

I'm also seeing a difference between the closing prices I'm getting while backtesting vs the data I pulled from Tradier API & Google API. I've tried all the normalization modes and they don't seem to have an affect on the closing price. Also tried switching between FillDataForward with no affect. It's usually very small differences, about 1-4/100th difference but it's enough to throw off the outcome of an algorithm compared to my local environment. For example: TVIX 12/3/2015 Time......BackTest | Tradier (5 Minute) 1:51pm | 6.4100 | 1:52pm | 6.4200 | 1:53pm | 6.4000 | 1:54pm | 6.4000 | 1:55pm | 6.4100 | 6.44 Most of the time it's differences in rounding, such as 6.11 vs 6.10, but on occasion they get a few digits off like above. Any thoughts?

Jared Broad

STAFF ,

Sorry its impossible to say - we don't know how Google or Tradier process their data, or where it comes from. Some public facing websites only display BATS exchange data which is much cheaper but is only 1/10th the volume of the primary exchanges. For backtesting we build our bars from ticks, taken from all the exchanges. We apply a filter on those ticks for the bars. Advanced users who request ticks can do their own filtering.


        public override bool PassesFilter(Tick tick)
        {
            // filter out bad ticks for equities
            if (tick.Suspicious) return false;
            if (tick.SaleCondition == "W") return false;
            switch (tick.Exchange)
            {
                case "W": //Chicago Options
                    return false;
                case "u": //OTC
                    return false;
                case "U": //OTC Bulletin
                    return false;
                case "D": //FINRA ADF
                    return false;
            }
            return true;
        }

In live trading our live data source has the same high fidelity tick stream, and we build those bars on the fly.

Levitikon

4.2k ,

Interesting.. I tried using the TickConsolidator->5min tradebars and applying your filtering. I get the exact sames results as before, which is good, I reached parity. I then varied combinations of exchanges where I started to see the prices start to change a little. I wasn't able to find the right combination to yield the results I'm getting from Tradier so i'm working with them to see if I can find out how they're building their data. Hopefully I can find the right combo. It's very important I get this. I've local scripts that spend hours calibrating combinations of configurations for each security that I plug into QC algo. When my local data differs from QC data, it's just a fail. In the meantime, could any suggest an API I could use that would yield the EXACT same results I get while back testing in QC? And I mean always precisely the exact same down to the 1/10000 of a cent. I know QuantQuote is recommended, but they took a week+ to get purchased data dump last year and I'm not that patient. I'd rather have access to an API where I can get what I want when I want and feed it into a local db. Any suggestions? Paid APIs are cool so long as it's reasonably priced.

Levitikon

4.2k ,

@Jared, thanks for the insight into how you build your data. That was very very helpful. I ended up abandoning Tradier API in favor of ActiveTick (AT) API which already seems to be much more inline with QC data. After closely comparing AT's tick data against QC's minute data, I've concluded you are indeed filtering your tick data as described in your filtering method. So I have two options to synchronize my local db with QC. 1. Rebuild my local db for 1 minute resolution from AT's consolidated tick data follow your filtering rules (probably have two fields for close, Close and QCClose so I could use either) 2. Or rebuild my local db with AT's provided 1 minute resolution & use the TickConsolidator in my QC Algo and use data from all exchanges (including W u U D) because it seems AT doesn't do any filtering. Both will yield me identical results local vs backtest. But I really have to wonder. Why are you filtering out those exchanges? Is there a good reason not to include ticks from them? It seems D was the last Tick of a minute more often than others, so I just don't know which strategy is better.

Jared Broad

STAFF ,

Hmm tricky questions @Levi. More fundamentally rather than what data to use; I think it comes down to what you can execute on and I'm not sure its practical to build a strategy which is sensitive to tick by tick variations without having co-located infrastructure and a brokerage which will let you execute on a tick by tick basis. These are available (Equinox Colo + Lime Brokerage) but will cost you roughly $1500/mo for equinox + $3000/mo for lime. Our data-feed is tick by tick, but the execution is through your brokerage (IB/Tradier/Oanda/FXCM) -- so if your brokerage is slow or a market maker there is no guarantee you'll fill at the quote price. Tradier for example patches their trades through to Knight which has to match the NBBO (national best bid or offer) more than 90% of the time. This has very little to do with the trades/quotes you'll receive from any datafeed. What are your holding periods on average? Is it ballpark 1 second or 1 minute? Typically we say QC.com is suitable for holdings periods down to 20-30 seconds on volatile tickers. If you write your own brokerage connections and self-host LEAN in NY4/Equinox you could probably get it down to milliseconds. We filter out those ticks as the majority of the time they have bad or delayed ticks, which cause spikes and are unrealistic price you can never fill at. The majority of "TradeBar" users don't care about those ticks but care about prices they can achieve. We provide the tick resolution data stream if you want to do something more advanced.

Sai P

69 ,

Hello,

I am new here and was testing QC backtest data and have noticed the following. I will not post long list of prices here but point out couple.

TSLA

Trade Date: 03/28/2017

5 Minute TradeBar: Time 8:55 has a High of 278.35 on IB, TDAmeritrade etc.

When the Same Bar, consolidated via code using Tick Resolution on Security and using TickConsolidator, to form 5 MinuteBar in QC

I am getting a price of 278.38.

I have noticed so many of these differences to loose my confidence at this point.

I have changed the resolution from minutes, seconds to Tick. This helped a little bit to more accurate.

How do we handle this in live? Use broker feed fo back testing to be more accurate? Please suggest for best way to move forward.

Sai P

69 ,

Time 9:55 EST sorry. I am in CST.

Jared Broad

STAFF ,

Sai; please read the thread above. Its laid out clearly there.

Ultimately we build our data from ticks which is much more accurate than Yahoo or Google's daily sources. In live trading you can't filter out ticks after the fact - you don't know what will be marked as invalid.

Ticks arrive late, or are invalid prices far from market price. When you request tick data we expect you to understand that. If you use second or minute bars we pre-filter the common causes of price error as shown in above discussion.

Also note the terms and conditions prohibit logging price data due to data vendor restrictions. If you'd like to download the raw data you can purchase it from QuantQuote.com

Qingpeng Niu INVESTOR

Update Backtest

Notebook

person upvoted this people upvoted this

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Organization

Organization Website

Update Competition

Team

Clone Strategy

Copy this strategy code to your QuantConnect account and deploy it live with your brokerage.

Clone

Previous Ranking

Browse strategies and organization entries from previous quarter's competitions.

Author:

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

TOP 5 Research PUblications

About Quant League

competition rules

previous competitions

284,500 Quants.

VOTE FOR UPCOMING FEATURES

Simulation price is different with price on Google and Yahoo finance

Organization

Team

Clone Strategy

Previous Ranking

IN THIS RESEARCH

PARTICIPANTS

Discussion Awards

Actions

Join QuantConnect for Free

Platform

SIGN IN

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

TOP 5 Research PUblications

About Quant League

competition rules

previous competitions

284,500 Quants.

VOTE FOR UPCOMING FEATURES

Simulation price is different with price on Google and Yahoo finance

Organization

Team

Clone Strategy

Previous Ranking

IN THIS RESEARCH

PARTICIPANTS

Discussion Awards

SHARE RESEARCH

SHARE DISCUSSION

SHARE ARTICLE

SHARE

Actions

Join QuantConnect for Free