Importing Custom Data by Akhil patel

Akhil patel

| Reward Discussion

Hi is it possible to import trade and quote data, that will keep the message structure intact, I use Reuters Tick History but need to stream it as trade bid or quote update with multiple data points at the same microsecond. So I dont want to split the data into seperate trade or bidask.

Update Backtest

Akhil patel

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Alexandre Catarino

Could you give us more details?
What is the data structure?

Akhil patel

19 ,

Can I give you a sample? Would that be easier.

so here we have csv format, with eurex fgbl mar 16 emdi updates.

datetime,tradeprice,tradesize,bidprice,bidsize,askprice,asksize

2016-01-25 10:48:46.777566,NaN,NaN,161.47,190,NaN,NaN
2016-01-25 10:48:47.117338,NaN,NaN,161.47,256,161.48,40
2016-01-25 10:48:47.466905,NaN,NaN,161.47,253,161.48,43
2016-01-25 10:48:47.796992,NaN,NaN,161.47,207,161.48,94
2016-01-25 10:48:51.716714,NaN,NaN,161.47,205,NaN,NaN
2016-01-25 10:48:51.926685,NaN,NaN,NaN,NaN,161.48,94
2016-01-25 10:48:51.926685,161.48,15,NaN,NaN,NaN,NaN
2016-01-25 10:48:52.266630,NaN,NaN,161.47,249,161.48,28
2016-01-25 10:48:52.305743,NaN,NaN,161.47,252,161.48,13
2016-01-25 10:48:52.305743,161.48,13,NaN,NaN,NaN,NaN
2016-01-25 10:48:52.636735,NaN,NaN,161.47,264,161.48,1
2016-01-25 10:48:52.686503,161.48,1,NaN,NaN,NaN,NaN
2016-01-25 10:48:52.706968,NaN,NaN,161.48,10,161.49,302
2016-01-25 10:48:52.706968,161.48,2,NaN,NaN,NaN,NaN
2016-01-25 10:48:52.706968,NaN,NaN,161.48,3,161.49,301
2016-01-25 10:48:52.706968,161.48,3,NaN,NaN,NaN,NaN
2016-01-25 10:48:52.907173,NaN,NaN,161.48,2,161.49,288
2016-01-25 10:48:52.907173,161.48,1,NaN,NaN,NaN,NaN
2016-01-25 10:48:53.246988,NaN,NaN,161.47,299,161.49,293
2016-01-25 10:48:53.417005,NaN,NaN,161.47,285,161.48,1
2016-01-25 10:48:53.426608,161.48,1,NaN,NaN,NaN,NaN
2016-01-25 10:48:53.756711,NaN,NaN,161.47,304,161.49,319
2016-01-25 10:48:54.136616,NaN,NaN,161.47,297,NaN,NaN
2016-01-25 10:48:54.907041,NaN,NaN,161.48,6,161.49,296
2016-01-25 10:48:54.907041,161.48,6,NaN,NaN,NaN,NaN
2016-01-25 10:48:55.246839,NaN,NaN,161.48,3,161.49,296
2016-01-25 10:48:55.586655,NaN,NaN,161.48,19,161.49,295
2016-01-25 10:48:56.086649,NaN,NaN,161.48,20,NaN,NaN
2016-01-25 10:48:56.436275,NaN,NaN,161.48,106,161.49,269
2016-01-25 10:48:56.527094,161.48,75,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.527094,NaN,NaN,161.48,29,161.49,300
2016-01-25 10:48:56.527094,161.48,7,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.527094,NaN,NaN,161.48,22,161.49,303
2016-01-25 10:48:56.536805,161.48,1,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.536805,NaN,NaN,161.48,21,161.49,304
2016-01-25 10:48:56.536805,161.48,21,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.536805,NaN,NaN,161.47,264,161.48,32
2016-01-25 10:48:56.536805,161.48,3,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.607224,NaN,NaN,161.47,246,161.48,3
2016-01-25 10:48:56.607224,161.48,3,NaN,NaN,NaN,NaN
2016-01-25 10:48:56.946949,NaN,NaN,161.48,8,161.49,335
2016-01-25 10:48:57.846354,NaN,NaN,161.48,10,NaN,NaN
2016-01-25 10:48:59.056279,NaN,NaN,161.48,11,NaN,NaN
2016-01-25 10:48:59.536665,NaN,NaN,NaN,NaN,161.49,336
2016-01-25 10:49:00.156964,161.48,1,NaN,NaN,NaN,NaN
2016-01-25 10:49:00.496611,NaN,NaN,161.48,10,NaN,NaN
2016-01-25 10:49:00.836415,NaN,NaN,161.48,92,161.49,278
2016-01-25 10:49:01.176334,NaN,NaN,161.48,90,NaN,NaN
2016-01-25 10:49:02.185997,NaN,NaN,161.48,88,NaN,NaN
2016-01-25 10:49:02.526792,NaN,NaN,161.48,26,161.49,289
2016-01-25 10:49:02.856935,NaN,NaN,161.48,116,161.49,238
2016-01-25 10:49:03.196667,NaN,NaN,161.48,99,161.49,232
2016-01-25 10:49:03.546252,NaN,NaN,NaN,NaN,161.49,239
2016-01-25 10:49:03.886120,NaN,NaN,NaN,NaN,161.49,220
2016-01-25 10:49:04.016096,NaN,NaN,161.48,108,NaN,NaN
2016-01-25 10:49:04.016096,161.48,10,NaN,NaN,NaN,NaN
2016-01-25 10:49:04.366569,NaN,NaN,161.48,97,161.49,217
2016-01-25 10:49:05.026691,NaN,NaN,NaN,NaN,161.49,215
2016-01-25 10:49:05.366505,NaN,NaN,NaN,NaN,161.49,212
2016-01-25 10:49:06.055896,NaN,NaN,161.48,133,161.49,156
2016-01-25 10:49:06.385933,NaN,NaN,161.48,130,161.49,156
2016-01-25 10:49:07.106665,161.49,1,NaN,NaN,NaN,NaN
2016-01-25 10:49:07.106665,NaN,NaN,NaN,NaN,161.49,155
2016-01-25 10:49:07.106665,161.49,1,NaN,NaN,NaN,NaN
2016-01-25 10:49:07.106665,NaN,NaN,161.48,156,161.49,113
2016-01-25 10:49:07.106665,161.49,113,NaN,NaN,NaN,NaN
2016-01-25 10:49:07.106761,NaN,NaN,161.49,26,161.5,714

Alexandre Catarino

Please checkout the docs, under Importing Custom Data:

// In Initialize method:

AddData<Weather>("KNYC", Resolution.Daily);

Here is how a custom class look like:

public class Weather : BaseData

{

    public decimal MaxC = 0;

    public decimal MinC = 0;

    public string errString = "";



    public override SubscriptionDataSource GetSource(

        SubscriptionDataConfig config, 

        DateTime date, 

        bool isLive)

    {

        var source = string.Format(

                "https://www.wunderground.com/history/airport/{0}/{1}/1/1/" + 

                "CustomHistory.html?dayend=31&monthend=12&yearend={1}&format=1",

                config.Symbol, date.Year);



          return new SubscriptionDataSource(source,

              SubscriptionTransportMedium.RemoteFile);

    }



    public override BaseData Reader(

        SubscriptionDataConfig config,

        string line,

        DateTime date,

        bool isLive)

    {

        if (string.IsNullOrWhiteSpace(line) ||

            char.IsLetter(line[0])) 

            return null;

            

        var data = line.Split(',');



        return new Weather()

        {

            // Make sure we only get this data AFTER trading day - don't want forward bias.

            Time = DateTime.Parse(data[0]).AddHours(20), 

            Symbol = config.Symbol,

            MaxC = Convert.ToDecimal(data[1]),

            Value = Convert.ToDecimal(data[2]),

            MinC = Convert.ToDecimal(data[3]),

        };

    }

}

In your case, you need to have members called BidPrice, BidSize, AskPrice, AskSize, TradePrice and TradeSize.
Value should be TradePrice.

Akhil patel

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

204,390 Quants.

Online Now

Stay Connected