[question] How to get historical data in several timeframes? by Ff fly

Ff fly

| Reward Discussion

Hi,

I'm new to QC. I'm trying to implement a strategy in QC. In the strategy I use several timeframes at same time to determine a signal. For example, I may use 30m timeframe to generate a candidate signal, then use 10m, 1h, 4h, and daily timeframes to confirm the signal. In each timeframe I may use different number of bars.

my question: when I have a new bar in 30m timeframe, how can I get last 100 bars in 10m timeframe, 50 bars in 1h, 30 bars in 4h, 20 bars in daily timeframe?

Thanks.

Update Backtest

Ff fly

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Andrestone

220 ,

Maybe you should consolidate custom bars and set event handlers for each one of those as described in here:

https://www.quantconnect.com/docs#Consolidating-Data

Alexandre Catarino

To answer this question, we need to use two features: Consolidating Data and RollingWindow. The consolidators will create bars with the desired period that will be saved in a rolling window, so we don't need to make history requests every 30 min:

private Dictionary<string, TradeBarConsolidator> _consolidators;

private Dictionary<string, RollingWindow<TradeBar>> _history;



public override void Initialize()

{

    SetStartDate(2017, 1, 1);  //Set Start Date

    AddEquity("SPY", Resolution.Minute);



    // Create a dictionary with the consolidators.

    // It will be easier to refer to them by the key

    _consolidators = new Dictionary<string, TradeBarConsolidator>

    {

        { "10m",  new TradeBarConsolidator(TimeSpan.FromMinutes(10)) },

        { "30m",  new TradeBarConsolidator(TimeSpan.FromMinutes(30)) },

        { "1h",  new TradeBarConsolidator(TimeSpan.FromHours(1)) },

        { "4h",  new TradeBarConsolidator(TimeSpan.FromHours(4)) },

        { "1d",  new TradeBarConsolidator(TimeSpan.FromDays(1)) }

    };

    

    // Create a dictionary with the RollingWindow

    _history = new Dictionary<string, RollingWindow<TradeBar>>

    {

        { "10m",  new RollingWindow<TradeBar>(100) },

        { "30m",  new RollingWindow<TradeBar>(33) },

        { "1h",  new RollingWindow<TradeBar>(50) },

        { "4h",  new RollingWindow<TradeBar>(30) },

        { "1d",  new RollingWindow<TradeBar>(20) }

    };



    // For each consolidator, we assign anonymous event handler that 

    // will update the rolling window

    foreach (var kvp in _consolidators)

    {

        _consolidators[kvp.Key].DataConsolidated += (s, e) => _history[kvp.Key].Add(e);

        SubscriptionManager.AddConsolidator("SPY", kvp.Value);

    }



    // Make a history request and update the consolidators

    foreach (var data in History("SPY", TimeSpan.FromDays(30)))

    {

         foreach (var consolidator in _consolidators.Values)

         {

             consolidator.Update(data);

         }

    }



    // Assign an event handler to the 30min consolidator

    _consolidators["30m"].DataConsolidated += On30mData;

}

If we want to access the close price 22 bars ago of the 10-min timeframe:

var value = _history["10m"][22].Close;

Andrestone

220 ,

How could I do the same as above for several securities?

Andrestone

220 ,

I came up with a very rough solution, I created my own bars.

If anyone is interested:

            //Adding pairs at base resolution 

            foreach (var pair in _currencyPairs)

            {

                AddForex(pair, Resolution.Minute);

                EMAFast[pair] = new Dictionary<string, ExponentialMovingAverage>();

                EMASlow[pair] = new Dictionary<string, ExponentialMovingAverage>();

                M[pair] = new Dictionary<string, Momentum>();

            }



            //*********************************

            //START OF MYBARS GENERATION LOGIC

            //**********************************

            foreach (var pair in _currencyPairs)

            {

                var quoteBars = History<QuoteBar>(pair, TimeSpan.FromDays(30));





                foreach (var interval in _Intervals)

                {





                    EMAFast[pair][interval.Key] = new ExponentialMovingAverage(9);

                    EMASlow[pair][interval.Key] = new ExponentialMovingAverage(90);

                    M[pair][interval.Key] = new Momentum(9);



                    int minutes = 0;

                    List<decimal> AskOpen = new List<decimal>();

                    List<decimal> AskHigh = new List<decimal>();

                    List<decimal> AskLow = new List<decimal>();

                    List<decimal> AskClose = new List<decimal>();

                    List<decimal> BidOpen = new List<decimal>();

                    List<decimal> BidHigh = new List<decimal>();

                    List<decimal> BidLow = new List<decimal>();

                    List<decimal> BidClose = new List<decimal>();



                    foreach (var data in quoteBars)

                    {



                        minutes++;



                        AskOpen.Add(data.Ask.Open);

                        AskHigh.Add(data.Ask.High);

                        AskLow.Add(data.Ask.Low);

                        AskClose.Add(data.Ask.Close);

                        BidOpen.Add(data.Bid.Open);

                        BidHigh.Add(data.Bid.High);

                        BidLow.Add(data.Bid.Low);

                        BidClose.Add(data.Bid.Close);



                        if (minutes == interval.Value)

                        {



                            //Why C# doesn't allow me to simply mydict[key1][key2][key3][keyN] = "my value", since it knows the data structure already?

                            if (!_myBars.ContainsKey(data.EndTime))

                            {

                                _myBars[data.EndTime] = new Dictionary<string, Dictionary<string, Dictionary<string, Dictionary<string, decimal>>>>();

                            }

                            if (!_myBars[data.EndTime].ContainsKey(pair))

                            {

                                _myBars[data.EndTime][pair] = new Dictionary<string, Dictionary<string, Dictionary<string, decimal>>>();

                            }

                            if (!_myBars[data.EndTime][pair].ContainsKey(interval.Key))

                            {



                                //Updating indicators

                                EMAFast[pair][interval.Key].Update(data.EndTime, AskClose[AskClose.Count - 1]); 

                                EMASlow[pair][interval.Key].Update(data.EndTime, AskClose[AskClose.Count - 1]); 

                                M[pair][interval.Key].Update(data.EndTime, AskClose[AskClose.Count - 1]);





                                //Adding each bar

                                _myBars[data.EndTime][pair][interval.Key] = new Dictionary<string, Dictionary<string, decimal>>()

                                {

                                    { "Ask", new Dictionary<string, decimal>() {

                                            { "Open", AskOpen[0] },

                                            { "High", AskHigh.Max() },

                                            { "Low", AskLow.Min() },

                                            { "Close", AskClose[AskClose.Count - 1] },

                                            { "EMAFast", EMAFast[pair][interval.Key].IsReady ? EMAFast[pair][interval.Key] : AskClose[AskClose.Count - 1] },

                                            { "EMASlow", EMASlow[pair][interval.Key].IsReady ? EMASlow[pair][interval.Key] : AskClose[AskClose.Count - 1] },

                                            { "M", M[pair][interval.Key].IsReady ? M[pair][interval.Key] : 0m },

                                        }

                                    },

                                    { "Bid", new Dictionary<string, decimal>() {

                                            { "Open", BidOpen[0] },

                                            { "High", BidHigh.Max() },

                                            { "Low", BidLow.Min() },

                                            { "Close", BidClose[BidClose.Count - 1] },

                                            { "EMAFast", 0m },

                                            { "EMASlow", 0m },

                                            { "M", 0m },

                                        }

                                    }

                                };

                            };



                            

                            //Resetting the LOOP

                            AskOpen.Clear();

                            AskHigh.Clear();

                            AskLow.Clear();

                            AskClose.Clear();

                            BidOpen.Clear();

                            BidHigh.Clear();

                            BidLow.Clear();

                            BidClose.Clear();

                            minutes = 0;



                        }



                    }

                }

            }

            //*********************************

            //END OF MYBARS GENERATION LOGIC

            //**********************************

PS: Alexandre tought me a much more elegant way, which is sending the instance SubscriptionManager as an argumento to another method so it could assume different values within the scope of each RollingWindow.

KV1214

1.1k ,

1. How would you do this part in Python?

2. What is the foramt of the data which is being passed into the Update method?

foreach (var data in History("SPY", TimeSpan.FromDays(30))) { foreach (var consolidator in _consolidators.Values) { consolidator.Update(data); } }

Thank you!

Shile Wen

STAFF ,

Hi KV1214,
Please see this thread on how to do this.

Best,
Shile Wen

Ff fly

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!