Insert object at beginning of RollingWindowT by Mike Ellertson

Mike Ellertson

| Reward Discussion

I see there's an Add() method in the RollingWindow class. It adds a new item to the end of the list.

I'd like to insert a new item at the beginning of the list, but the ivar _list is private inside of RollingWindow. I've been scouring the internet trying to find a way to accomplish this. It seems like creating a static extension method might be a good way. But, so far I can't seem to get it work.

The reason I'm trying to do this is, I'm getting some data for a custom indicator from Yahoo Finance. Yahoo finance seems to only provide the data sorted by date. I need to reverse the sort order of the data. I think it provides it in ascending and I need to sort it in descending order (although I might have the ascending and descending backwards, I'm getting a bit sleepy eyed at the moment).

Here's the URL I'm using to get the CSV data from Yahoo Finance:
http://ichart.yahoo.com/table.csv?s=GOOG&a=0&b=2&c=2004&g=d&ignore=.csv

Does anyone know how I could insert a new item to the beginning of a RollingWindow object? Or perhaps just get the data sorted from Yahoo Finance in the order I specify? My end goal is to simply reverse the sort order of the RollingWindow after it completely loaded.

Update Backtest

Mike Ellertson

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Nicholas Stein

4.3k ,

Hey Mike,

I can understand why you might want to enter an item at the beginning of the List. I also spent quite a bit of time staring at the List. However, let me suggest a different way of thinking about it.

tl;dr
I suggest you forget about the underlying List in RollingWindow and just use it the way it was intended. Put your parsed lines into a List or List, sort it and put the result into the RollingWindow.

Discussion
The purpose of the RollingWindow is to be able to refer to the elements of the Window the way they do in other languages such as is EasyScript. In EasyScript the first element is RollingWindowName and the second element is RollingWindowName[1] and it means the value one bar ago. Likewise [2] is the value two bars ago and so forth. So the RollingWindow works like a stack where each .Add pushes each of the elements in the Window down. Michael did not implemented it as a stack, but that is how it works. Rather than try to change the character of a RollingWindow, why not just use it the way it was intended. Adding things to the indicator pushes the rest of the elements down one place.

If you need to change the first element of the RollingWindow then

RollingWindowName[0] = somedecimalvariable;

will do the trick. I often use this technique. Add a dummy value to the RollingWindow then use the other values to compute something and then change the first value. It is not very elegant, but it works.

Using your objective and data source you might want to think about putting the bars into a regular C# List and then use Linq to sort or select before you add them to the RollingWindow.

So let's say you get back a Stream object as a result of your WebClient call to get the data from the URI, and you parse for lines with a strng line = result.Split('\n') and each column in the line with a string columns = line.Split(',') so that you have an array for each day. You could then spin up a new TradeBar and assign the array elements to the object.



private TradeBar stringToTradeBar( string[] columns )

{

   TradeBar bar = new TradeBar();

    bar.Date = DateTime.TryParse(columns[0], out bar.Date);

    bar.Open =  System.Convert.ToIntDecimal(columns[1]);

    ...

    bar.Volume - System.Convert.ToInt32(columns[5]);

    return bar;

}

Then you can add the TradeBar to a List with a



      var tradeBarList = new List();

      foreach (string line in result)

      {   

          string[] columns = line.Split(',');

          tradeBarList.Add(stringToTradeBar(columns);

      }

Then you can simply sort the resulting list using Linq and add the TradeBars to the RollingWindow.
as suggested here



    RollingWindow rw = new RollingWindow(length);

    // verbose for clarity

    var sortedList = tradeBarList.OrderBy(d => d.Date);

    foreach (var tradeBar in sortedList)

    {

        rw.Add(tradeBar);

    }

I hope this helps.

Nick

Jared Broad

Great answer Nick -

Mike - it might help to understand why you need a CSV list of Yahoo daily data in your algorithm? Its definitely possible but I would recommend using the existing OnData infrastructure if you're running some kind of analysis (it will prevent forward look ahead bias).

Michael Handschuh

STAFF ,

Hey Mike, you may benefit from the QCU How do I import Yahoo data example. I've attached it here for your convenience. This example shows how you can use custom data types to access Yahoo finance data. For a more advanced example, check out QCU How Do I Import Quandl Data.

Mike Ellertson

1.6k ,

Firstly, thanks for the all the suggestions!

The long and short of it is, the code presently works as a class derived from BaseData. I didn't develop all of the code, so I can't speak to why it's that way. But the developer I'm working with is very good, so I trust he has good reasons. :-)

I worked for several hours attempting to re-write some of the code to use the Quandl data, but I had a hard time figuring out how to get the data I need. The two funds I need historical data for are VIX and VXV. To make a little bit more interesting, on Yahoo, the symbols are "^VIX" and "^VXV" respectively.

When I looked on Quandl I found many funds matching "VIX" and "VXV". So I was trying for the simplest solution that I know would give me the correct data, at around midnight to boot. :-)

@MichaelH, I cloned the code you posted, it seems to do the trick nicely. So thanks for that!

The only thing I'm still having trouble with is how to get the "VXV" data from Quandl. Quandl does have the data, I just haven't figured out how to access it from QuantConnect. Here's the data I'm seeing on Quandl:
https://www.quandl.com/data/CBOE/VXV

I tried using the following prefixes in the GetSource(...) method of Yahoo.cs, neither seemed to work.
- https://www.quandl.com/api/v1/datasets/YAHOO/INDEX_VXV/...
- https://www.quandl.com/api/v1/datasets/CBOE/VXV/...

I searched through the published list of symbole (found here: https://quantquote.com/docs/symbol_map_comnam.csv), but couldn't find "VXV".

So while MichaelH's code helped a lot, I'm still stumped. :-)

Mike Ellertson

1.6k ,

Ok... Murphy's law. A few moments after I posted my reply, I figured out my problem. :-)

Turns out https://www.quandl.com/api/v1/datasets/CBOE/VXV/... is the correct way to access VXV data from Quandl.

I had made another silly mistake in the Yahoo class, which is derived from BaseData. With that fixed, it works like a charm.

Thanks so much for the help!!!

Mike Ellertson

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!