Back

History() using Tiingo Daily Data?

I'm interested in using the History() method to ingest a block of data from Tiingo. I'm using SPY just as test because it's available from both QC data and Tiingo. History() works fine, if I use:

self.AddEquity(self.ticker, Resolution.Daily)
h = self.History([self.ticker], TimeSpan.FromDays(5), Resolution.Daily)

However, if I write:

self.AddData(TiingoDailyData, self.ticker, Resolution.Daily)
h = self.History([self.ticker], TimeSpan.FromDays(5), Resolution.Daily)

I get the following error:

at Initialize in main.py:line 31
Exception : cannot handle a non-unique multi-index!

I was unable to figure out from the documentation how to use History() with TiingoDailyData (or other custom data source). Perhaps there is another way to retrieve blocks of historical data?

Update Backtest







Being pretty new at working with this code, I'm not so familiar with the internals. After some probing, it seems clear that the interface for Tiingo data does not support History(). In order to support algorithms that require historical lookback, I wrote the following class that agreggates price data from slices. Not beautiful or necessarily efficient, but it works. If you like it, feel free to improve it and post back here.

 

class DataAggregator:
'''
A class for aggregating one field from a slice from external data source
It collect a rolling window of n points
tickers: List of tickers to extract
field: String defining which field to extract
n: number of points to keep
'''
def __init__(self, tickers, field, n):
self.tickers = tickers
self.field = field
self.n = n
self.i = 0
self.data= pd.DataFrame(index=range(self.n), columns=self.tickers)

def agg(self, slice):
if (self.i < self.n):
for t in self.tickers:
self.data.iloc[self.i][t] = getattr(slice[t], self.field)
self.i += 1
else:
self.data = self.data.shift(-1,)
for t in self.tickers:
self.data.iloc[-1][t] = getattr(slice[t], self.field)

 

To use, put something like this in your Initialize() method:

self.tickers = ['SPY', 'TLT']
self.history = DataAggregator(self.tickers, 'AdjustedClose', 200)

 Then, in OnData(), put:

self.history.agg(slice)

I did not try to replicate the full functionality of History(), but this simple hack should serve for most simple use cases.

Greg

0

Quick update: I cleaned this up a bit. I looked at using RollingWindow(), but it seemed overly complicated and I did not like it's asyncronous approach. I prefer using pd.DataFrame(), so I stuck with that. The code below is simpler than above and now includes a DatetimeIndex rather than in an integer index. Only tested (so far) on Tiingo dailly data.

class DataAggregator:
'''
A class for aggregating one field from a slice from external data source
It collects a rolling window of n points
tickers: List of tickers to extract
field: String defining which field to extract, eg. 'AdjustedClose'
n: number of points to keep
'''
def __init__(self, tickers, field, n):
self.tickers = tickers
self.field = field
self.n = n
self.data = pd.DataFrame()

def agg(self, slice):

for t in self.tickers:
self.data.loc[slice.Time, t] = getattr(slice[t], self.field)
if len(self.data) > self.n:
self.data.drop(self.data.index[0], inplace=True)

  

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed