Back

How to use only trade data in LEAN (Crypto)?

I'm trying to run Pairs trading strategy for BTCUSD and XMRUSD. Since data is not freely available, I have re-formatted my existing data (manually imported from API) to LEAN format, according to this - https://www.quantconnect.com/lean/documentation/topic16.html

But, I have only trade data, no quote. So, when I run my algo, it's throwing errors about missing "_quote" data:

20190426 09:27:12.450 ERROR:: DefaultDataProvider.Fetch(): The specified file was not found: ../../../Data/crypto/bitfinex/minute/xmrusd/20190201_quote.zip

To me it seems like this should be runnable with trade data only. Or am I wrong? Is there something I need to configure explicitly in my strategy for it to not look for quote data?

from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Algorithm")
AddReference("QuantConnect.Indicators")
AddReference("QuantConnect.Common")

from System import *
from QuantConnect import *
from QuantConnect.Algorithm import *
from QuantConnect.Brokerages import *
from QuantConnect.Data import BaseData
from QuantConnect.Data.Market import *
from QuantConnect.Securities import *
from QuantConnect.Algorithm import *
from QuantConnect.Indicators import *

from sklearn import linear_model
import numpy as np
import pandas as pd
from scipy import stats
from math import floor
from datetime import timedelta


class Pairs1(QCAlgorithm):

def Initialize(self):

self.SetStartDate(2019,1,1)
self.SetEndDate(2019,1,31)
self.SetCash(10000)
self.numdays = 7 # set the length of training period

self.symbols = []

self.threshold = 1.

self.AddCrypto("BTCUSD", Resolution.Minute, Market.Bitfinex);
self.AddCrypto("XMRUSD", Resolution.Minute, Market.Bitfinex);

for i in self.symbols:
i.hist_window = RollingWindow[TradeBar](self.numdays)


def OnData(self, data):

if not (data.ContainsKey("BTCUSD") and data.ContainsKey("XMRUSD")): return

for symbol in self.symbols:
symbol.hist_window.Add(data[symbol])


price_x = pd.Series([float(i.Close) for i in self.symbols[0].hist_window],
index = [i.Time for i in self.symbols[0].hist_window])

price_y = pd.Series([float(i.Close) for i in self.symbols[1].hist_window],
index = [i.Time for i in self.symbols[1].hist_window])
if len(price_x) < 250: return
spread = self.regr(np.log(price_x), np.log(price_y))
mean = np.mean(spread)
std = np.std(spread)
ratio = floor(self.Portfolio[self.symbols[1]].Price / self.Portfolio[self.symbols[0]].Price)
# quantity = float(self.CalculateOrderQuantity(self.symbols[0],0.4))

if spread[-1] > mean + self.threshold * std:
if not self.Portfolio[self.symbols[0]].Quantity > 0 and not self.Portfolio[self.symbols[0]].Quantity < 0:
self.Sell(self.symbols[1], 100)
self.Buy(self.symbols[0], ratio * 100)

elif spread[-1] < mean - self.threshold * std:
if not self.Portfolio[self.symbols[0]].Quantity < 0 and not self.Portfolio[self.symbols[0]].Quantity > 0:
self.Sell(self.symbols[0], 100)
self.Buy(self.symbols[1], ratio * 100)

else:
self.Liquidate()


def regr(self,x,y):
regr = linear_model.LinearRegression()
x_constant = np.column_stack([np.ones(len(x)), x])
regr.fit(x_constant, y)
beta = regr.coef_[0]
alpha = regr.intercept_
spread = y - x*beta - alpha
return spread

 

Update Backtest







0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


EDIT: fixed type in code, below is the actual one.

With all the errors, code seems to run and provide results/statistics at the end, does  that mean I can just ignore the errors? Does LEAN detects missing quote data and simply ignores that and uses trade only?

from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Algorithm")
AddReference("QuantConnect.Indicators")
AddReference("QuantConnect.Common")

from System import *
from QuantConnect import *
from QuantConnect.Algorithm import *
from QuantConnect.Brokerages import *
from QuantConnect.Data import BaseData
from QuantConnect.Data.Market import *
from QuantConnect.Securities import *
from QuantConnect.Algorithm import *
from QuantConnect.Indicators import *

from sklearn import linear_model
import numpy as np
import pandas as pd
from scipy import stats
from math import floor
from datetime import timedelta


class Pairs1(QCAlgorithm):

def Initialize(self):

self.SetStartDate(2019,1,1)
self.SetEndDate(2019,1,31)
self.SetCash(10000)
self.numdays = 7 # set the length of training period

self.symbols = []

self.threshold = 1.

btc = self.AddCrypto("BTCUSD", Resolution.Minute, Market.Bitfinex).Symbol
self.symbols.append(btc)

xmr = self.AddCrypto("XMRUSD", Resolution.Minute, Market.Bitfinex).Symbol
self.symbols.append(xmr)

for i in self.symbols:
i.hist_window = RollingWindow[TradeBar](self.numdays)


def OnData(self, data):

if not (data.ContainsKey("BTCUSD") and data.ContainsKey("XMRUSD")): return

for symbol in self.symbols:
symbol.hist_window.Add(data[symbol])


price_x = pd.Series([float(i.Close) for i in self.symbols[0].hist_window],
index = [i.Time for i in self.symbols[0].hist_window])

price_y = pd.Series([float(i.Close) for i in self.symbols[1].hist_window],
index = [i.Time for i in self.symbols[1].hist_window])
if len(price_x) < 250: return
spread = self.regr(np.log(price_x), np.log(price_y))
mean = np.mean(spread)
std = np.std(spread)
ratio = floor(self.Portfolio[self.symbols[1]].Price / self.Portfolio[self.symbols[0]].Price)
# quantity = float(self.CalculateOrderQuantity(self.symbols[0],0.4))

if spread[-1] > mean + self.threshold * std:
if not self.Portfolio[self.symbols[0]].Quantity > 0 and not self.Portfolio[self.symbols[0]].Quantity < 0:
self.Sell(self.symbols[1], 100)
self.Buy(self.symbols[0], ratio * 100)

elif spread[-1] < mean - self.threshold * std:
if not self.Portfolio[self.symbols[0]].Quantity < 0 and not self.Portfolio[self.symbols[0]].Quantity > 0:
self.Sell(self.symbols[0], 100)
self.Buy(self.symbols[1], ratio * 100)

else:
self.Liquidate()


def regr(self,x,y):
regr = linear_model.LinearRegression()
x_constant = np.column_stack([np.ones(len(x)), x])
regr.fit(x_constant, y)
beta = regr.coef_[0]
alpha = regr.intercept_
spread = y - x*beta - alpha
return spread

 

0

Hi Deandree,

Here is a workaround: in Lean/Common/Data/SubscriptionManager.cs : line 213 , delete 

, TickType.Quote

Then rebuild and your algorithm should not look for quote data for crypto type securities any more.

Hope it helps!

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed