Back

Not all custom data being passed into OnData

Hi,

I've been having problems getting custom data into my own algo so I went back to the docuemented example (CustomDataBitcoinAlgorithm.py) and made a small change that illustrates the problem. I'm curious if this is a bug in QC or if I'm doing something wrong...

Using the CustomDataBitcoinAlgorithm.py example, all i'm trying to do is access one of the other fields from the custom data (VolumeBTC instead of Close). When I do, I get this runtime error:


Runtime Error: AttributeError : 'Data' object has no attribute 'VolumeBTC'
at OnData in main.py:line 52
AttributeError : 'Data' object has no attribute 'VolumeBTC' (Open Stacktrace)

Instead of attempting to access:

data["BTC"].Close

I'm attempting to access:

data["BTC"].VolumeBTC

Any idea what's going on?

Thanks!

BTW, here's the entire algo (with the only real change on line 52:

# QUANTCONNECT.COM - Democratizing Finance, Empowering Individuals.
# Lean Algorithmic Trading Engine v2.0. Copyright 2014 QuantConnect Corporation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Algorithm")
AddReference("QuantConnect.Common")

from System import *
from QuantConnect import *
from QuantConnect.Algorithm import *
from QuantConnect.Data import SubscriptionDataSource
from QuantConnect.Python import PythonData

from datetime import date, timedelta, datetime
import decimal
import numpy as np
import json

### <summary>
### Demonstration of using an external custom datasource. LEAN Engine is incredibly flexible and allows you to define your own data source.
### This includes any data source which has a TIME and VALUE. These are the *only* requirements. To demonstrate this we're loading in "Bitcoin" data.
### </summary>
### <meta name="tag" content="using data" />
### <meta name="tag" content="custom data" />
### <meta name="tag" content="crypto" />
class CustomDataBitcoinAlgorithm(QCAlgorithm):

def Initialize(self):
self.SetStartDate(2011, 9, 13)
self.SetEndDate(datetime.now().date() - timedelta(1))
self.SetCash(100000)

# Define the symbol and "type" of our generic data:
self.AddData(Bitcoin, "BTC")


def OnData(self, data):
if not data.ContainsKey("BTC"): return

close = data["BTC"].Close
volumeBTC = data["BTC"].VolumeBTC
self.Debug("volumeBTC: {0} {1}".format(datetime.now(), volumeBTC))

# If we don't have any weather "SHARES" -- invest"
if not self.Portfolio.Invested:
# Weather used as a tradable asset, like stocks, futures etc.
self.SetHoldings("BTC", 1)
self.Debug("Buying BTC 'Shares': BTC: {0}".format(close))

#self.Debug("Time: {0} {1}".format(datetime.now(), close))


class Bitcoin(PythonData):
'''Custom Data Type: Bitcoin data from Quandl - http://www.quandl.com/help/api-for-bitcoin-data'''

def GetSource(self, config, date, isLiveMode):
if isLiveMode:
return SubscriptionDataSource("https://www.bitstamp.net/api/ticker/", SubscriptionTransportMedium.Rest);

#return "http://my-ftp-server.com/futures-data-" + date.ToString("Ymd") + ".zip";
# OR simply return a fixed small data file. Large files will slow down your backtest
return SubscriptionDataSource("http://www.quandl.com/api/v1/datasets/BCHARTS/BITSTAMPUSD.csv?sort_order=asc", SubscriptionTransportMedium.RemoteFile);


def Reader(self, config, line, date, isLiveMode):
coin = Bitcoin()
coin.Symbol = config.Symbol

if isLiveMode:
# Example Line Format:
# {"high": "441.00", "last": "421.86", "timestamp": "1411606877", "bid": "421.96", "vwap": "428.58", "volume": "14120.40683975", "low": "418.83", "ask": "421.99"}
try:
liveBTC = json.loads(line)

# If value is zero, return None
value = decimal.Decimal(liveBTC["last"])
if value == 0: return None

coin.Time = datetime.now()
coin.Value = value
coin["Open"] = float(liveBTC["open"])
coin["High"] = float(liveBTC["high"])
coin["Low"] = float(liveBTC["low"])
coin["Close"] = float(liveBTC["last"])
coin["Ask"] = float(liveBTC["ask"])
coin["Bid"] = float(liveBTC["bid"])
coin["VolumeBTC"] = float(liveBTC["volume"])
coin["WeightedPrice"] = float(liveBTC["vwap"])
return coin
except ValueError:
# Do nothing, possible error in json decoding
return None

# Example Line Format:
# Date Open High Low Close Volume (BTC) Volume (Currency) Weighted Price
# 2011-09-13 5.8 6.0 5.65 5.97 58.37138238, 346.0973893944 5.929230648356
if not (line.strip() and line[0].isdigit()): return None

try:
data = line.split(',')

# If value is zero, return None
value = decimal.Decimal(data[4])
if value == 0: return None

coin.Time = datetime.strptime(data[0], "%Y-%m-%d")
coin.Value = value
coin["Open"] = float(data[1])
coin["High"] = float(data[2])
coin["Low"] = float(data[3])
coin["Close"] = float(data[4])
coin["VolumeBTC"] = float(data[5])
coin["VolumeUSD"] = float(data[6])
coin["WeightedPrice"] = float(data[7])
return coin;

except ValueError:
# Do nothing, possible error in json decoding
return None
Update Backtest








please attach the whole backtest if you can next time ...  :)

0

i will try to help you later:


Backtest Handled Error: Error downloading custom data source file, skipped: RemoteFile: Csv http://www.quandl.com/api/v1/datasets/BCHARTS/BITSTAMPUSD.csv?sort_order=asc Error: The remote server returned an error: (429) Too Many Requests. (Open Stacktrace)

 

too many requests

0

H Michael,

Attached is the backtest. You have uncomment line 56 to get the error.

Thanks!

0


Hi, thanks

i always get the same message: The remote server returned an error: (429) Too Many Requests

I wanted to try to output the class structure with log or print (self.print/self.Log) whatever works.

it would show maybe the fields of this data["BTC"] class. it seems it does not have this VolumeBTC member you are searching for. but  i cant prove it.

i would start with a clone of a already postet bitcoin strategy from the community or take the basic example and go with that

https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/BasicTemplateCryptoAlgorithm.py
0

Hi,

I did clone the example (but a different one) from here:

https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/CustomDataBitcoinAlgorithm.py

You can get throttled if you run the algo too often in backtesting. I'm not sure  what the throttle target is, but if you wait a minute you can rerun it.

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed