QUANTCONNECT COMMUNITY
Hi,
I have a custom data type extending BaseData, this data is minute by minute and I'd like to consolidate the bars into hourly bars. I've been looking at the examples but I have not been able to figure this out just yet.
Does anyone have a sample of something similar?
Also fwiw I am running the Lean engine via VS2013 Community Ed. on my Windows 8.1 workstation and the custom data is a csv on the workstation.
Thanks,
Greg L.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
QuantConnect™ 2023. All Rights Reserved
|
|
Share New Research 
Michael Handschuh
var consolidator = ResolveConsolidator("CustomSymbol", Resolution.Hour);Greg L
var consolidator = ResolveConsolidator("CustomSymbol", Resolution.Daily );work the same? From my test it did not seem to. When its set to Resolution.Hour the HourBarHandler event is triggered and the Console.Writeline happens. If I change Resolution.Hour to Resolution.Daily the event does not seem to fire. Here is my code:public class TestingCustomDataAlgo : QCAlgorithm { ///
/// Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.
///
public override void Initialize()
{
SetStartDate(new DateTime(2010, 04, 02, 00, 00, 00));
SetEndDate(2010, 05, 04);
AddData("EURUSD");
//wire up an event to handle the bars
var consolidator = ResolveConsolidator("EURUSD", Resolution.Hour);
//var consolidator = ResolveConsolidator("EURUSD", Resolution.Daily);
consolidator.DataConsolidated += HourBarHandler;
SubscriptionManager.AddConsolidator("EURUSD", consolidator);
}
///
/// OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
///
/// TradeBars IDictionary object with your stock data
public void OnData(EURUSD data)
{
//Console.WriteLine("Time: " + data.Time);
}
private void HourBarHandler(object sender, BaseData consolidated)
{
Console.WriteLine("Consolidated Time: " + consolidated.Time);
}
}
///
/// EURUSD is a custom data type we create for this algorithm
///
public class EURUSD : BaseData
{
///
/// Opening Price
///
public decimal Open = 0;
///
/// High Price
///
public decimal High = 0;
///
/// Low Price
///
public decimal Low = 0;
///
/// Closing Price
///
public decimal Close = 0;
///
/// Default initializer for EURUSD.
///
public EURUSD()
{
Symbol = "EURUSD";
}
///
/// Return the URL string source of the file. This will be converted to a stream
///
public override string GetSource(SubscriptionDataConfig config, DateTime date, DataFeedEndpoint datafeed)
{
return @"..\..\..\Data\custom\EURUSD_1M_20100401_20141111.csv";
}
///
/// Reader converts each line of the data source into BaseData objects. Each data type creates its own factory method, and returns a new instance of the object
/// each time it is called.
///
public override BaseData Reader(SubscriptionDataConfig config, string line, DateTime date, DataFeedEndpoint datafeed)
{
EURUSD index = new EURUSD();
try
{
string[] data = line.Split(',');
string timeValue = (Convert.ToInt32(data[2])).ToString("00:00:00");
string dateValue = (Convert.ToInt32(data[1])).ToString("0000/00/00");
index.Symbol = data[0];
index.Time = DateTime.Parse(dateValue + " " + timeValue);
index.Open = Convert.ToDecimal(data[3], CultureInfo.InvariantCulture);
index.High = Convert.ToDecimal(data[4], CultureInfo.InvariantCulture);
index.Low = Convert.ToDecimal(data[5], CultureInfo.InvariantCulture);
index.Close = Convert.ToDecimal(data[6], CultureInfo.InvariantCulture);
index.Value = index.Close;
}
catch (Exception ex)
{
ex.ToString();
}
return index;
}
}
Basically instead of re-formatting / packaging my csv data I found it easier to manipulate the time and date fields using the custom type Reader method. Thanks again for the help, GregMichael Handschuh
Greg L
Greg L
Michael Handschuh
Greg L
Greg L
var dtbConsolidator = new DailyTradeBarConsolidator(); dtbConsolidator.DataConsolidated += DayBarHandler; SubscriptionManager.AddConsolidator("EURUSD", dtbConsolidator);I get a runtime error like this: Type mismatch found between consolidator and symbol. Symbol: EURUSD expects type EURUSD but tried to register consolidator with input type TradeBar Conceptually I think I get difference between a 'symbol' and a 'bar' of that symbol but without actually reading all the source code I'm not seeing how to get a 'TradeBar' from 'BaseData.' Is there a method I can override to return a TradeBar from my type that extends BaseData or do I need to use a different approach? Thanks again for your assistance, GregMichael Handschuh
Greg L
using System; using System.IO; using System.Collections.Generic; using System.Globalization; using System.Linq; using System.Text; using System.Threading.Tasks; using QuantConnect.Data; using QuantConnect.Data.Market; using QuantConnect.Data.Consolidators; namespace QuantConnect.Algorithm.Examples { ///
/// Testing algorithm simply initializes the date range and cash
///
public class TestingCustomDataAlgo : QCAlgorithm
{
///
/// Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.
///
public override void Initialize()
{
SetStartDate(new DateTime(2010, 05, 03, 00, 00, 00));
SetEndDate(2010, 06, 01);
AddData("EURUSD");
//create an event to handle the hourly bars
var consolidator = ResolveConsolidator("EURUSD", Resolution.Daily);
consolidator.DataConsolidated += HourBarHandler;
SubscriptionManager.AddConsolidator("EURUSD", consolidator);
var dtbConsolidator = new DailyTradeBarConsolidator();
dtbConsolidator.DataConsolidated += DayBarHandler;
SubscriptionManager.AddConsolidator("EURUSD", dtbConsolidator);
}
private void DayBarHandler(object sender, BaseData consolidated)
{
Console.WriteLine("\nDBH Consolidated Time: " + consolidated.Time.DayOfWeek + " at " + consolidated.Time);
}
///
/// OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
///
/// TradeBars IDictionary object with your stock data
public void OnData(EURUSD data)
{
//Console.WriteLine("Time: " + data.Time);
}
private void HourBarHandler(object sender, BaseData consolidated)
{
Console.WriteLine("\nConsolidated Time: " + consolidated.Time.DayOfWeek + " at " + consolidated.Time);
//Debug("\nConsolidated Daily Time: " + consolidated.Time);
}
}
///
/// ad hoc fix for daily close issue
///
public class DailyTradeBarConsolidator : DataConsolidator
{
private TradeBar _working;
///
/// OutputType returns a Type?
///
public override Type OutputType
{
get { return typeof(TradeBar); }
}
///
/// Update method
///
/// Tradebar type
public override void Update(TradeBar data)
{
if (_working != null && _working.Time.Date != data.Time.Date)
{
OnDataConsolidated(_working);
_working = null;
}
AggregateBar(ref _working, data);
}
///
/// Aggregates the new 'data' into the 'workingBar'. The 'workingBar' will be
/// null following the event firing
///
/// The bar we're building, null if the event was just fired and we're starting a new trade bar
/// The new data
protected void AggregateBar(ref TradeBar workingBar, TradeBar data)
{
if (workingBar == null)
{
workingBar = new TradeBar
{
Time = data.Time,
Symbol = data.Symbol,
Open = data.Open,
High = data.High,
Low = data.Low,
Close = data.Close,
Volume = data.Volume,
DataType = MarketDataType.TradeBar,
Period = data.Period
};
}
else
{
//Aggregate the working bar
workingBar.Close = data.Close;
workingBar.Volume += data.Volume;
workingBar.Period += data.Period;
if (data.Low < workingBar.Low) workingBar.Low = data.Low;
if (data.High > workingBar.High) workingBar.High = data.High;
}
}
}
///
/// EURUSD is a custom data type we create for this algorithm
///
public class EURUSD : BaseData
{
///
/// Opening Price
///
public decimal Open = 0;
///
/// High Price
///
public decimal High = 0;
///
/// Low Price
///
public decimal Low = 0;
///
/// Closing Price
///
public decimal Close = 0;
///
/// Default initializer for EURUSD.
///
public EURUSD()
{
Symbol = "EURUSD";
}
///
/// Return the URL string source of the file. This will be converted to a stream
///
public override string GetSource(SubscriptionDataConfig config, DateTime date, DataFeedEndpoint datafeed)
{
return @"..\..\..\Data\custom\EURUSD_1M_20100401_20141111.csv";
}
///
/// Reader converts each line of the data source into BaseData objects. Each data type creates its own factory method, and returns a new instance of the object
/// each time it is called.
///
public override BaseData Reader(SubscriptionDataConfig config, string line, DateTime date, DataFeedEndpoint datafeed)
{
EURUSD index = new EURUSD();
try
{
string[] data = line.Split(',');
string timeValue = (Convert.ToInt32(data[2])).ToString("00:00:00");
string dateValue = (Convert.ToInt32(data[1])).ToString("0000/00/00");
index.Symbol = data[0];
index.Time = DateTime.Parse(dateValue + " " + timeValue);
index.Open = Convert.ToDecimal(data[3], CultureInfo.InvariantCulture);
index.High = Convert.ToDecimal(data[4], CultureInfo.InvariantCulture);
index.Low = Convert.ToDecimal(data[5], CultureInfo.InvariantCulture);
index.Close = Convert.ToDecimal(data[6], CultureInfo.InvariantCulture);
index.Value = index.Close;
}
catch (Exception ex)
{
ex.ToString();
}
return index;
}
}
}
Michael Handschuh
Greg L
JP B
DailyTradeBarConsolidator()in a new release of LEAN? Also, when backtesting with normal equity data, can I just useResolveConsolidator("MSFT", Resolution.Daily);or is it better to use the customDailyTradeBarConsolidator() ?Michael Handschuh
var consolidator = new TradeBarConsolidator(TimeSpan.FromDays(1));That being said, using the following is always safe:var consolidator = ResolveConsolidator("SPY", Resolution.Daily);JP B
Greg L
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!