OHLCV properties of Security objects incorrectly set for custom data.

From the information provided in the extracts, it seems like you are trying to map OHLCV data to a custom data object. However, the custom data object in QuantConnect doesn't directly support OHLCV fields. The custom data object only has a Value field which is used as the default data point value.

In your code, you are setting the Value field to the close price (ct.Value = ohlcv[3]). This is why all the OHLCV attributes are set to the close price. The volume is set to 0 because the custom data object doesn't have a volume attribute.

If you want to use OHLCV data, you might want to consider using the TradeBar object instead of a custom data object. The TradeBar object has OHLCV fields that you can set directly. Here is an example of how you can do this:

def Reader(self, config, line, date, isLiveMode):
    data = line.split(",")
    bar = TradeBar()
    bar.Time = datetime.strptime(data[0], "%Y-%m-%d %H:%M:%S")
    bar.Symbol = config.Symbol
    bar.Open = float(data[1])
    bar.High = float(data[2])
    bar.Low = float(data[3])
    bar.Close = float(data[4])
    bar.Volume = float(data[5])
    bar.EndTime = bar.Time + timedelta(minutes=1)  # adjust the timedelta according to your data's timeframe
    return bar

This code will create a TradeBar object for each line in your data, and set the OHLCV fields accordingly.

Please note that this is just a suggestion based on the information provided in the extracts. You might need to adjust the code to fit your specific needs.

For more information on how to use custom data in QuantConnect, you can refer to the official documentation.

But returning a TradeBar object instead of CustomData won't break internally the backtest engine since it's expecting a PythonData extension?

Returning a TradeBar object from the Reader method of a custom data type should not break the backtest engine. The Reader method is expected to return an instance of a class that extends BaseData. Both PythonData and TradeBar extend BaseData, so they can be used interchangeably in this context.

However, as mentioned in the forum discussion titled "Custom data: returning TradeBar instead of custom class object", if you use your custom data TradeBar in Lean where the symbol signals a type of DynamicData (the type for custom data which gets attributed to your symbol by AddData()), it may fail as TradeBar is not DynamicData.

So, while you can return a TradeBar from your Reader method, you should be aware of these potential issues when using the data in your algorithm. If you encounter any issues, you may need to adjust your code accordingly or consider using a custom class that extends BaseData and includes the specific fields you need.

That's my point, isn't there any way to have custom data Security object updates without sacrifying the use of Lean helper methods?

Based on the information provided in the extracts, it seems that you want to use custom data with OHLCV fields and also want to take advantage of Lean's helper methods.

Currently, Lean's custom data feature is designed to be flexible and can handle any type of data, not just OHLCV data. This is why custom data objects only have a Value field by default.

However, there is a discussion in the forum titled "The SymbolData Pattern" about adding some “sugar” in QuantConnect/Lean, for example, attaching the custom properties to the Security object to eliminate the user-defined class dictionary. This feature might help in your case, but it seems it's still under consideration.

In the meantime, you can create a custom class that extends BaseData and includes the specific fields you need. You can then use this custom class in your algorithm and access the OHLCV fields directly. Here is an example:

class MyCustomData(BaseData):
    def __init__(self):
        self.Open = 0
        self.High = 0
        self.Low = 0
        self.Close = 0
        self.Volume = 0

    def Reader(self, config, line, date, isLiveMode):
        data = line.split(",")
        myData = MyCustomData()
        myData.Symbol = config.Symbol
        myData.Time = datetime.strptime(data[0], "%Y-%m-%d %H:%M:%S")
        myData.Open = float(data[1])
        myData.High = float(data[2])
        myData.Low = float(data[3])
        myData.Close = float(data[4])
        myData.Volume = float(data[5])
        myData.Value = myData.Close  # set Value to Close price
        return myData

This code will create a MyCustomData object for each line in your data, and set the OHLCV fields accordingly. You can then use these fields in your algorithm.

Please note that this is just a suggestion based on the information provided in the extracts. You might need to adjust the code to fit your specific needs.

For more information on how to use custom data in QuantConnect, you can refer to the official documentation.

For anyone interested I found a workaround in C#:

• Implement both BaseClass and Data.Market.IBaseDataBar when defining the custom data class.
• The implementation of IBaseDataBar must be explicit, with it's OHLC read-only properties pointing at user defined OHLC fields as getters.

The following is an example custom data class that implements this workaround:

        public class Bitcoin : BaseData, Data.Market.IBaseDataBar
        {
            [JsonProperty("timestamp")]
            public int Timestamp = 0;
            [JsonProperty("open")]
            public decimal Open = 0;
            [JsonProperty("high")]
            public decimal High = 0;
            [JsonProperty("low")]
            public decimal Low = 0;
            [JsonProperty("last")]
            public decimal Close = 0;
            [JsonProperty("bid")]
            public decimal Bid = 0;
            [JsonProperty("ask")]
            public decimal Ask = 0;
            [JsonProperty("vwap")]
            public decimal WeightedPrice = 0;
            [JsonProperty("volume")]
            public decimal VolumeBTC = 0;
            public decimal VolumeUSD = 0;

            /// <summary>
            /// The end time of this data. Some data covers spans (trade bars)
            /// and as such we want to know the entire time span covered
            /// </summary>
            /// <remarks>
            /// This property is overriden to allow different values for Time and EndTime
            /// if they are set in the Reader. In the base implementation EndTime equals Time
            /// </remarks>
            public override DateTime EndTime { get; set; }

            decimal IBar.Open => Open;

            decimal IBar.High => High;

            decimal IBar.Low => Low;

            decimal IBar.Close => Close;

            /// <summary>
            /// 1. DEFAULT CONSTRUCTOR: Custom data types need a default constructor.
            /// We search for a default constructor so please provide one here. It won't be used for data, just to generate the "Factory".
            /// </summary>
            public Bitcoin()
            {
                Symbol = "BTC";
            }


            /// <summary>
            /// 2. RETURN THE STRING URL SOURCE LOCATION FOR YOUR DATA:
            /// This is a powerful and dynamic select source file method. If you have a large dataset, 10+mb we recommend you break it into smaller files. E.g. One zip per year.
            /// We can accept raw text or ZIP files. We read the file extension to determine if it is a zip file.
            /// </summary>
            /// <param name="config">Configuration object</param>
            /// <param name="date">Date of this source file</param>
            /// <param name="isLiveMode">true if we're in live mode, false for backtesting mode</param>
            /// <returns>String URL of source file.</returns>
            public override SubscriptionDataSource GetSource(SubscriptionDataConfig config, DateTime date, bool isLiveMode)
            {
                if (isLiveMode)
                {
                    return new SubscriptionDataSource("https://www.bitstamp.net/api/ticker/", SubscriptionTransportMedium.Rest);
                }

                //return "http://my-ftp-server.com/futures-data-" + date.ToString("Ymd") + ".zip";
                // OR simply return a fixed small data file. Large files will slow down your backtest
                return new SubscriptionDataSource("https://www.quantconnect.com/api/v2/proxy/quandl/api/v3/datasets/BCHARTS/BITSTAMPUSD.csv?order=asc&api_key=WyAazVXnq7ATy_fefTqm", SubscriptionTransportMedium.RemoteFile);
            }

            /// <summary>
            /// 3. READER METHOD: Read 1 line from data source and convert it into Object.
            /// Each line of the CSV File is presented in here. The backend downloads your file, loads it into memory and then line by line
            /// feeds it into your algorithm
            /// </summary>
            /// <param name="line">string line from the data source file submitted above</param>
            /// <param name="config">Subscription data, symbol name, data type</param>
            /// <param name="date">Current date we're requesting. This allows you to break up the data source into daily files.</param>
            /// <param name="isLiveMode">true if we're in live mode, false for backtesting mode</param>
            /// <returns>New Bitcoin Object which extends BaseData.</returns>
            public override BaseData Reader(SubscriptionDataConfig config, string line, DateTime date, bool isLiveMode)
            {
                var coin = new Bitcoin();
                if (isLiveMode)
                {
                    //Example Line Format:
                    //{"high": "441.00", "last": "421.86", "timestamp": "1411606877", "bid": "421.96", "vwap": "428.58", "volume": "14120.40683975", "low": "418.83", "ask": "421.99"}
                    try
                    {
                        coin = JsonConvert.DeserializeObject<Bitcoin>(line);
                        coin.EndTime = DateTime.UtcNow.ConvertFromUtc(config.ExchangeTimeZone);
                        coin.Value = coin.Close;
                    }
                    catch { /* Do nothing, possible error in json decoding */ }
                    return coin;
                }

                //Example Line Format:
                //Date      Open   High    Low     Close   Volume (BTC)    Volume (Currency)   Weighted Price
                //2011-09-13 5.8    6.0     5.65    5.97    58.37138238,    346.0973893944      5.929230648356
                try
                {
                    string[] data = line.Split(',');
                    coin.Time = DateTime.Parse(data[0], CultureInfo.InvariantCulture);
                    coin.EndTime = coin.Time.AddDays(1);
                    coin.Open = Convert.ToDecimal(data[1], CultureInfo.InvariantCulture);
                    coin.High = Convert.ToDecimal(data[2], CultureInfo.InvariantCulture);
                    coin.Low = Convert.ToDecimal(data[3], CultureInfo.InvariantCulture);
                    coin.Close = Convert.ToDecimal(data[4], CultureInfo.InvariantCulture);
                    coin.VolumeBTC = Convert.ToDecimal(data[5], CultureInfo.InvariantCulture);
                    coin.VolumeUSD = Convert.ToDecimal(data[6], CultureInfo.InvariantCulture);
                    coin.WeightedPrice = Convert.ToDecimal(data[7], CultureInfo.InvariantCulture);
                    coin.Value = coin.Close;
                }
                catch { /* Do nothing, skip first title row */ }

                return coin;
            }