Get historical data from custom data

To access historical data for custom data added via AddData in the OnData method, you can use the self.History() method. Here's how you can achieve this:

1. Initialize the Custom Data: Ensure that your custom data is added using the AddData method in the Initialize method.

2. Access Historical Data: In the OnData method, use the self.History() method to request historical data for the custom data symbol.

3. Specify the Time Frame: Define the time frame for which you want to retrieve historical data, such as 1 day ago or 2 days ago.

4. Handle the DataFrame: The self.History() method returns a DataFrame, so handle it accordingly to extract the required historical values.

5. Check for Data Availability: Ensure that the DataFrame is not empty before attempting to access specific data points.

6. Extract Specific Data Points: Use the DataFrame's indexing capabilities to access the data for specific days, such as 1 day ago or 2 days ago.

Here's an example implementation:

class CustomDataAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2022, 12, 31)
        self.set_cash(100000)
        # Add custom data
        self.signal_subscription = self.add_data(SignalData, "SIGNALS", Resolution.DAILY)
    def on_data(self, data):
        # Retrieve historical data for the custom data symbol
        history = self.history(self.signal_subscription.Symbol, 3, Resolution.DAILY)
        if not history.empty:
            # Access data from 1 day ago
            value_1_day_ago = history.iloc[-2]['value']
            # Access data from 2 days ago
            value_2_days_ago = history.iloc[-3]['value']
            self.debug(f"Value 1 day ago: {value_1_day_ago}, Value 2 days ago: {value_2_days_ago}")

In this example, self.History() is used to request the last 3 days of historical data for the custom data symbol. The values from 1 and 2 days ago are accessed using the DataFrame's iloc method. Make sure to adjust the column name 'value' to match the actual column name in your custom data.

Sources: Using Data - QuantConnect.com, Custom Data - QuantConnect.com, Custom Data Type History Algorithm, Key Concepts - QuantConnect.com, Custom Data - QuantConnect.com

Thank you for the solution. However, the history often becomes empty although there seem to be enough data in the csv. What is the problem?

self.log(f"History data:\n{history}")
>>
 History data: Empty DataFrame Columns: [] Index: []

Hi Norris

We recommend using a RollingWindow to cache the data.

Best
Louis