---
title: "Custom Data"
description: "You can import external datasets into your algorithm to use alongside other datasets from the Dataset Market."
canonical: https://www.quantconnect.com/docs/v2/writing-algorithms/historical-data/custom-data
markdown: https://www.quantconnect.com/docs/v2/writing-algorithms/historical-data/custom-data.md
language: python
source: https://github.com/QuantConnect/Documentation/tree/master/03%20Writing%20Algorithms/20%20Historical%20Data/06%20Custom%20Data
modified: 2026-10-30
index: https://www.quantconnect.com/llms.txt
---

# Custom Data

[Documentation](https://www.quantconnect.com/docs/v2.md) > [Writing Algorithms](https://www.quantconnect.com/docs/v2/writing-algorithms.md) > Historical Data > Custom Data

> Code examples are shown in Python. Append `?lang=csharp` to this URL for the C# version of this page.

## Introduction

You can import external datasets into your algorithm to use alongside other datasets from the Dataset Market. This page explains how to get historical data for custom datasets. Before you can get historical data for the dataset, define the [`get_source`](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/key-concepts.md#04-Set-Data-Sources) and [`reader`](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/key-concepts.md#05-Parse-Custom-Data) methods of the custom data class. For examples of custom dataset implementations, see [Key Concepts](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/key-concepts.md).

## Slices

To request `Slice` objects of historical data, call the `history` method without providing any `Symbol` objects. It returns data for all the data subscriptions in your notebook, so the result may include more than just custom data.

```python
# Get the latest 3 data points of all the securities/datasets in the notebook, packaged into Slice objects.
history = self.history(3)
```

When your history request returns `Slice` objects, the `time` properties of these objects are based on the algorithm time zone, but the `end_time` properties of the individual data objects are based on the [exchange time zone](https://www.quantconnect.com/docs/v2/writing-algorithms/key-concepts/time-modeling/time-zones.md#03-Exchange-Time-Zone). The `end_time` is the end of the sampling period and when the data is actually available.

## Data Points

To get historical data points for a custom dataset, call the `history` method with the dataset `Symbol`. This method returns a DataFrame that contains the data point attributes of the dataset class. For an example definition of a custom data class, see the [CSV Format Example](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/custom-securities/csv-format-example.md).

```python
class CustomSecurityHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 9, 1)
        self.set_end_date(2024, 12, 31)
        # Add a custom dataset and save a reference to it's Symbol.
        dataset_symbol = self.add_data(MyCustomDataType, "MyCustomDataType", Resolution.DAILY).symbol
        # Get the trailing 5 days of MyCustomDataType data in DataFrame format.
        history = self.history(dataset_symbol, 5, Resolution.DAILY)
```

| symbol | time | close | high | low | open | value |
| --- | --- | --- | --- | --- | --- | --- |
| MYCUSTOMDATATYPE.MyCustomDataType | 2014-07-08 | 7787.15 | 7792.00 | 7755.10 | 7780.40 | 7787.15 |
| MYCUSTOMDATATYPE.MyCustomDataType | 2014-07-09 | 7623.20 | 7808.85 | 7595.90 | 7804.05 | 7623.20 |
| MYCUSTOMDATATYPE.MyCustomDataType | 2014-07-10 | 7585.00 | 7650.10 | 7551.65 | 7637.95 | 7585.00 |

If you intend to use the data in the DataFrame to create `customDatasetClass` objects, request that the history request returns the data type you need. Otherwise, LEAN consumes unnecessary computational resources populating the DataFrame. To get a list of dataset objects instead of a DataFrame, call the `history[ customDatasetClass ]` method.

```python
# Get the trailing 5 days of MyCustomDataType data for an asset in MyCustomDataType format.
history = self.history[MyCustomDataType](dataset_symbol, 5, Resolution.DAILY)
```

If the dataset provides multiple entries per [time step](https://www.quantconnect.com/docs/v2/writing-algorithms/key-concepts/time-modeling/timeslices.md), in the `get_source` method of your custom data class, return a `SubscriptionDataSource` that uses `FileFormat.UNFOLDING_COLLECTION`. To get the historical data of this custom data type in a DataFrame, set the `flatten` argument to `True`.

```python
history = self.history(dataset_symbol, 1, Resolution.DAILY, flatten=True)
```

## Universes

To get historical data for a [custom data universe](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/custom-universes/key-concepts.md), call the `history` method with the `Universe` object. For an example definition of a custom data universe class, see the [CSV Format Example](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/custom-universes/csv-format-example.md).

```python
class CustomDataUniverseHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 9, 1)
        self.set_end_date(2024, 12, 31)
        # Add a universe from a custom data source and save a reference to it.
        universe = self.add_universe(
            StockDataSource, "my-stock-data-source", Resolution.DAILY, lambda data: [x.symbol for x in data]
        )
        # Get the historical universe data over the last 5 days in DataFrame format.
        history = self.history(universe, timedelta(5))
```

| time | symbols |
| --- | --- |
| 2017-07-05 | [SPY, QQQ, FB, AAPL, IWM] |
| 2017-07-06 | [SPY, QQQ, FB, AAPL, IWM] |
| 2017-07-07 | [QQQ, AAPL, IWM, FB, GOOGL] |
| 2017-07-08 | [IWM, AAPL, FB, BAC, GOOGL] |
| 2017-07-09 | [AAPL, FB, GOOGL, GOOG, BAC] |

```python
# Count the number of assets in the universe each day.
universe_size_by_day = history.apply(lambda row: len(row['symbols']), axis=1)
```

```python
time
2017-07-05    5
2017-07-06    5
2017-07-07    5
2017-07-08    5
2017-07-09    5
Name: symbols, dtype: int64
```

## Missing Data Points

History requests for a [trailing number of data samples](https://www.quantconnect.com/docs/v2/writing-algorithms/historical-data/history-requests.md#02-Trailing-Data-Samples) return data based on the [market hours](https://www.quantconnect.com/docs/v2/writing-algorithms/securities/market-hours.md) of assets. The default market hours for custom securities is to be always open. Therefore, history requests for a trailing number of data samples may return fewer samples than you expect. To set the market hours of the dataset, see [Market Hours](https://www.quantconnect.com/docs/v2/writing-algorithms/importing-data/streaming-data/custom-securities/key-concepts.md#12-Market-Hours).

## Examples

The following examples demonstrate some common practices for trading with historical custom data.

### Example 1: Custom Universe Data

The following algorithm trades based on a custom universe dataset. It obtains historical data on the universe's weights and uses the exponential moving average as the investment weight.

```python
class CustomUniverseExampleAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 9, 1)
        self.set_end_date(2024, 12, 31)
        # Add a universe that reads from the Object Store.
        self._universe = self.add_universe(
            CustomUniverseData, "CustomUniverse", Resolution.DAILY, lambda x: [y.symbol for y in x]
        )
        # Add a Scheduled Event to rebalance the portfolio.
        spy = Symbol.create('SPY', SecurityType.EQUITY, Market.USA)
        self.schedule.on(
            self.date_rules.every_day(spy),
            self.time_rules.after_market_open(spy, 1),
            self.rebalance
        )

    def rebalance(self) -> None:
        # We invest by the EMA of the given weight.
        history = self.history(self._universe, 5, Resolution.DAILY)
        if history.empty:
            return
        history = history.set_index('symbol', append=True).unstack('symbol').fillna(0)
        weights = history.ewm(5).mean().iloc[-1].droplevel(0)
        # Normalized weights.
        weights = weights / weights.sum()
        self.set_holdings([PortfolioTarget(symbol, weight) for symbol, weight in weights.items()], liquidate_existing_holdings=True)

class CustomUniverseData(PythonData):
    def get_source(self, config: SubscriptionDataConfig, date: datetime, is_live_mode: bool) -> SubscriptionDataSource:
        # Define the location and format of the data file.
        return SubscriptionDataSource(
            "portfolio-targets.csv", SubscriptionTransportMedium.OBJECT_STORE, FileFormat.CSV
        )

    def reader(self, config: SubscriptionDataConfig, line: str, date: datetime, is_live_mode: bool) -> BaseData:
        # Skip the header row.
        if not line[0].isnumeric():
            return None
        # Split the line by each comma.
        items = line.split(",")
        # Parse the data from the CSV file.
        data = CustomUniverseData()
        data.end_time = datetime.strptime(items[0], "%Y-%m-%d")
        data.time = data.end_time - timedelta(1)
        data['symbol'] = Symbol.create(items[1], SecurityType.EQUITY, Market.USA)
        data["weight"] = float(items[2])
        return data
```

### Other Examples

For more examples, see the following algorithms:

**Demonstration Algorithm**

- [HistoryAlgorithm.py](https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/HistoryAlgorithm.py) (Python)