QuantConnect

The US Futures dataset depends on the following datasets:

• US Futures Security Master - data to construct continuous Futures.
• US Future Universe - data on the available US Future contracts, their daily trading volume, and Open Interest.

Before you download US Futures data, open the Pricing page of your organization and subscribe to the US Futures Security Master by QuantConnect and US Future Universe History by QuantConnect data packages. You need billing permissions to change the organization's subscriptions.

After you subscribe to local access (see Prerequisites), open a terminal in your organization workspace and run the following commands to bulk download the data and its prerequisites.

To download the US Futures Security Master, run:

$ lean data download --dataset "US Futures Security Master"

To download the US Future Universe data, run:

$ lean data download --dataset "US Future Universe" --data-type "Bulk" --start "20250403" --end "20250403"

To download the US Futures data for a resolution, run the following command, replacing <resolution> with daily, hour, minute, second, or tick and adjusting the date range:

$ lean data download --dataset "US Futures" --data-type "Bulk" --resolution "<resolution>" --start "20230101" --end "20230105"

You can also use the CLI Command Generator. For more information, see Using the CLI.

After you bulk download the US Futures dataset, new daily updates are available at 7 AM Eastern Time (ET) after each trading day. Instead of directly calling the lean data download command, you can place a Python script in the data directory of your organization workspace and run it to update your data files. The following example script updates all data resolutions and markets:

import os
import pandas as pd
from datetime import datetime, time, timedelta
from pytz import timezone
from os.path import abspath, dirname
os.chdir(dirname(abspath(__file__)))

OVERWRITE = False

# Define a method to download the data
def __download_data(resolution, start=None, end=None):
    print(f"Updating {resolution} data...")
    command = f'lean data download --dataset "US Futures" --data-type "Bulk" --market "CBOT" --resolution "{resolution}"'
    if start:
        end = end if end else start
        command += f" --start {start} --end {end}"
    if OVERWRITE:
        command += " --overwrite"
    print(command)
    os.system(command)

def __get_end_date() -> str:
    now = datetime.now(timezone("US/Eastern"))
    if now.time() > time(7,30):
        return (now - timedelta(1)).strftime("%Y%m%d")
    print('New data is available at 07:30 AM EST')
    return (now - timedelta(2)).strftime("%Y%m%d")

def __download_high_frequency_data(latest_on_cloud):
    for resolution in ["minute", "second", "tick"]:
        dir_name = f"future/cbot/{resolution}/zc".lower()
        if not os.path.exists(dir_name):
            __download_data(resolution, '19980101')
            continue
        latest_on_disk = sorted(os.listdir(dir_name))[-1].split('_')[0]
        if latest_on_disk >= latest_on_cloud:
            print(f"{resolution} data is already up to date.")
            continue
        __download_data(resolution, latest_on_disk, latest_on_cloud)

def __download_low_frequency_data(latest_on_cloud):
    for resolution in ["daily", "hour"]:
        file_name = f"future/cbot/{resolution}/zc.zip".lower()
        if not os.path.exists(file_name):
            __download_data(resolution)
            continue
        latest_on_disk = str(pd.read_csv(file_name, header=None)[0].iloc[-1])[:8]
        if latest_on_disk >= latest_on_cloud:
            print(f"{resolution} data is already up to date.")
            continue
        __download_data(resolution)

if __name__ == "__main__":
    latest_on_cloud = __get_end_date()
    __download_low_frequency_data(latest_on_cloud)
    __download_high_frequency_data(latest_on_cloud)

The preceding script checks the date of the most recent ZC data you have from the CBOT market for tick, second, and minute resolutions. If there is new data available for any of these resolutions, it downloads the new data files and overwrites your hourly and daily files. If you don't intend to download all resolutions and markets, adjust this script to your needs.

To update your local copy of the US Futures Security Master, run:

$ lean data download --dataset "US Futures Security Master"

To update your local copy of the US Future Universe data, run:

$ lean data download --dataset "US Future Universe" --data-type "Bulk" --start "20250403" --end "20250403"

To download data for selected tickers instead of the full dataset, run a non-interactive lean data download command. First, download the prerequisite US Futures Security Master:

$ lean data download --dataset "US Futures Security Master"

Next, download the US Future Universe data for the ticker to get the available contracts:

$ lean data download --dataset "US Future Universe" --data-type "Trade" --ticker "ES" --start "20250630" --end "20260630"

Then download the US Futures data for the tickers, resolution, and date range you need. The dataset provides trade, quote, and open interest data, so run a command for each data type you need. For example, to download minute-resolution data for ES:

$ lean data download --dataset "US Futures" --data-type "Trade" --ticker "ES" --resolution "Minute" --start "20230101" --end "20230105"
$ lean data download --dataset "US Futures" --data-type "Quote" --ticker "ES" --resolution "Minute" --start "20230101" --end "20230105"
$ lean data download --dataset "US Futures" --data-type "Open Interest" --ticker "ES" --resolution "Minute" --start "20230101" --end "20230105"

To download data interactively or to use the CLI Command Generator, see Using the CLI.

The following table shows the size and format of the US Futures dataset for each resolution:

For more information about the file format, see the Data / future directory in the LEAN repository.

Both the bulk and by-ticker downloads require the US Futures Security Master. The following table shows the cost of an annual subscription to the US Futures Security Master for each organization tier:

Download in Bulk

To download the US Futures dataset in bulk, subscribe to it on the Pricing page of your organization. The price depends on your organization tier and the resolution you need. The bulk download also requires the US Future Universe subscription. The following table shows the price ($/year) to download the historical data of each resolution for each organization tier:

After the first bulk subscription ends, subscribe to the updates to keep your local data current. The following table shows the price ($/year) of the updates of each resolution for each organization tier:

The following table shows the annual price ($/year) of the US Future Universe historical and updates subscriptions for each organization tier:

The following table shows the total cost of downloading the required datasets in bulk at minute resolution on the Quant Researcher tier. Other organization tiers apply their own rates, shown in the preceding tables.

Download by Ticker

The file format of the US Future Universe dataset is one file per Future per day and each file costs 100 QCC = $1 USD.

The US Futures dataset is available in several resolutions. The resolution you need depends on the US Future subscriptions you create in your algorithm and the resolution of data you get in history requests. The following table describes the file format and costs of each resolution:

For example, the following algorithm subscribes to minute resolution data for a universe of ES Futures contracts and creates a continuous contract:

public class USFuturesDataAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2020, 1, 1);
        SetEndDate(2021, 1, 1);
        var future = AddFuture(
            Futures.Indices.SP500EMini,
            dataNormalizationMode: DataNormalizationMode.BackwardsRatio,
            dataMappingMode: DataMappingMode.OpenInterest,
            contractDepthOffset: 0
        );
        future.SetFilter(0, 90);
    }
}

class USFuturesDataAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2020, 1, 1)
        self.set_end_date(2021, 1, 1)
        future = self.add_future(
            Futures.Indices.SP_500_E_MINI,
            data_normalization_mode=DataNormalizationMode.BACKWARDS_RATIO,
            data_mapping_mode=DataMappingMode.OPEN_INTEREST,
            contract_depth_offset=0
        )
        future.set_filter(0, 90)

The following table shows the data cost of the preceding algorithm on the Quant Researcher tier:

The preceding table assumes you download trade, quote, and open interest data. However, you can run backtests with only trade data.