QuantConnect

US ETF Constituents

Introduction

The US ETF Constituents dataset by QuantConnect tracks the constituents and weighting of US Equities in 2,650 ETF listings. The data starts in June 2009 and is delivered on a daily basis (monthly basis before January 2015). This dataset is created by tracking the host ETF websites and can be delayed by up to 1 week.

This dataset depends on the US Equity Security Master dataset because the US Equity Security Master dataset contains information on splits, dividends, and symbol changes.

For more information about the US ETF Constituents dataset, including CLI commands and pricing, see the dataset listing.

About the Provider

QuantConnect was founded in 2012 to serve quants everywhere with the best possible algorithmic trading technology. Seeking to disrupt a notoriously closed-source industry, QuantConnect takes a radically open-source approach to algorithmic trading. Through the QuantConnect web platform, more than 160,000 quants are served every month.

Getting Started

The following snippet demonstrates how to request data from the US ETF Constituents dataset:

def initialize(self) -> None:
    self.universe_settings.asynchronous = True
    # Use the following method for a Classic Algorithm
    self._universe = self.add_universe(self.universe.etf("SPY", Market.USA, self.universe_settings, self.etf_constituents_filter))

    symbol = Symbol.create("SPY", SecurityType.EQUITY, Market.USA)
    # Use the following method for a Framework Algorithm
    self.add_universe_selection(ETFConstituentsUniverseSelectionModel(symbol, self.universe_settings, self.etf_constituents_filter))

    def etf_constituents_filter(self, constituents: List[ETFConstituentUniverse]) -> List[Symbol]:
        # Add all Symbols of the ETFConstituentUniverse
       return [x.symbol for x in constituents]
public override void Initialize()
{
    UniverseSettings.Asynchronous = true;
    // Use the following method for a Classic Algorithm
    _universe = AddUniverse(Universe.ETF("SPY", Market.USA, UniverseSettings, ETFConstituentsFilter));

    var symbol = QuantConnect.Symbol.Create("SPY", SecurityType.Equity, Market.USA);
    // Use the following method for a Framework Algorithm
    AddUniverseSelection(new ETFConstituentsUniverseSelectionModel(symbol, UniverseSettings, ETFConstituentsFilter));
}
private IEnumerable<Symbol> ETFConstituentsFilter(IEnumerable <ETFConstituentUniverse> constituents)
{
    // Add all Symbols of the ETFConstituentUniverse
    return constituents.Select(x => x.Symbol);
}

Data Summary

The following table describes the dataset properties:

PropertyValue
Start DateJune 2009
Asset Coverage2,650 US ETF Listings
Data DensityDense
ResolutionDaily (Monthly before Jan 2015)
TimezoneNew York

Requesting Data

To add US ETF Constituents data to your algorithm, call the AddUniverseadd_universe and Universe.ETFuniverse.etf methods. To select which constituents occupy the universe, provide the ETF Symbol and a selection function. 

class ETFConstituentUniverseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2018, 1, 1)
        self.set_end_date(2020, 8, 25)
        self.set_cash(100000)
        self.universe_settings.asynchronous = True
        self._universe = self.add_universe(self.universe.etf("SPY", self.universe_settings, self.etf_constituents_filter))
public class ETFConstituentUniverseAlgorithm : QCAlgorithm
{
    private Universe _universe;

    public override void Initialize()
    {
        SetStartDate(2018, 1, 1);
        SetEndDate(2020, 8, 25);
        SetCash(100000);
        UniverseSettings.Asynchronous = true;
        _universe = AddUniverse(Universe.ETF("SPY", UniverseSettings, ETFConstituentsFilter));
    }
}

For more information about universe settings, see Settings.

Accessing Data

To access the US ETF Constituent data, use the ETFConstituentUniverse objects in your selection function. The data is available in daily resolution. The Symbol objects you return from your selection function defines the universe constituents.

def etf_constituents_filter(self, constituents: List[ETFConstituentUniverse]) -> List[Symbol]:
    for c in constituents:
        self.debug(f'{c.end_time} :: {c.last_update} :: {c.weight} :: {c.shares_held} :: {c.market_value}')
    return [x.symbol for x in constituents]
public IEnumerable<Symbol> ETFConstituentsFilter(IEnumerable<ETFConstituentUniverse> constituents)
{
    foreach (var c in constituents)
    {
        Debug($"{c.EndTime} :: {c.LastUpdate} :: {c.Weight} :: {c.SharesHeld} :: {c.MarketValue}");
    }

    return constituents.Select(c => c.Symbol);
}

Historical Data

You can get historical universe data in an algorithm and in the Research Environment.

Historical Universe Data in Algorithms

To get historical universe data in an algorithm, call the Historyhistory method with the Universe object and the lookback period. If there is no data in the period you request, the history result is empty.

var history = History(_universe, 30, Resolution.Daily);
foreach (var constituents in history)
{
    foreach (ETFConstituentUniverse constituent in constituents)
    {
        Log($"{constituent.Symbol} weight at {constituent.EndTime}: {constituent.Weight}");
    }
}
# DataFrame example where the columns are the ETFConstituentUniverse attributes: 
df_history = self.history(self.universe, 30, Resolution.DAILY, flatten=True)

# Series example where the values are lists of ETFConstituentUniverse objects: 
series_history = self.history(self.universe, 30, Resolution.DAILY)
for (universe_symbol, time), constituents in series_history.items():
    for constituent in constituents:
        self.log(f'{constituent.symbol} weight at {constituent.end_time}: {constituent.weight}')

Historical Universe Data in Research

To get historical universe data in research, call the UniverseHistoryuniverse_history method with the Universe object and the lookback period. The UniverseHistoryuniverse_history returns the filtered universe. If there is no data in the period you request, the history result is empty.

var universeHistory = qb.UniverseHistory(universe, qb.Time.AddDays(-30), qb.Time);
foreach (var constituents in universeHistory )
{
    foreach (ETFConstituentUniverse constituent in constituents)
    {
        Console.WriteLine($"{constituent.Symbol} weight at {constituent.EndTime}: {constituent.Weight}");
    }
}
# DataFrame example where the columns are the ETFConstituentUniverse attributes: 
df_history = qb.universe_history(universe, qb.time-timedelta(30), qb.time, flatten=True)

# Series example where the values are lists of ETFConstituentUniverse objects: 
series_history = qb.universe_history(universe, qb.time-timedelta(30), qb.time)
for (universe_symbol, time), constituents in series_history.items():
    for constituent in constituents:
       print(f"{constituent.symbol} weight at {constituent.end_time}: {constituent.weight}")

You can call the Historyhistory method in Research.

Supported ETFs

The following table shows the available ETFs:

Example Applications

The ETF Constituents dataset provides an excellent source of tradable universes for strategies without selection bias. When you use an ETF universe, the original ETF can serve as an excellent benchmark for your strategy performance. Other use cases include the following:

  • Creating an index-tracking algorithm for customized passive portfolio management
  • Performing statistical arbitrage with the base ETF

Classic Algorithm Example

The following example algorithm creates a dynamic universe of the 10 largest US Equities in the SPY ETF. Each day, the algorithm forms a dollar-neutral and market-neutral portfolio by buying the 10 ETF constituents and shorting the SPY ETF.

from AlgorithmImports import *


class ETFConstituentUniverseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 9, 1)
        self.set_end_date(2024, 12, 31)
        self.set_cash(100000)
        # Add the SPY to trade.
        self._spy = self.add_equity("SPY").symbol
        # Add an ETF constituents universe that selects the large caps 
        # in SPY. Save the universe object so you can get history
        # for the universe.
        universe = self.add_universe(
            self.universe.etf(
                self._spy, 
                universe_filter_func=self._select_assets
            )
        )
        # Get historical universe data.
        history = self.history(universe, 30, Resolution.DAILY, flatten=True)
        # Show an example of wrangling the historical data.
        etf_weights = history.weight.unstack(1)  
        # Create a dictionary to store the ETF weights each day.
        self._weight_by_symbol = {}
        # Add a Scheduled Event to rebalance the portfolio each day.
        self.schedule.on(
            self.date_rules.every_day(self._spy),
            self.time_rules.after_market_open(self._spy, 1),
            self._rebalance
        )

    def _select_assets(
            self, constituents: List[ETFConstituentUniverse]) -> List[Symbol]:
        # Select the 10 largest stocks in the SPY.
        selected = sorted(
            [c for c in constituents if c.weight], 
            key=lambda c: c.weight
        )[-10:]
        # Save the weights for position sizing.
        self._weight_by_symbol = {c.symbol: c.weight for c in selected}
        # Return the selected assets.
        return list(self._weight_by_symbol.keys())

    def _rebalance(self) -> None:
        # Create a long-short portfolio to earn the excess return of the 
        # top 10 weighted stocks from SPY.
        spy_weight = sum(self._weight_by_symbol.values())
        targets = [PortfolioTarget(self._spy, -0.5)]
        for symbol, weight in self._weight_by_symbol.items():
            targets.append(PortfolioTarget(symbol, 0.5*weight/spy_weight))
        # Liquidate the stocks that aren't in top 10.
        self.set_holdings(targets, True)

public class ETFConstituentUniverseAlgorithm : QCAlgorithm
{ 
    private Symbol _spy;
    // Create a dictionary to store the ETF weights each day.
    private Dictionary<Symbol, decimal> _weightBySymbol = new();
    
    public override void Initialize()
    {
        SetStartDate(2024, 9, 1);
        SetEndDate(2024, 12, 31);
        SetCash(100000);
        // Add the SPY to trade.
        _spy = AddEquity("SPY").Symbol;
        // Add an ETF constituents universe that selects the large caps 
        // in SPY. Save the universe object so you can get history
        // for the universe.
        var universe = AddUniverse(Universe.ETF(_spy, universeFilterFunc: SelectAssets));
        // Get historical universe data.
        var history = History(universe, 30, Resolution.Daily);
        // Show an example of iterating through the historical data.
        foreach (var constituents in history)
        {
            var t = constituents.EndTime;
            foreach (ETFConstituentUniverse constituent in constituents)
            {
                var symbol = constituent.Symbol;
                var weight = constituent.Weight;
            }
        }
        // Add a Scheduled Event to rebalance the portfolio each day.
        Schedule.On(DateRules.EveryDay(_spy), TimeRules.AfterMarketOpen(_spy, 1), Rebalance);
    }
    
    private IEnumerable<Symbol> SelectAssets(IEnumerable<ETFConstituentUniverse> constituents)
    {
        // Select the 10 largest stocks in the SPY. 
        _weightBySymbol = constituents.OrderByDescending(c => c.Weight).Take(10)
            // Save the weights for position sizing.
            .ToDictionary(c => c.Symbol, c => c.Weight ?? 0m);
        // Return the selected assets.
        return _weightBySymbol.Keys;
    }
    
    private void Rebalance()
    {
        // Create a long-short portfolio to earn the excess return of the 
        // top 10 weighted stocks from SPY.
        var spyWeight = _weightBySymbol.Values.Sum();
        var targets = new List<PortfolioTarget>() { new PortfolioTarget(_spy, -0.5m) };
        foreach (var kvp in _weightBySymbol)
        {
            targets.Add(new PortfolioTarget(kvp.Key, 0.5m * kvp.Value / spyWeight));
        }
        // Liquidate the stocks that aren't in top 10.
        SetHoldings(targets, true);
    }
}

Framework Algorithm Example

The following example algorithm creates a dynamic universe of the 10 largest US Equities in the SPY ETF. Each day, the algorithm forms a dollar-neutral and market-neutral portfolio by buying the 10 ETF constituents and shorting the SPY ETF.

class ETFConstituentUniverseFrameworkAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 9, 1)
        self.set_end_date(2024, 12, 31)
        self.set_cash(100000)
        self.universe_settings.asynchronous = True
        self.universe_settings.resolution = Resolution.MINUTE
        self.weight_by_symbol = {}
        
        # Add universe selection on SPY's constituents to select only from large cap stocks
        # Save the universe to access its members for historical data call
        spy = self.add_equity("SPY").symbol
        self.add_universe_selection(ETFConstituentsUniverseSelectionModel(spy, self.universe_settings, self.etf_constituents_filter))

        # Add alpha model that set normalized weight as investment insight
        self.add_alpha(ETFConstituentsAlphaModel(self, spy))

        # Set up portfolio construction model that invest by the insight weights
        pcm = InsightWeightingPortfolioConstructionModel()
        # Avoid excessive rebalance on insight changes
        pcm.rebalance_on_insight_changes = False
        self.set_portfolio_construction(pcm)

        self.add_risk_management(NullRiskManagementModel())

        self.set_execution(ImmediateExecutionModel())

    def etf_constituents_filter(self, constituents: List[ETFConstituentUniverse]) -> List[Symbol]:
        # The top 10 weighted securities are considered better active selections
        # Save the weights for position sizing
        selected = sorted([c for c in constituents if c.weight],
            key=lambda c: c.weight, reverse=True)[:10]
        self.weight_by_symbol = {c.symbol: c.weight for c in selected}

        return list(self.weight_by_symbol.keys())

class ETFConstituentsAlphaModel(AlphaModel):

    def __init__(self, algorithm: QCAlgorithm, etf: Symbol) -> None:
        self.algorithm = algorithm
        self.etf = etf
        self.day = -1

    def update(self, algorithm: QCAlgorithm, slice: Slice) -> List[Insight]:
        # Rebalance daily since selection is on daily basis
        if self.day == algorithm.time.day:
            return []

        self.day = algorithm.time.day

        insights = []

        # Create a long-short portfolio to earn excess return of the top 10 weighted stocks from SPY
        etf_weight = sum(self.algorithm.weight_by_symbol.values())
        if etf_weight> 0:
            # Invest half the portfolio by normalized weights of the top 10 constituents
            for symbol, weight in self.algorithm.weight_by_symbol.items():
                if algorithm.securities.contains_key(symbol):
                    insights.append(Insight.price(symbol, Expiry.END_OF_DAY, InsightDirection.UP, weight=0.5*weight/etf_weight))
            
            # Short the other half with SPY, looking to profit from the active selection
            insights.append(Insight.price(self.etf, Expiry.END_OF_DAY, InsightDirection.DOWN, weight=0.5))

        return insights

    def on_securities_changed(self, algorithm: QCAlgorithm, changes: SecurityChanges) -> None:
        # Liquidate the ones not in top 10 weights
        symbols = [x.symbol for x in changes.removed_securities if x.invested]
        algorithm.liquidate(symbols, tag='Removed From Universe')
public class ETFConstituentUniverseFrameworkAlgorithm : QCAlgorithm
{ 
    public Dictionary<Symbol, decimal> WeightBySymbol = [];
    
    public override void Initialize()
    {
        SetStartDate(2024, 9, 1);
        SetEndDate(2024, 12, 31);
        SetCash(100000);
        UniverseSettings.Asynchronous = true;
        UniverseSettings.Resolution = Resolution.Minute;

        // Add universe selection on SPY's constituents to select only from large cap stocks
        // Save the universe to access its members for historical data call
        var spy = AddEquity("SPY").Symbol;
        AddUniverseSelection(new ETFConstituentsUniverseSelectionModel(spy, UniverseSettings, ETFConstituentsFilter));
        
        // Add alpha model that set normalized weight as investment insight
        AddAlpha(new ETFConstituentsAlphaModel(this, spy));

        // Set up portfolio construction model that invest by the insight weights
        SetPortfolioConstruction(new InsightWeightingPortfolioConstructionModel
        {
            // Avoid excessive rebalance on insight changes
            RebalanceOnInsightChanges = false
        });
        
        AddRiskManagement(new NullRiskManagementModel());
        
        SetExecution(new ImmediateExecutionModel());
    }
    
    private IEnumerable<Symbol> ETFConstituentsFilter(IEnumerable<ETFConstituentUniverse> constituents)
    {
        // The top 10 weighted securities are considered better active selections
        // Save the weights for position sizing
        WeightBySymbol = constituents.Where(c=> c.Weight.HasValue).OrderByDescending(c => c.Weight).Take(10)
            .ToDictionary(c => c.Symbol, c => c.Weight.Value);

        return WeightBySymbol.Keys;
    }
}

public class ETFConstituentsAlphaModel : AlphaModel
{
    private int _day = -1;
    private Symbol _etf;
    private ETFConstituentUniverseFrameworkAlgorithm _algorithm;
    
    public ETFConstituentsAlphaModel(ETFConstituentUniverseFrameworkAlgorithm algorithm, Symbol etf)
    {
        _etf = etf;
        _algorithm = algorithm;
    }
    
    public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice slice)
    {
        // Rebalance daily since selection is on daily basis
        if (_day == algorithm.Time.Day)
        {
            return [];
        }
        
        _day = algorithm.Time.Day;
        
        var insights = new List<Insight>();
        
        // Create a long-short portfolio to earn excess return of the top 10 weighted stocks from SPY
        var etfWeight = (double)_algorithm.WeightBySymbol.Values.Sum();
        if (etfWeight > 0)
        {
            // Invest half the portfolio by normalized weights of the top 10 constituents
            foreach(var kvp in _algorithm.WeightBySymbol)
            {
                insights.Add(Insight.Price(kvp.Key, Expiry.EndOfDay, InsightDirection.Up, weight: (double)kvp.Value/etfWeight * 0.5));
            }

            // Short the other half with SPY, looking to profit from the active selection
            insights.Add(Insight.Price(_etf, Expiry.EndOfDay, InsightDirection.Down, weight: 0.5));
        }
        
        return insights;
    }
    
    public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes)
    {
        // Liquidate the ones not in top 10 weights
        var symbols = changes.RemovedSecurities.Where(x => x.Invested).Select(x => x.Symbol);
        algorithm.Liquidate(symbols, tag: "Removed From Universe");
    }
}

Research Example

The following example lists ETF constituents with the greatest weight in the SPY:

var qb = new QuantBook();

// Add the ETF
var symbol = qb.AddEquity("SPY").Symbol;

// Add ETF Universe Selection
IEnumerable<Symbol> ETFConstituentsFilter(IEnumerable<ETFConstituentUniverse> constituents)
{
    // Take the top 10 weighted constituents
    return constituents
        .OrderByDescending(c => c.Weight)
        .Take(10)
        .Select(c => c.Symbol);
}

var universe = qb.AddUniverse(qb.Universe.ETF(spy, qb.UniverseSettings, ETFConstituentsFilter)); 

// Historical Universe data
var universeHistory = qb.UniverseHistory(universe, qb.Time.AddDays(-30), qb.Time);
foreach (var constituents in universeHistory )
{
    foreach (ETFConstituentUniverse constituent in constituents)
    {
        Console.WriteLine($"{constituent.Symbol} weight at {constituent.EndTime}: {constituent.Weight}");
    }
}
qb = QuantBook()

# Add the ETF
qb.spy = qb.add_equity("SPY").symbol

# Add ETF Universe Selection
def etf_constituents_filter(constituents):
    # Take the top 10 weighted constituents
    selected = sorted([c for c in constituents if c.weight],
        key=lambda c: c.weight, reverse=True)[:10]
    return [c.symbol for c in selected]

universe = qb.add_universe(qb.universe.etf(qb.spy, qb.universe_settings, etf_constituents_filter))

# Historical Universe data
universe_history = qb.universe_history(universe, qb.time-timedelta(30), qb.time)
for (universe_symbol, time), constituents in universe_history.items():
    for constituent in constituents:
        print(f"{constituent.symbol} weight at {constituent.end_time}: {constituent.weight}")

Data Point Attributes

The ETF Constituents dataset provides ETFConstituentUniverse objects, which have the following attributes:

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: