Universes

Equity

Introduction

There are several ways to create an Equities universe. You can select a universe based on CoarseFundamental data or the constituents of an ETF, and then you can further filter your universe down with corporate fundamentals. The following sections explain each of these techniques in detail.

Coarse Universe Selection

A coarse universe enables you pick a set of stocks based on their trading volume, price, or whether they have fundamental data. To add a coarse universe, in the Initialize method, pass a filter function to the AddUniverse method. The coarse filter function receives a list of CoarseFundamental objects and must return a list of Symbol objects. The Symbol objects you return from the function are the constituents of the universe and LEAN automatically creates subscriptions for them. Don't call AddEquity in the filter function.

public class MyCoarseUniverseAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        AddUniverse(CoarseFilterFunction);
    }

    private IEnumerable<Symbol> CoarseFilterFunction(IEnumerable<CoarseFundamental> coarse)
    {
        return (from c in coarse
            orderby c.DollarVolume descending
            select c.Symbol).Take(100);
    }
}
class MyCoarseUniverseAlgorithm(QCAlgorithm):
    def Initialize(self) -> None:
        self.AddUniverse(self.CoarseFilterFunction)

    def CoarseFilterFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
        sorted_by_dollar_volume = sorted(coarse, key=lambda x: x.DollarVolume, reverse=True) 
        return [c.Symbol for c in sorted_by_dollar_volume[:100]]

CoarseFundamental objects have the following attributes:

The total number of stocks in the US Equity Security Master dataset is 30,000 but your coarse filter function won't receive all of these at one time because the US Equity Security Master dataset is free of survivorship bias and some of the securities have delisted over time. The number of securities that are passed into your coarse filter function depends on the date of your algorithm. Currently, there are about 10,000 securities that LEAN passes into your coarse filter function.

Dollar Volume Selection

A dollar volume universe enables you pick the most liquid stocks in the market with just one line of code. To add a dollar volume universe, call the Universe.DollarVolume.Top helper method and pass the result to the AddUniverse method.

// Add the 50 stocks with the highest dollar volume
AddUniverse(Universe.DollarVolume.Top(50));
// Add the 50 stocks with the highest dollar volume
self.AddUniverse(self.Universe.DollarVolume.Top(50))

ETF Constituents Selection

An ETF constituents universe lets you select a universe of securities in an ETF. The US ETF Constituents dataset includes 2,650 US ETFs you can use to create your universe. To add an ETF Constituents universe, call the Universe.ETF method.

public class ETFConstituentsAlgorithm : QCAlgorithm
{
    public override void Initialize() 
    {
        var universe = Universe.ETF("SPY", Market.USA, UniverseSettings, 
                                    constituents => from x in constituents select x.Symbol);
        AddUniverse(universe);
    }
}
class ETFConstituentsAlgorithm(QCAlgorithm):
    def Initialize(self) -> None:        
        filter_function = lambda constituents: [x.Symbol for x in constituents]
        universe = self.Universe.ETF("SPY", Market.USA, self.UniverseSettings, filter_function)
        self.AddUniverse(universe)

The following table describes the ETF method arguments:

ArgumentData TypeDescriptionDefault Value
etfTickerstringstrThe ETF ticker.
marketstringstrThe market of the ETF. If you don't provide an argument, it uses the default Equity market of the brokerage model.nullNone
universeSettingsUniverseSettingsThe universe settings. If you don't provide an argument, it uses the algorithm UniverseSettings.nullNone
universeFilterFuncFunc<IEnumerable<ETFConstituentData>, IEnumerable<Symbol>>Callable[[List[ETFConstituentData]], List[Symbol]]A function to select some of the ETF constituents for the universe. If you don't provide an argument, it selects all of the constituents.
nullNone

The filter function receives ETFConstituentData objects, which represent one of the ETF constituents. ETFConstituentsData objects have the following attributes:

To perform thorough filtering on the ETFConstituentsData objects, define an isolated filter method.

public class ETFConstituentsAlgorithm : QCAlgorithm 
{
    public override void Initialize() 
    {
        var universe = Universe.ETF("SPY", Market.USA, UniverseSettings, ETFConstituentsFilter);
        AddUniverse(universe);
    }

    private IEnumerable<Symbol> ETFConstituentsFilter(IEnumerable<ETFConstituentData> constituents)
    {
        // Get the 10 securities with the largest weight in the index
        return constituents.OrderByDescending(c => c.Weight).Take(10).Select(c => c.Symbol);
    }
}
class ETFConstituentsAlgorithm(QCAlgorithm):
    def Initialize(self) -> None:
        universe = self.Universe.ETF("SPY", Market.USA, self.UniverseSettings, self.ETFConstituentsFilter)
        self.AddUniverse(universe)

    def ETFConstituentsFilter(self, constituents: List[ETFConstituentData]) -> List[Symbol]:
        # Get the 10 securities with the largest weight in the index
        selected = sorted([c for c in constituents if c.Weight],
            key=lambda c: c.Weight, reverse=True)[:10]
        return [c.Symbol for c in selected]

Fundamentals Selection

A fundamental universe lets you select stocks based on corporate fundamental data. This data is powered by MorningstarĀ® and includes approximately 5,000 tickers with 900 properties each. Due to the sheer volume of information, fundamental selection is performed on the output of another universe filter. Think of this process as a 2-stage filter. An initial filter function selects a set of stocks and then a fine fundamental filter function selects a subset of those stocks.

QuantConnect Coarse and Fine Universe Selection

To add a fundamental universe, in the Initialize method, pass two filter functions to the AddUniverse method. The first filter function can be a coarse universe filter, dollar volume filter, or an ETF constituents filter. The second filter function receives a list of FineFundamental objects and must return a list of Symbol objects. The list of FineFundamental objects contains a subset of the Symbol objects that the first filter function returned. The Symbol objects you return from the second function are the constituents of the fundamental universe and LEAN automatically creates subscriptions for them. Don't call AddEquity in the filter function.

Tip:

Only 5,000 assets have fundamental data. If your first filter function receives CoarseFundamental data, you should only select assets that have a true value for their HasFundamentalData property.

public class MyUniverseAlgorithm : QCAlgorithm {
    public override void Initialize() 
    {
        AddUniverse(CoarseFilterFunction, FineFundamentalFilterFunction);
    }
    // filter based on CoarseFundamental
    IEnumerable<Symbol> CoarseFilterFunction(IEnumerable<CoarseFundamental> coarse) 
    {
         // In addition to further coarse universe selection, ensure the security has fundamental data
         return (from c in coarse
             where c.HasFundamentalData
             select c.Symbol);
    }
    // filter based on FineFundamental
    public IEnumerable<Symbol> FineFundamentalFilterFunction(IEnumerable<FineFundamental> fine)
    {
        // Return a list of Symbols
    }
}
class MyUniverseAlgorithm(QCAlgorithm):
    def Initialize(self) -> None:
        self.AddUniverse(self.CoarseFilterFunction, self.FineFundamentalFunction)

    def CoarseFilterFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
        # In addition to further coarse universe selection, ensure the security has fundamental data
        return [c.Symbol for c in coarse if c.HasFundamentalData]

    def FineFundamentalFunction(self, fine: List[FineFundamental]) -> List[Symbol]:
        # Return a list of Symbols

FineFundamental objects have the following attributes:

Example

The simplest example of accessing the fundamental object would be harnessing the iconic PE ratio for a stock. This is a ratio of the price it commands to the earnings of a stock. The lower the PE ratio for a stock, the more affordable it appears.

// Take the top 50 by dollar volume using coarse
// Then the top 10 by PERatio using fine
AddUniverse(
    coarse => {
        return (from c in coarse
            where c.Price > 10 && c.HasFundamentalData
            orderby c.DollarVolume descending
            select c.Symbol).Take(50);
    },
    fine => {
        return (from f in fine
            orderby f.ValuationRatios.PERatio ascending
            select f.Symbol).Take(10);
    });
# In Initialize:
self.AddUniverse(self.CoarseSelectionFunction, self.FineSelectionFunction)

def CoarseSelectionFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
    sortedByDollarVolume = sorted(coarse, key=lambda x: x.DollarVolume, reverse=True)
    filtered = [x.Symbol for x in sortedByDollarVolume if x.HasFundamentalData]
    return filtered[:50]

def FineSelectionFunction(self, fine: List[FineFundamental]) -> List[Symbol]:
    sortedByPeRatio = sorted(fine, key=lambda x: x.ValuationRatios.PERatio, reverse=False)
    return [x.Symbol for x in sortedByPeRatio[:10]]

Asset Categories

In addition to valuation ratios, the US Fundamental Data from Morningstar has many other data point attributes, including over 200 different categorization fields for each US stock. Morningstar groups these fields into sectors, industry groups, and industries.

Sectors are large super categories of data. To get the sector of a stock, use the MorningstarSectorCode property.

var tech = fine.Where(x => x.AssetClassification.MorningstarSectorCode == MorningstarSectorCode.Technology);
tech = [x for x in fine if x.AssetClassification.MorningstarSectorCode == MorningstarSectorCode.Technology]

Industry groups are clusters of related industries that tie together. To get the industry group of a stock, use the MorningstarIndustryGroupCode property.

var ag = fine.Where(x => x.AssetClassification.MorningstarIndustryGroupCode == MorningstarIndustryGroupCode.Agriculture);
ag = [x for x in fine if x.AssetClassification.MorningstarIndustryGroupCode == MorningstarIndustryGroupCode.Agriculture]

Industries are the finest level of classification available. They are the individual industries according to the Morningstar classification system. To get the industry of a stock, use the MorningstarIndustryCode.

var coal = fine.Where(x => x.AssetClassification.MorningstarIndustryCode == MorningstarSectorCode.Coal);
coal = [x for x in fine if x.AssetClassification.MorningstarIndustryCode == MorningstarSectorCode.Coal]

Practical Limitations

Like coarse universes, fine universes allow you to select an unlimited universe of assets to analyze. Each asset in the universe consumes approximately 5MB of RAM, so you may quickly run out of memory if your universe filter selects many assets. If you backtest your algorithms in the Algorithm Lab, familiarize yourself with the RAM capacity of your backtesting and live trading nodes. To keep your algorithm fast and efficient, only subscribe to the assets you need.

Selection Frequency

Equity universes run on a daily basis.

Live Trading Considerations

The live data for coarse and fine universe selection arrives at 7 AM Eastern Time (ET), so coarse and fine universe selection runs for live algorithms between 7 and 8 AM ET. This timing allows you to place trades before the market opens. Don't schedule anything for midnight because the universe selection data isn't ready yet.

Examples

The following examples are typical filter functions you may want.

Example 1: Take 500 stocks that are worth more than $10 and have more than $10M daily trading volume

The most common use case is to select a lot of liquid stocks. With a coarse universe filter, this is simple and fast. The following example selects the top most liquid 500 stocks over $10 per share.

IEnumerable<Symbol> CoarseFilterFunction(IEnumerable<CoarseFundamental> coarse) 
{
    // Linq makes this a piece of cake;
    return (from c in coarse
        where c.DollarVolume > 10000000 &&
            c.Price > 10
        orderby c.DollarVolume descending
        select c.Symbol).Take(500);
}
def CoarseFilterFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
    sortedByDollarVolume = sorted(coarse, key=lambda x: x.DollarVolume, reverse=True)
    filtered = [ x.Symbol for x in sortedByDollarVolume 
                if x.Price > 10 and x.DollarVolume > 10000000 ]
    return filtered[:500]

Example 2: Take 10 stocks above their 200-Day EMA and have more than $1B daily trading volume

Another common request is to filter the universe by a technical indicator, such as only picking stocks above their 200-day EMA. The CoarseFundamental object has adjusted price and volume information, so you can do any price-related analysis.

ConcurrentDictionary<Symbol, SelectionData>
    _stateData = new ConcurrentDictionary<Symbol, SelectionData>();

// Coarse filter function
IEnumerable<Symbol> CoarseFilterFunction(IEnumerable<CoarseFundamental> coarse) {
    // Linq makes this a piece of cake;
    return (from c in coarse
        let avg = _stateData.GetOrAdd(c.Symbol, sym => new SelectionData(200))
        where avg.Update(c.EndTime, c.AdjustedPrice)
        where c.DollarVolume > 1000000000 &&
                c.Price > avg.Ema
        orderby c.DollarVolume descending
        select c.Symbol).Take(10);
}
# setup state storage in initialize method
self.stateData = { }

def CoarseFilterFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
    # We are going to use a dictionary to refer the object that will keep the moving averages
    for c in coarse:
        if c.Symbol not in self.stateData:
            self.stateData[c.Symbol] = SelectionData(c.Symbol, 200)

        # Updates the SymbolData object with current EOD price
        avg = self.stateData[c.Symbol]
        avg.update(c.EndTime, c.AdjustedPrice, c.DollarVolume)

    # Filter the values of the dict to those above EMA and more than $1B vol.
    values = [x for x in self.stateData.values() if x.is_above_ema and x.volume > 1000000000]
    
    # sort by the largest in volume.
    values.sort(key=lambda x: x.volume, reverse=True)

    # we need to return only the symbol objects
    return [ x.symbol for x in values[:10] ]

In this example, the SelectionData class group variables for the universe selection and updates the indicator of each asset. We highly recommend you follow this pattern to keep your algorithm tidy and bug free. The following snippet shows an example implementation of the SelectionData class, but you can make this whatever you need to store your custom universe filters.

class SelectionData(object):
    def __init__(self, symbol, period):
        self.symbol = symbol
        self.ema = ExponentialMovingAverage(period)
        self.is_above_ema = False
        self.volume = 0

    def update(self, time, price, volume):
        self.volume = volume
        if self.ema.Update(time, price):
            self.is_above_ema = price > ema
// example selection data class
private class SelectionData
{
    // variables you need for selection
    public readonly ExponentialMovingAverage Ema;

    // initialize your variables and indicators.
    public SelectionData(int period)
    {
        Ema = new ExponentialMovingAverage(period);
    }

    // update your variables and indicators with the latest data.
    // you may also want to use the History API here.
    public bool Update(DateTime time, decimal value)
    {
        return Ema.Update(time, value);
    }
}

Note that the preceding SelectionData class uses a manual EMA indicator instead of the automatic version. For more information about universes that select assets based on indicators, see Indicator Universes.

Example 3: Take 10 stocks that are the furthest above their 10-day SMA of volume

The process to get the 10-day SMA stock volume is the same process as in Example 2. First, you should define a SelectionData class that performs the averaging. For this example, the following class will serve this purpose:

class SelectionData(object):
    def __init__(self, symbol, period):
        self.symbol = symbol
        self.volume = 0
        self.volume_ratio = 0
        self.sma = SimpleMovingAverage(period)

    def update(self, time, price, volume):
        self.volume = volume
        if self.sma.Update(time, volume):
            # get ratio of this volume bar vs previous 10 before it.
            self.volume_ratio = volume / self.sma.Current.Value
private class SelectionData
{
    public readonly Symbol Symbol;
    public readonly SimpleMovingAverage VolumeSma;
    public decimal VolumeRatio;
    public SelectionData(Symbol symbol, int period)
    {
        Symbol = symbol;
        VolumeSma = new SimpleMovingAverage(period);
    }
    public bool Update(DateTime time, decimal value)
    {
        var ready = VolumeSma.Update(time, value);
        VolumeRatio = value / VolumeSma;
        return ready;
    }
}

This class tracks the ratio of today's volume relative to historical volumes. You can use this ratio to select assets that are above their 10-day simple moving average and sort the results by the ones that have had the biggest jump since yesterday.

def CoarseFilterFunction(self, coarse: List[CoarseFundamental]) -> List[Symbol]:
    for c in coarse:
        if c.Symbol not in self.stateData:
            self.stateData[c.Symbol] = SelectionData(c.Symbol, 10)
        avg = self.stateData[c.Symbol]
        avg.update(c.EndTime, c.AdjustedPrice, c.DollarVolume)

    # filter the values of selectionData(sd) above SMA
    values = [sd for sd in self.stateData.values() if sd.volume > sd.sma.Current.Value and sd.volume_ratio > 0]
        
    # sort sd by the largest % jump in volume.
    values.sort(key=lambda sd: sd.volume_ratio, reverse=True)

    # return the top 10 symbol objects
    return [ sd.symbol for sd in values[:10] ]
 
IEnumerable<Symbol> CoarseFilterFunction(IEnumerable<CoarseFundamental> coarse) 
{ return (from c in coarse let avg = _stateData.GetOrAdd(c.Symbol, sym => new SelectionData(10)) where avg.Update(c.EndTime, c.Volume) where c.Volume > avg.VolumeSma orderby avg.VolumeRatio descending select c.Symbol).Take(10); }

Example 4: Take the top 10 "fastest moving" stocks with a 50-Day EMA > 200 Day EMA

You can construct complex universe filters with the SelectionData helper class pattern. To view a full example of this algorithm, see the EmaCrossUniverseSelectionAlgorithmEmaCrossUniverseSelectionAlgorithm in the LEAN GitHub repository or take the related Boot Camp lesson.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: