Financial Industry Regulatory Authority

OTC Transparency

Introduction

The FINRA OTC Transparency dataset by FINRA (the Financial Industry Regulatory Authority) shows you what US stocks are trading away from the exchanges, plus how heavily each one is being shorted. It covers around 59,000 US Equities, goes back to January 2017, and updates daily. FINRA builds it from the off-exchange volume and short interest reports that broker-dealers have to file with them.

For more information about the OTC Transparency dataset, including CLI commands and pricing, see the dataset listing.

About the Provider

FINRA (the Financial Industry Regulatory Authority) was created in 2007, when NASD merged with the member regulation arm of the New York Stock Exchange. Its job is to protect investors and keep the markets honest, and it oversees nearly every broker-dealer that does business with the public in the United States. As part of that role, FINRA collects the off-exchange trading and short interest reports those firms are required to file, and publishes the results for free so researchers, investors, and anyone else can get a clearer view of what is happening away from the exchanges.

Getting Started

The following snippet demonstrates how to request data from the FINRA OTC Transparency dataset:

self.symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
self.short_interest = self.add_data(FINRAShortInterest, self.symbol).symbol
self.weekly_summary = self.add_data(FINRAWeeklySummary, self.symbol).symbol
self.monthly_summary = self.add_data(FINRAMonthlySummary, self.symbol).symbol
_symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
_shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
_weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
_monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;

Data Summary

The following table describes the dataset properties:

PropertyValue
Start DateJanuary 2017
Data Points9,360,438
Asset Coverage59,712 US Equities
ResolutionDaily
TimezoneAmerica/New_York

Requesting Data

To add FINRA OTC Transparency data to your algorithm, call the AddDataadd_data method. The dataset is linked to US Equities, so pass the equity Symbol you want the data for. Save a reference to the dataset Symbol so you can access the data later in your algorithm.

The dataset comes in three classes: FINRAShortInterest, FINRAWeeklySummary, and FINRAMonthlySummary. Subscribe to the ones your strategy needs.

class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        self.symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
        self.short_interest = self.add_data(FINRAShortInterest, self.symbol).symbol
        self.weekly_summary = self.add_data(FINRAWeeklySummary, self.symbol).symbol
        self.monthly_summary = self.add_data(FINRAMonthlySummary, self.symbol).symbol
public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
    private Symbol _symbol, _shortInterest, _weeklySummary, _monthlySummary;

    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        _symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
        _shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
        _weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
        _monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;
    }
}

Accessing Data

To get the current FINRA OTC Transparency data, index the current Slice with the dataset Symbol. Slice objects deliver unique events to your algorithm as they happen, but the Slice may not contain data for your dataset at every time step. To avoid issues, check if the Slice contains the data you want before you index it.

def on_data(self, slice: Slice) -> None:
    if slice.contains_key(self.short_interest):
        data_point = slice[self.short_interest]
        self.log(f"{self.short_interest} days to cover at {slice.time}: {data_point.days_to_cover}")
public override void OnData(Slice slice)
{
    if (slice.ContainsKey(_shortInterest))
    {
        var dataPoint = slice[_shortInterest];
        Log($"{_shortInterest} days to cover at {slice.Time}: {dataPoint.DaysToCover}");
    }
}

To iterate through all of the dataset objects in the current Slice, call the Get method.

def on_data(self, slice: Slice) -> None:
    for symbol, data_point in slice.get(FINRAShortInterest).items():
        self.log(f"{symbol} days to cover at {slice.time}: {data_point.days_to_cover}")

    for symbol, data_point in slice.get(FINRAWeeklySummary).items():
        self.log(f"{symbol} ATS weekly shares at {slice.time}: {data_point.share_quantity}")

    for symbol, data_point in slice.get(FINRAMonthlySummary).items():
        self.log(f"{symbol} non-ATS monthly shares at {slice.time}: {data_point.share_quantity}")
public override void OnData(Slice slice)
{
    foreach (var kvp in slice.Get<FINRAShortInterest>())
    {
        Log($"{kvp.Key} days to cover at {slice.Time}: {kvp.Value.DaysToCover}");
    }

    foreach (var kvp in slice.Get<FINRAWeeklySummary>())
    {
        Log($"{kvp.Key} ATS weekly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
    }

    foreach (var kvp in slice.Get<FINRAMonthlySummary>())
    {
        Log($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
    }
}

Historical Data

To get historical FINRA OTC Transparency data, call the History method with the dataset Symbol. If there is no data in the period you request, the history result is empty.

# DataFrames
short_interest_history = self.history(self.short_interest, 100, Resolution.DAILY)
weekly_history = self.history(self.weekly_summary, 100, Resolution.DAILY)
monthly_history = self.history(self.monthly_summary, 100, Resolution.DAILY)
history = self.history([self.short_interest, self.weekly_summary, self.monthly_summary], 100, Resolution.DAILY)

# Dataset objects
short_interest_bars = self.history[FINRAShortInterest](self.short_interest, 100, Resolution.DAILY)
weekly_bars = self.history[FINRAWeeklySummary](self.weekly_summary, 100, Resolution.DAILY)
monthly_bars = self.history[FINRAMonthlySummary](self.monthly_summary, 100, Resolution.DAILY)
// Dataset objects
var shortInterestHistory = History<FINRAShortInterest>(_shortInterest, 100, Resolution.Daily);
var weeklyHistory = History<FINRAWeeklySummary>(_weeklySummary, 100, Resolution.Daily);
var monthlyHistory = History<FINRAMonthlySummary>(_monthlySummary, 100, Resolution.Daily);

// Slice objects
var history = History(new[] { _shortInterest, _weeklySummary, _monthlySummary }, 100, Resolution.Daily);

For more information about historical data, see History Requests.

Universe Selection

To select a dynamic universe of US Equities based on FINRA OTC Transparency data, call the AddUniverseadd_universe method with one of the dataset's universe classes (FINRAShortInterestUniverse, FINRAWeeklySummaryUniverse, or FINRAMonthlySummaryUniverse) and a selection function. The following example selects the most heavily shorted names by days-to-cover:

def initialize(self) -> None:
    self._universe = self.add_universe(FINRAShortInterestUniverse, self.universe_selection)

def universe_selection(self, alt_coarse: List[FINRAShortInterestUniverse]) -> List[Symbol]:
    return [d.symbol for d in alt_coarse
            if d.days_to_cover is not None and d.days_to_cover > 5]
private Universe _universe;
public override void Initialize()
{
    _universe = AddUniverse<FINRAShortInterestUniverse>(altCoarse=>
    {
        return from d in altCoarse.OfType<FINRAShortInterestUniverse>()
            where d.DaysToCover > 5m
            select d.Symbol;
    });
}

For more information about dynamic universes, see Universes.

Remove Subscriptions

To remove a subscription, call the RemoveSecurity method.

self.remove_security(self.short_interest)
self.remove_security(self.weekly_summary)
self.remove_security(self.monthly_summary)
RemoveSecurity(_shortInterest);
RemoveSecurity(_weeklySummary);
RemoveSecurity(_monthlySummary);

If you subscribe to FINRA OTC Transparency data for assets in a dynamic universe, remove the dataset subscription when the asset leaves your universe. To view a common design pattern, see Track Security Changes.

Example Applications

The FINRA OTC Transparency dataset lets you see the short positioning and off-exchange order flow behind a stock. Examples include:

  • Hunting for short squeeze candidates by ranking the universe on days-to-cover, the short interest divided by average daily volume.
  • Following or fading a stock as its open short interest builds up or unwinds from one reporting period to the next.
  • Reading institutional accumulation or distribution from a shift in how much of a stock's volume prints on dark pools, using the weekly ATS shares.
  • Combining the weekly ATS and monthly non-ATS volume into a total off-exchange participation measure and trading the names where it breaks from their norm.

Classic Algorithm Example

The following example algorithm subscribes to the three FINRA OTC Transparency report types for a US Equity and trades it on the change in open short interest:

from AlgorithmImports import *

class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        aapl = self.add_equity("AAPL", Resolution.DAILY).symbol
        self.short_interest = self.add_data(FINRAShortInterest, aapl).symbol
        self.weekly_summary = self.add_data(FINRAWeeklySummary, aapl).symbol
        self.monthly_summary = self.add_data(FINRAMonthlySummary, aapl).symbol

        history = self.history(FINRAShortInterest, self.short_interest, 30, Resolution.DAILY)
        self.debug(f"We got {len(history)} items from our history request")

    def on_data(self, slice):
        # Off-exchange volume prints as an activity signal
        for symbol, point in slice.get(FINRAMonthlySummary).items():
            self.debug(f"{symbol} non-ATS monthly shares at {slice.time}: {point.share_quantity}")

        for symbol, point in slice.get(FINRAShortInterest).items():
            if point.change is None:
                continue

            # Go short when open short interest is building; go long when it is unwinding
            if point.change > 0:
                self.set_holdings(symbol.underlying, -1)
            else:
                self.set_holdings(symbol.underlying, 1)
public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
    private Symbol _shortInterest, _weeklySummary, _monthlySummary;

    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        var aapl = AddEquity("AAPL", Resolution.Daily).Symbol;
        _shortInterest = AddData<FINRAShortInterest>(aapl).Symbol;
        _weeklySummary = AddData<FINRAWeeklySummary>(aapl).Symbol;
        _monthlySummary = AddData<FINRAMonthlySummary>(aapl).Symbol;

        var history = History<FINRAShortInterest>(_shortInterest, 30, Resolution.Daily);
        Debug($"We got {history.Count()} items from our history request");
    }

    public override void OnData(Slice slice)
    {
        // Off-exchange volume prints as an activity signal
        foreach (var kvp in slice.Get<FINRAMonthlySummary>())
        {
            Debug($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
        }

        foreach (var kvp in slice.Get<FINRAShortInterest>())
        {
            var point = kvp.Value;
            if (point.Change == null)
            {
                continue;
            }

            // Go short when open short interest is building; go long when it is unwinding
            if (point.Change > 0)
            {
                SetHoldings(kvp.Key.Underlying, -1);
            }
            else
            {
                SetHoldings(kvp.Key.Underlying, 1);
            }
        }
    }
}

Framework Algorithm Example

The following example algorithm uses the Algorithm Framework to trade a manually selected universe from FINRA OTC Transparency data. An alpha model subscribes each security to the consolidated short interest feed and emits insights from the change in open short interest:

from AlgorithmImports import *

class FINRAOtcTransparencyFrameworkAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        self.universe_settings.resolution = Resolution.DAILY

        symbols = [Symbol.create("AAPL", SecurityType.EQUITY, Market.USA)]
        self.set_universe_selection(ManualUniverseSelectionModel(symbols))

        self.add_alpha(FINRAShortInterestAlphaModel())

        self.set_portfolio_construction(EqualWeightingPortfolioConstructionModel())
        self.set_execution(ImmediateExecutionModel())


class FINRAShortInterestAlphaModel(AlphaModel):

    def __init__(self) -> None:
        self._symbols_by_security = {}

    def update(self, algorithm: QCAlgorithm, data: Slice) -> List[Insight]:
        insights = []
        for security_symbol, short_interest_symbol in self._symbols_by_security.items():
            if short_interest_symbol not in data:
                continue
            change = data[short_interest_symbol].change
            if change is None:
                continue
            # Building short interest is bearish, unwinding short interest is bullish.
            direction = InsightDirection.DOWN if change > 0 else InsightDirection.UP
            insights.append(Insight.price(security_symbol, timedelta(days=14), direction))
        return insights

    def on_securities_changed(self, algorithm: QCAlgorithm, changes: SecurityChanges) -> None:
        for security in changes.added_securities:
            symbol = security.symbol
            short_interest_symbol = algorithm.add_data(FINRAShortInterest, symbol).symbol
            self._symbols_by_security[symbol] = short_interest_symbol
            history = algorithm.history(FINRAShortInterest, short_interest_symbol, 10, Resolution.DAILY)
            algorithm.debug(f"Got {len(history)} historical short interest rows for {symbol}")
        for security in changes.removed_securities:
            short_interest_symbol = self._symbols_by_security.pop(security.symbol, None)
            if short_interest_symbol is not None:
                algorithm.remove_security(short_interest_symbol)
public class FINRAOtcTransparencyFrameworkAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        UniverseSettings.Resolution = Resolution.Daily;

        var symbols = new[] { QuantConnect.Symbol.Create("AAPL", SecurityType.Equity, Market.USA) };
        SetUniverseSelection(new ManualUniverseSelectionModel(symbols));

        AddAlpha(new FINRAShortInterestAlphaModel());

        SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel());
        SetExecution(new ImmediateExecutionModel());
    }
}

public class FINRAShortInterestAlphaModel : AlphaModel
{
    private readonly Dictionary<Symbol, Symbol> _symbolsBySecurity = new();

    public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice data)
    {
        foreach (var kvp in _symbolsBySecurity)
        {
            var securitySymbol = kvp.Key;
            var shortInterestSymbol = kvp.Value;
            if (!data.ContainsKey(shortInterestSymbol))
            {
                continue;
            }
            var change = data.Get<FINRAShortInterest>(shortInterestSymbol).Change;
            if (change == null)
            {
                continue;
            }
            // Building short interest is bearish, unwinding short interest is bullish.
            var direction = change > 0 ? InsightDirection.Down : InsightDirection.Up;
            yield return Insight.Price(securitySymbol, TimeSpan.FromDays(14), direction);
        }
    }

    public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes)
    {
        foreach (var security in changes.AddedSecurities)
        {
            var symbol = security.Symbol;
            var shortInterestSymbol = algorithm.AddData<FINRAShortInterest>(symbol).Symbol;
            _symbolsBySecurity[symbol] = shortInterestSymbol;
            var history = algorithm.History<FINRAShortInterest>(shortInterestSymbol, 10, Resolution.Daily);
            algorithm.Debug($"Got {history.Count()} historical short interest rows for {symbol}");
        }
        foreach (var security in changes.RemovedSecurities)
        {
            if (_symbolsBySecurity.Remove(security.Symbol, out var shortInterestSymbol))
            {
                algorithm.RemoveSecurity(shortInterestSymbol);
            }
        }
    }
}

Data Point Attributes

The FINRA OTC Transparency dataset provides FINRAShortInterest, FINRAWeeklySummary, and FINRAMonthlySummary objects, each with a matching universe class.

FINRAShortInterest

FINRAShortInterest objects have the following attributes:

FINRAShortInterestUniverse

FINRAShortInterestUniverse objects have the following attributes:

FINRAWeeklySummary

FINRAWeeklySummary objects have the following attributes:

FINRAWeeklySummaryUniverse

FINRAWeeklySummaryUniverse objects have the following attributes:

FINRAMonthlySummary

FINRAMonthlySummary objects have the following attributes:

FINRAMonthlySummaryUniverse

FINRAMonthlySummaryUniverse objects have the following attributes:

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: