Financial Industry Regulatory Authority

OTC Transparency

Introduction

The OTC Transparency dataset by the Financial Industry Regulatory Authority (FINRA) tracks short interest and off-exchange trading volume for US Equities. The data covers 59,712 US Equities, starts in January 2017, and is delivered on a daily frequency. This dataset is created by processing the short interest and off-exchange volume reports that broker-dealers file with FINRA.

For more information about the OTC Transparency dataset, including CLI commands and pricing, see the dataset listing.

About the Provider

The Financial Industry Regulatory Authority (FINRA) is a not-for-profit organization that oversees broker-dealers in the United States, under the supervision of the SEC. It was formed in 2007 when NASD merged with the member regulation arm of the New York Stock Exchange. Its mission is to protect investors and safeguard market integrity.

Getting Started

The following snippet demonstrates how to request data from the FINRA OTC Transparency dataset:

self._symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
self.short_interest = self.add_data(FINRAShortInterest, self._symbol).symbol
self.weekly_summary = self.add_data(FINRAWeeklySummary, self._symbol).symbol
self.monthly_summary = self.add_data(FINRAMonthlySummary, self._symbol).symbol
_symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
_shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
_weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
_monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;

Data Summary

The following table describes the dataset properties:

PropertyValue
Start DateJanuary 2017
Asset Coverage59,712 US Equities
Data DensitySparse
ResolutionDaily*
TimezoneNew York

* Weekly, twice-monthly, and monthly data we fetch daily.

Requesting Data

To add FINRA OTC Transparency data to your algorithm, call the AddDataadd_data method. Save a reference to the dataset Symbol so you can access the data later in your algorithm.

class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        self._symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
        self.short_interest = self.add_data(FINRAShortInterest, self._symbol).symbol
        self.weekly_summary = self.add_data(FINRAWeeklySummary, self._symbol).symbol
        self.monthly_summary = self.add_data(FINRAMonthlySummary, self._symbol).symbol
public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
    private Symbol _symbol, _shortInterest, _weeklySummary, _monthlySummary;

    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        _symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
        _shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
        _weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
        _monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;
    }
}

Accessing Data

To get the current FINRA OTC Transparency data, index the current Slice with the dataset Symbol. Slice objects deliver unique events to your algorithm as they happen, but the Slice may not contain data for your dataset at every time step. To avoid issues, check if the Slice contains the data you want before you index it.

def on_data(self, slice: Slice) -> None:
    if slice.contains_key(self.short_interest):
        data_point = slice[self.short_interest]
        self.log(f"{self.short_interest} days to cover at {slice.time}: {data_point.days_to_cover}")
public override void OnData(Slice slice)
{
    if (slice.ContainsKey(_shortInterest))
    {
        var dataPoint = slice[_shortInterest];
        Log($"{_shortInterest} days to cover at {slice.Time}: {dataPoint.DaysToCover}");
    }
}

To iterate through all of the dataset objects in the current Slice, call the Getget method.

def on_data(self, slice: Slice) -> None:
    for symbol, data_point in slice.get(FINRAShortInterest).items():
        self.log(f"{symbol} days to cover at {slice.time}: {data_point.days_to_cover}")

    for symbol, data_point in slice.get(FINRAWeeklySummary).items():
        self.log(f"{symbol} ATS weekly shares at {slice.time}: {data_point.share_quantity}")

    for symbol, data_point in slice.get(FINRAMonthlySummary).items():
        self.log(f"{symbol} non-ATS monthly shares at {slice.time}: {data_point.share_quantity}")
public override void OnData(Slice slice)
{
    foreach (var kvp in slice.Get<FINRAShortInterest>())
    {
        Log($"{kvp.Key} days to cover at {slice.Time}: {kvp.Value.DaysToCover}");
    }

    foreach (var kvp in slice.Get<FINRAWeeklySummary>())
    {
        Log($"{kvp.Key} ATS weekly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
    }

    foreach (var kvp in slice.Get<FINRAMonthlySummary>())
    {
        Log($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
    }
}

Historical Data

To get historical FINRA OTC Transparency data, call the Historyhistory method with the dataset Symbol. If there is no data in the period you request, the history result is empty.

# DataFrames
short_interest_history = self.history(self.short_interest, 100, Resolution.DAILY)
weekly_history = self.history(self.weekly_summary, 100, Resolution.DAILY)
monthly_history = self.history(self.monthly_summary, 100, Resolution.DAILY)
history = self.history([self.short_interest, self.weekly_summary, self.monthly_summary], 100, Resolution.DAILY)

# Dataset objects
short_interest_bars = self.history[FINRAShortInterest](self.short_interest, 100, Resolution.DAILY)
weekly_bars = self.history[FINRAWeeklySummary](self.weekly_summary, 100, Resolution.DAILY)
monthly_bars = self.history[FINRAMonthlySummary](self.monthly_summary, 100, Resolution.DAILY)
// Dataset objects
var shortInterestHistory = History<FINRAShortInterest>(_shortInterest, 100, Resolution.Daily);
var weeklyHistory = History<FINRAWeeklySummary>(_weeklySummary, 100, Resolution.Daily);
var monthlyHistory = History<FINRAMonthlySummary>(_monthlySummary, 100, Resolution.Daily);

// Slice objects
var history = History(new[] { _shortInterest, _weeklySummary, _monthlySummary }, 100, Resolution.Daily);

For more information about historical data, see History Requests.

Universe Selection

To select a dynamic universe of US Equities based on FINRA OTC Transparency data, call the AddUniverseadd_universe method with one of the dataset's universe classes and a selection function. The universe classes are FINRAShortInterestUniverse, FINRAWeeklySummaryUniverse, and FINRAMonthlySummaryUniverse.

def initialize(self) -> None:
    self._universe = self.add_universe(FINRAShortInterestUniverse, self.universe_selection)

def universe_selection(self, alt_coarse: List[FINRAShortInterestUniverse]) -> List[Symbol]:
    return [d.symbol for d in alt_coarse
            if d.days_to_cover is not None and d.days_to_cover > 5]
private Universe _universe;
public override void Initialize()
{
    _universe = AddUniverse<FINRAShortInterestUniverse>(altCoarse=>
    {
        return from d in altCoarse.OfType<FINRAShortInterestUniverse>()
            where d.DaysToCover > 5m
            select d.Symbol;
    });
}

For more information about dynamic universes, see Universes.

Remove Subscriptions

To remove a subscription, call the RemoveSecurityremove_security method.

self.remove_security(self.short_interest)
self.remove_security(self.weekly_summary)
self.remove_security(self.monthly_summary)
RemoveSecurity(_shortInterest);
RemoveSecurity(_weeklySummary);
RemoveSecurity(_monthlySummary);

If you subscribe to FINRA OTC Transparency data for assets in a dynamic universe, remove the dataset subscription when the asset leaves your universe. To view a common design pattern, see Track Security Changes.

Example Applications

The OTC Transparency dataset enables you to monitor short interest and off-exchange trading. Examples include the following strategies:

  • Finding short squeeze candidates by comparing short interest to average daily volume
  • Trading stocks when their short interest rises or falls sharply between reports
  • Spotting institutional buying or selling from changes in alternative trading system (ATS) volume

Classic Algorithm Example

The following example algorithm short sells AAPL when its open short interest increases. Otherwise, it buys AAPL.

from AlgorithmImports import *

class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        aapl = self.add_equity("AAPL", Resolution.DAILY).symbol
        self.short_interest = self.add_data(FINRAShortInterest, aapl).symbol
        self.weekly_summary = self.add_data(FINRAWeeklySummary, aapl).symbol
        self.monthly_summary = self.add_data(FINRAMonthlySummary, aapl).symbol

        history = self.history(FINRAShortInterest, self.short_interest, 30, Resolution.DAILY)
        self.debug(f"We got {len(history)} items from our history request")

    def on_data(self, slice):
        # Log the monthly non-ATS share volume.
        for symbol, point in slice.get(FINRAMonthlySummary).items():
            self.debug(f"{symbol} non-ATS monthly shares at {slice.time}: {point.share_quantity}")

        for symbol, point in slice.get(FINRAShortInterest).items():
            if point.change is None:
                continue

            # Short sell when open short interest increases. Otherwise, buy.
            if point.change > 0:
                self.set_holdings(symbol.underlying, -1)
            else:
                self.set_holdings(symbol.underlying, 1)
public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
    private Symbol _shortInterest, _weeklySummary, _monthlySummary;

    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        var aapl = AddEquity("AAPL", Resolution.Daily).Symbol;
        _shortInterest = AddData<FINRAShortInterest>(aapl).Symbol;
        _weeklySummary = AddData<FINRAWeeklySummary>(aapl).Symbol;
        _monthlySummary = AddData<FINRAMonthlySummary>(aapl).Symbol;

        var history = History<FINRAShortInterest>(_shortInterest, 30, Resolution.Daily);
        Debug($"We got {history.Count()} items from our history request");
    }

    public override void OnData(Slice slice)
    {
        // Log the monthly non-ATS share volume.
        foreach (var kvp in slice.Get<FINRAMonthlySummary>())
        {
            Debug($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
        }

        foreach (var kvp in slice.Get<FINRAShortInterest>())
        {
            var point = kvp.Value;
            if (point.Change == null)
            {
                continue;
            }

            // Short sell when open short interest increases. Otherwise, buy.
            if (point.Change > 0)
            {
                SetHoldings(kvp.Key.Underlying, -1);
            }
            else
            {
                SetHoldings(kvp.Key.Underlying, 1);
            }
        }
    }
}

Framework Algorithm Example

The following example algorithm emits a down insight for AAPL when its open short interest increases. Otherwise, it emits an up insight.

from AlgorithmImports import *

class FINRAOtcTransparencyFrameworkAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)

        self.universe_settings.resolution = Resolution.DAILY

        symbols = [Symbol.create("AAPL", SecurityType.EQUITY, Market.USA)]
        self.set_universe_selection(ManualUniverseSelectionModel(symbols))

        self.add_alpha(FINRAShortInterestAlphaModel())

        self.set_portfolio_construction(EqualWeightingPortfolioConstructionModel())
        self.set_execution(ImmediateExecutionModel())


class FINRAShortInterestAlphaModel(AlphaModel):

    def __init__(self) -> None:
        self._symbols_by_security = {}

    def update(self, algorithm: QCAlgorithm, data: Slice) -> List[Insight]:
        insights = []
        for security_symbol, short_interest_symbol in self._symbols_by_security.items():
            if short_interest_symbol not in data:
                continue
            change = data[short_interest_symbol].change
            if change is None:
                continue
            # Emit a down insight when short interest increases, otherwise an up insight.
            direction = InsightDirection.DOWN if change > 0 else InsightDirection.UP
            insights.append(Insight.price(security_symbol, timedelta(days=14), direction))
        return insights

    def on_securities_changed(self, algorithm: QCAlgorithm, changes: SecurityChanges) -> None:
        for security in changes.added_securities:
            symbol = security.symbol
            short_interest_symbol = algorithm.add_data(FINRAShortInterest, symbol).symbol
            self._symbols_by_security[symbol] = short_interest_symbol
            history = algorithm.history(FINRAShortInterest, short_interest_symbol, 10, Resolution.DAILY)
            algorithm.debug(f"Got {len(history)} historical short interest rows for {symbol}")
        for security in changes.removed_securities:
            short_interest_symbol = self._symbols_by_security.pop(security.symbol, None)
            if short_interest_symbol is not None:
                algorithm.remove_security(short_interest_symbol)
public class FINRAOtcTransparencyFrameworkAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2022, 1, 1);
        SetEndDate(2023, 1, 1);
        SetCash(100000);

        UniverseSettings.Resolution = Resolution.Daily;

        var symbols = new[] { QuantConnect.Symbol.Create("AAPL", SecurityType.Equity, Market.USA) };
        SetUniverseSelection(new ManualUniverseSelectionModel(symbols));

        AddAlpha(new FINRAShortInterestAlphaModel());

        SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel());
        SetExecution(new ImmediateExecutionModel());
    }
}

public class FINRAShortInterestAlphaModel : AlphaModel
{
    private readonly Dictionary<Symbol, Symbol> _symbolsBySecurity = new();

    public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice data)
    {
        foreach (var kvp in _symbolsBySecurity)
        {
            var securitySymbol = kvp.Key;
            var shortInterestSymbol = kvp.Value;
            if (!data.ContainsKey(shortInterestSymbol))
            {
                continue;
            }
            var change = data.Get<FINRAShortInterest>(shortInterestSymbol).Change;
            if (change == null)
            {
                continue;
            }
            // Emit a down insight when short interest increases, otherwise an up insight.
            var direction = change > 0 ? InsightDirection.Down : InsightDirection.Up;
            yield return Insight.Price(securitySymbol, TimeSpan.FromDays(14), direction);
        }
    }

    public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes)
    {
        foreach (var security in changes.AddedSecurities)
        {
            var symbol = security.Symbol;
            var shortInterestSymbol = algorithm.AddData<FINRAShortInterest>(symbol).Symbol;
            _symbolsBySecurity[symbol] = shortInterestSymbol;
            var history = algorithm.History<FINRAShortInterest>(shortInterestSymbol, 10, Resolution.Daily);
            algorithm.Debug($"Got {history.Count()} historical short interest rows for {symbol}");
        }
        foreach (var security in changes.RemovedSecurities)
        {
            if (_symbolsBySecurity.Remove(security.Symbol, out var shortInterestSymbol))
            {
                algorithm.RemoveSecurity(shortInterestSymbol);
            }
        }
    }
}

Data Point Attributes

The FINRA OTC Transparency dataset provides FINRAShortInterest, FINRAWeeklySummary, and FINRAMonthlySummary objects, each with a matching universe class.

FINRAShortInterest

FINRAShortInterest objects have the following attributes:

FINRAShortInterestUniverse

FINRAShortInterestUniverse objects have the following attributes:

FINRAWeeklySummary

FINRAWeeklySummary objects have the following attributes:

FINRAWeeklySummaryUniverse

FINRAWeeklySummaryUniverse objects have the following attributes:

FINRAMonthlySummary

FINRAMonthlySummary objects have the following attributes:

FINRAMonthlySummaryUniverse

FINRAMonthlySummaryUniverse objects have the following attributes:

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: