Financial Industry Regulatory Authority
OTC Transparency
Introduction
The FINRA OTC Transparency dataset by FINRA (the Financial Industry Regulatory Authority) shows you what US stocks are trading away from the exchanges, plus how heavily each one is being shorted. It covers around 59,000 US Equities, goes back to January 2017, and updates daily. FINRA builds it from the off-exchange volume and short interest reports that broker-dealers have to file with them.
For more information about the OTC Transparency dataset, including CLI commands and pricing, see the dataset listing.
About the Provider
FINRA (the Financial Industry Regulatory Authority) was created in 2007, when NASD merged with the member regulation arm of the New York Stock Exchange. Its job is to protect investors and keep the markets honest, and it oversees nearly every broker-dealer that does business with the public in the United States. As part of that role, FINRA collects the off-exchange trading and short interest reports those firms are required to file, and publishes the results for free so researchers, investors, and anyone else can get a clearer view of what is happening away from the exchanges.
Getting Started
The following snippet demonstrates how to request data from the FINRA OTC Transparency dataset:
self.symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
self.short_interest = self.add_data(FINRAShortInterest, self.symbol).symbol
self.weekly_summary = self.add_data(FINRAWeeklySummary, self.symbol).symbol
self.monthly_summary = self.add_data(FINRAMonthlySummary, self.symbol).symbol _symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
_shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
_weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
_monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;
Requesting Data
To add FINRA OTC Transparency data to your algorithm, call the AddDataadd_data method. The dataset is linked to US Equities, so pass the equity Symbol you want the data for. Save a reference to the dataset Symbol so you can access the data later in your algorithm.
The dataset comes in three classes: FINRAShortInterest, FINRAWeeklySummary, and FINRAMonthlySummary. Subscribe to the ones your strategy needs.
class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
def initialize(self) -> None:
self.set_start_date(2022, 1, 1)
self.set_end_date(2023, 1, 1)
self.set_cash(100000)
self.symbol = self.add_equity("AAPL", Resolution.DAILY).symbol
self.short_interest = self.add_data(FINRAShortInterest, self.symbol).symbol
self.weekly_summary = self.add_data(FINRAWeeklySummary, self.symbol).symbol
self.monthly_summary = self.add_data(FINRAMonthlySummary, self.symbol).symbol public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
private Symbol _symbol, _shortInterest, _weeklySummary, _monthlySummary;
public override void Initialize()
{
SetStartDate(2022, 1, 1);
SetEndDate(2023, 1, 1);
SetCash(100000);
_symbol = AddEquity("AAPL", Resolution.Daily).Symbol;
_shortInterest = AddData<FINRAShortInterest>(_symbol).Symbol;
_weeklySummary = AddData<FINRAWeeklySummary>(_symbol).Symbol;
_monthlySummary = AddData<FINRAMonthlySummary>(_symbol).Symbol;
}
}
Accessing Data
To get the current FINRA OTC Transparency data, index the current Slice with the dataset Symbol. Slice objects deliver unique events to your algorithm as they happen, but the Slice may not contain data for your dataset at every time step. To avoid issues, check if the Slice contains the data you want before you index it.
def on_data(self, slice: Slice) -> None:
if slice.contains_key(self.short_interest):
data_point = slice[self.short_interest]
self.log(f"{self.short_interest} days to cover at {slice.time}: {data_point.days_to_cover}") public override void OnData(Slice slice)
{
if (slice.ContainsKey(_shortInterest))
{
var dataPoint = slice[_shortInterest];
Log($"{_shortInterest} days to cover at {slice.Time}: {dataPoint.DaysToCover}");
}
}
To iterate through all of the dataset objects in the current Slice, call the Get method.
def on_data(self, slice: Slice) -> None:
for symbol, data_point in slice.get(FINRAShortInterest).items():
self.log(f"{symbol} days to cover at {slice.time}: {data_point.days_to_cover}")
for symbol, data_point in slice.get(FINRAWeeklySummary).items():
self.log(f"{symbol} ATS weekly shares at {slice.time}: {data_point.share_quantity}")
for symbol, data_point in slice.get(FINRAMonthlySummary).items():
self.log(f"{symbol} non-ATS monthly shares at {slice.time}: {data_point.share_quantity}") public override void OnData(Slice slice)
{
foreach (var kvp in slice.Get<FINRAShortInterest>())
{
Log($"{kvp.Key} days to cover at {slice.Time}: {kvp.Value.DaysToCover}");
}
foreach (var kvp in slice.Get<FINRAWeeklySummary>())
{
Log($"{kvp.Key} ATS weekly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
}
foreach (var kvp in slice.Get<FINRAMonthlySummary>())
{
Log($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
}
}
Historical Data
To get historical FINRA OTC Transparency data, call the History method with the dataset Symbol. If there is no data in the period you request, the history result is empty.
# DataFrames short_interest_history = self.history(self.short_interest, 100, Resolution.DAILY) weekly_history = self.history(self.weekly_summary, 100, Resolution.DAILY) monthly_history = self.history(self.monthly_summary, 100, Resolution.DAILY) history = self.history([self.short_interest, self.weekly_summary, self.monthly_summary], 100, Resolution.DAILY) # Dataset objects short_interest_bars = self.history[FINRAShortInterest](self.short_interest, 100, Resolution.DAILY) weekly_bars = self.history[FINRAWeeklySummary](self.weekly_summary, 100, Resolution.DAILY) monthly_bars = self.history[FINRAMonthlySummary](self.monthly_summary, 100, Resolution.DAILY)
// Dataset objects
var shortInterestHistory = History<FINRAShortInterest>(_shortInterest, 100, Resolution.Daily);
var weeklyHistory = History<FINRAWeeklySummary>(_weeklySummary, 100, Resolution.Daily);
var monthlyHistory = History<FINRAMonthlySummary>(_monthlySummary, 100, Resolution.Daily);
// Slice objects
var history = History(new[] { _shortInterest, _weeklySummary, _monthlySummary }, 100, Resolution.Daily);
For more information about historical data, see History Requests.
Universe Selection
To select a dynamic universe of US Equities based on FINRA OTC Transparency data, call the AddUniverseadd_universe method with one of the dataset's universe classes (FINRAShortInterestUniverse, FINRAWeeklySummaryUniverse, or FINRAMonthlySummaryUniverse) and a selection function. The following example selects the most heavily shorted names by days-to-cover:
def initialize(self) -> None:
self._universe = self.add_universe(FINRAShortInterestUniverse, self.universe_selection)
def universe_selection(self, alt_coarse: List[FINRAShortInterestUniverse]) -> List[Symbol]:
return [d.symbol for d in alt_coarse
if d.days_to_cover is not None and d.days_to_cover > 5] private Universe _universe;
public override void Initialize()
{
_universe = AddUniverse<FINRAShortInterestUniverse>(altCoarse=>
{
return from d in altCoarse.OfType<FINRAShortInterestUniverse>()
where d.DaysToCover > 5m
select d.Symbol;
});
}
For more information about dynamic universes, see Universes.
Remove Subscriptions
To remove a subscription, call the RemoveSecurity method.
self.remove_security(self.short_interest) self.remove_security(self.weekly_summary) self.remove_security(self.monthly_summary)
RemoveSecurity(_shortInterest); RemoveSecurity(_weeklySummary); RemoveSecurity(_monthlySummary);
If you subscribe to FINRA OTC Transparency data for assets in a dynamic universe, remove the dataset subscription when the asset leaves your universe. To view a common design pattern, see Track Security Changes.
Example Applications
The FINRA OTC Transparency dataset lets you see the short positioning and off-exchange order flow behind a stock. Examples include:
- Hunting for short squeeze candidates by ranking the universe on days-to-cover, the short interest divided by average daily volume.
- Following or fading a stock as its open short interest builds up or unwinds from one reporting period to the next.
- Reading institutional accumulation or distribution from a shift in how much of a stock's volume prints on dark pools, using the weekly ATS shares.
- Combining the weekly ATS and monthly non-ATS volume into a total off-exchange participation measure and trading the names where it breaks from their norm.
Classic Algorithm Example
The following example algorithm subscribes to the three FINRA OTC Transparency report types for a US Equity and trades it on the change in open short interest:
from AlgorithmImports import *
class FINRAOtcTransparencyDataAlgorithm(QCAlgorithm):
def initialize(self):
self.set_start_date(2022, 1, 1)
self.set_end_date(2023, 1, 1)
self.set_cash(100000)
aapl = self.add_equity("AAPL", Resolution.DAILY).symbol
self.short_interest = self.add_data(FINRAShortInterest, aapl).symbol
self.weekly_summary = self.add_data(FINRAWeeklySummary, aapl).symbol
self.monthly_summary = self.add_data(FINRAMonthlySummary, aapl).symbol
history = self.history(FINRAShortInterest, self.short_interest, 30, Resolution.DAILY)
self.debug(f"We got {len(history)} items from our history request")
def on_data(self, slice):
# Off-exchange volume prints as an activity signal
for symbol, point in slice.get(FINRAMonthlySummary).items():
self.debug(f"{symbol} non-ATS monthly shares at {slice.time}: {point.share_quantity}")
for symbol, point in slice.get(FINRAShortInterest).items():
if point.change is None:
continue
# Go short when open short interest is building; go long when it is unwinding
if point.change > 0:
self.set_holdings(symbol.underlying, -1)
else:
self.set_holdings(symbol.underlying, 1) public class FINRAOtcTransparencyDataAlgorithm : QCAlgorithm
{
private Symbol _shortInterest, _weeklySummary, _monthlySummary;
public override void Initialize()
{
SetStartDate(2022, 1, 1);
SetEndDate(2023, 1, 1);
SetCash(100000);
var aapl = AddEquity("AAPL", Resolution.Daily).Symbol;
_shortInterest = AddData<FINRAShortInterest>(aapl).Symbol;
_weeklySummary = AddData<FINRAWeeklySummary>(aapl).Symbol;
_monthlySummary = AddData<FINRAMonthlySummary>(aapl).Symbol;
var history = History<FINRAShortInterest>(_shortInterest, 30, Resolution.Daily);
Debug($"We got {history.Count()} items from our history request");
}
public override void OnData(Slice slice)
{
// Off-exchange volume prints as an activity signal
foreach (var kvp in slice.Get<FINRAMonthlySummary>())
{
Debug($"{kvp.Key} non-ATS monthly shares at {slice.Time}: {kvp.Value.ShareQuantity}");
}
foreach (var kvp in slice.Get<FINRAShortInterest>())
{
var point = kvp.Value;
if (point.Change == null)
{
continue;
}
// Go short when open short interest is building; go long when it is unwinding
if (point.Change > 0)
{
SetHoldings(kvp.Key.Underlying, -1);
}
else
{
SetHoldings(kvp.Key.Underlying, 1);
}
}
}
}
Framework Algorithm Example
The following example algorithm uses the Algorithm Framework to trade a manually selected universe from FINRA OTC Transparency data. An alpha model subscribes each security to the consolidated short interest feed and emits insights from the change in open short interest:
from AlgorithmImports import *
class FINRAOtcTransparencyFrameworkAlgorithm(QCAlgorithm):
def initialize(self) -> None:
self.set_start_date(2022, 1, 1)
self.set_end_date(2023, 1, 1)
self.set_cash(100000)
self.universe_settings.resolution = Resolution.DAILY
symbols = [Symbol.create("AAPL", SecurityType.EQUITY, Market.USA)]
self.set_universe_selection(ManualUniverseSelectionModel(symbols))
self.add_alpha(FINRAShortInterestAlphaModel())
self.set_portfolio_construction(EqualWeightingPortfolioConstructionModel())
self.set_execution(ImmediateExecutionModel())
class FINRAShortInterestAlphaModel(AlphaModel):
def __init__(self) -> None:
self._symbols_by_security = {}
def update(self, algorithm: QCAlgorithm, data: Slice) -> List[Insight]:
insights = []
for security_symbol, short_interest_symbol in self._symbols_by_security.items():
if short_interest_symbol not in data:
continue
change = data[short_interest_symbol].change
if change is None:
continue
# Building short interest is bearish, unwinding short interest is bullish.
direction = InsightDirection.DOWN if change > 0 else InsightDirection.UP
insights.append(Insight.price(security_symbol, timedelta(days=14), direction))
return insights
def on_securities_changed(self, algorithm: QCAlgorithm, changes: SecurityChanges) -> None:
for security in changes.added_securities:
symbol = security.symbol
short_interest_symbol = algorithm.add_data(FINRAShortInterest, symbol).symbol
self._symbols_by_security[symbol] = short_interest_symbol
history = algorithm.history(FINRAShortInterest, short_interest_symbol, 10, Resolution.DAILY)
algorithm.debug(f"Got {len(history)} historical short interest rows for {symbol}")
for security in changes.removed_securities:
short_interest_symbol = self._symbols_by_security.pop(security.symbol, None)
if short_interest_symbol is not None:
algorithm.remove_security(short_interest_symbol) public class FINRAOtcTransparencyFrameworkAlgorithm : QCAlgorithm
{
public override void Initialize()
{
SetStartDate(2022, 1, 1);
SetEndDate(2023, 1, 1);
SetCash(100000);
UniverseSettings.Resolution = Resolution.Daily;
var symbols = new[] { QuantConnect.Symbol.Create("AAPL", SecurityType.Equity, Market.USA) };
SetUniverseSelection(new ManualUniverseSelectionModel(symbols));
AddAlpha(new FINRAShortInterestAlphaModel());
SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel());
SetExecution(new ImmediateExecutionModel());
}
}
public class FINRAShortInterestAlphaModel : AlphaModel
{
private readonly Dictionary<Symbol, Symbol> _symbolsBySecurity = new();
public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice data)
{
foreach (var kvp in _symbolsBySecurity)
{
var securitySymbol = kvp.Key;
var shortInterestSymbol = kvp.Value;
if (!data.ContainsKey(shortInterestSymbol))
{
continue;
}
var change = data.Get<FINRAShortInterest>(shortInterestSymbol).Change;
if (change == null)
{
continue;
}
// Building short interest is bearish, unwinding short interest is bullish.
var direction = change > 0 ? InsightDirection.Down : InsightDirection.Up;
yield return Insight.Price(securitySymbol, TimeSpan.FromDays(14), direction);
}
}
public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes)
{
foreach (var security in changes.AddedSecurities)
{
var symbol = security.Symbol;
var shortInterestSymbol = algorithm.AddData<FINRAShortInterest>(symbol).Symbol;
_symbolsBySecurity[symbol] = shortInterestSymbol;
var history = algorithm.History<FINRAShortInterest>(shortInterestSymbol, 10, Resolution.Daily);
algorithm.Debug($"Got {history.Count()} historical short interest rows for {symbol}");
}
foreach (var security in changes.RemovedSecurities)
{
if (_symbolsBySecurity.Remove(security.Symbol, out var shortInterestSymbol))
{
algorithm.RemoveSecurity(shortInterestSymbol);
}
}
}
}
Data Point Attributes
The FINRA OTC Transparency dataset provides FINRAShortInterest, FINRAWeeklySummary, and FINRAMonthlySummary objects, each with a matching universe class.
FINRAShortInterest
FINRAShortInterest objects have the following attributes:
FINRAShortInterestUniverse
FINRAShortInterestUniverse objects have the following attributes:
FINRAWeeklySummary
FINRAWeeklySummary objects have the following attributes:
FINRAWeeklySummaryUniverse
FINRAWeeklySummaryUniverse objects have the following attributes:
FINRAMonthlySummary
FINRAMonthlySummary objects have the following attributes:
FINRAMonthlySummaryUniverse
FINRAMonthlySummaryUniverse objects have the following attributes: