Brain
Brain Sentiment Indicator
Introduction
The Brain Sentiment Indicator dataset by Brain tracks the public sentiment around US Equities. The data covers 4,500 US Equities, starts in August 2016, and is delivered on a daily frequency. This dataset is created by analyzing financial news using Natural Language Processing techniques while taking into account the similarity and repetition of news on the same topic. The sentiment score assigned to each stock ranges from -1 (most negative) to +1 (most positive). The sentiment score corresponds to the average sentiment for each piece of news. The score is updated daily and is available on two time scales: 7 days and 30 days. For more information, see Brain's summary paper.
This dataset depends on the US Equity Security Master dataset because the US Equity Security Master dataset contains information on splits, dividends, and symbol changes.
For more information about the Brain Sentiment Indicator dataset, including CLI commands and pricing, see the dataset listing.
About the Provider
Brain is a Research Company that creates proprietary datasets and algorithms for investment strategies, combining experience in financial markets with strong competencies in Statistics, Machine Learning, and Natural Language Processing. The founders share a common academic background of research in Physics as well as extensive experience in Financial markets.
Getting Started
The following snippet demonstrates how to request data from the Brain Sentiment Indicator dataset:
self.aapl = self.add_equity("AAPL", Resolution.DAILY).symbol self.dataset_7day_symbol = self.add_data(BrainSentimentIndicator7Day, self.aapl).symbol self.dataset_30day_symbol = self.add_data(BrainSentimentIndicator30Day, self.aapl).symbol self._universe = self.add_universe(BrainSentimentIndicatorUniverse, self.universe_selection)
_symbol = AddEquity("AAPL", Resolution.Daily).Symbol; _dataset7DaySymbol = AddData<BrainSentimentIndicator7Day>(_symbol).Symbol; _dataset30DaySymbol = AddData<BrainSentimentIndicator30Day>(_symbol).Symbol; _universe = AddUniverse<BrainSentimentIndicatorUniverse>(UniverseSelection);
Requesting Data
To add Brain Sentiment Indicator data to your algorithm, call the AddData
add_data
method. Save a reference to the dataset Symbol
so you can access the data later in your algorithm.
class BrainSentimentDataAlgorithm(QCAlgorithm): def initialize(self) -> None: self.set_start_date(2019, 1, 1) self.set_end_date(2021, 7, 8) self.set_cash(100000) symbol = self.add_equity("AAPL", Resolution.DAILY).symbol self.dataset_7day_symbol = self.add_data(BrainSentimentIndicator7Day, symbol).symbol self.dataset_30day_symbol = self.add_data(BrainSentimentIndicator30Day, symbol).symbol
namespace QuantConnect { public class BrainSentimentDataAlgorithm : QCAlgorithm { private Symbol _dataset7DaySymbol, _dataset30DaySymbol; public override void Initialize() { SetStartDate(2019, 1, 1); SetEndDate(2021, 7, 8); SetCash(100000); var symbol = AddEquity("AAPL", Resolution.Daily).Symbol; _dataset7DaySymbol = AddData<BrainSentimentIndicator7Day>(symbol).Symbol; _dataset30DaySymbol = AddData<BrainSentimentIndicator30Day>(symbol).Symbol; } } }
Accessing Data
To get the current Brain Sentiment Indicator data, index the current Slice
with the dataset Symbol
. Slice objects deliver unique events to your algorithm as they happen, but the Slice
may not contain data for your dataset at every time step. To avoid issues, check if the Slice
contains the data you want before you index it.
def on_data(self, slice: Slice) -> None: if slice.contains_key(self.dataset_7day_symbol): data_point = slice[self.dataset_7day_symbol] self.log(f"{self.dataset_7day_symbol} sentiment at {slice.time}: {data_point.sentiment}") if slice.contains_key(self.dataset_30day_symbol): data_point = slice[self.dataset_30day_symbol] self.log(f"{self.dataset_30day_symbol} sentiment at {slice.time}: {data_point.sentiment}")
public override void OnData(Slice slice) { if (slice.ContainsKey(_dataset7DaySymbol)) { var dataPoint = slice[_dataset7DaySymbol]; Log($"{_dataset7DaySymbol} sentiment at {slice.Time}: {dataPoint.Sentiment}"); } if (slice.ContainsKey(_dataset30DaySymbol)) { var dataPoint = slice[_dataset30DaySymbol]; Log($"{_dataset30DaySymbol} sentiment at {slice.Time}: {dataPoint.Sentiment}"); } }
To iterate through all of the dataset objects in the current Slice
, call the Get
get
method.
def on_data(self, slice: Slice) -> None: for dataset_symbol, data_point in slice.get(BrainSentimentIndicator7Day).items(): self.log(f"{dataset_symbol} sentiment at {slice.time}: {data_point.sentiment}") for dataset_symbol, data_point in slice.get(BrainSentimentIndicator30Day).items(): self.log(f"{dataset_symbol} sentiment at {slice.time}: {data_point.sentiment}")
public override void OnData(Slice slice) { foreach (var kvp in slice.Get<BrainSentimentIndicator7Day>()) { var datasetSymbol = kvp.Key; var dataPoint = kvp.Value; Log($"{datasetSymbol} sentiment at {slice.Time}: {dataPoint.Sentiment}"); } foreach (var kvp in slice.Get<BrainSentimentIndicator30Day>()) { var datasetSymbol = kvp.Key; var dataPoint = kvp.Value; Log($"{datasetSymbol} sentiment at {slice.Time}: {dataPoint.Sentiment}"); } }
Historical Data
To get historical Brain Sentiment Indicator data, call the History
history
method with the dataset Symbol
. If there is no data in the period you request, the history result is empty.
# DataFrames week_history_df = self.history(self.dataset_7day_symbol, 100, Resolution.DAILY) month_history_df = self.history(self.dataset_30day_symbol, 100, Resolution.DAILY) history_df = self.history([self.dataset_7day_symbol, self.dataset_30day_symbol], 100, Resolution.DAILY) # Dataset objects week_history_bars = self.history[BrainSentimentIndicator7Day](self.dataset_7day_symbol, 100, Resolution.DAILY) month_history_bars = self.history[BrainSentimentIndicator30Day](self.dataset_30day_symbol, 100, Resolution.DAILY)
// Dataset objects var weekHistory = History<BrainSentimentIndicator7Day>(_dataset7DaySymbol, 100, Resolution.Daily); var monthHistory = History<BrainSentimentIndicator30Day>(_dataset30DaySymbol, 100, Resolution.Daily); // Slice objects var history = History(new[] {_dataset7DaySymbol, _dataset30DaySymbol}, 100, Resolution.Daily);
For more information about historical data, see History Requests.
Universe Selection
To select a dynamic universe of US Equities based on Brain Sentiment Indicator data, call the AddUniverse
add_universe
method with the BrainSentimentIndicatorUniverse
class and a selection function.
def initialize(self) -> None: self._universe = self.add_universe(BrainSentimentIndicatorUniverse, self.universe_selection) def universe_selection(self, alt_coarse: List[BrainSentimentIndicatorUniverse]) -> List[Symbol]: return [d.symbol for d in alt_coarse \ if d.total_article_mentions7_days > 0 \ and d.sentiment7_days]
private Universe _universe; public override void Initialize() { _universe = AddUniverse<BrainSentimentIndicatorUniverse>(altCoarse=> { return from d in altCoarse.OfType<BrainSentimentIndicatorUniverse>() where d.TotalArticleMentions7Days > 0m && d.Sentiment7Days > 0m select d.Symbol; }); }
The Brain Sentiment Indicator universe runs at 7 AM Eastern Time (ET) in live trading. For more information about dynamic universes, see Universes.
Universe History
You can get historical universe data in an algorithm and in the Research Environment.
Historical Universe Data in Algorithms
To get historical universe data in an algorithm, call the History
history
method with the Universe
object and the lookback period. If there is no data in the period you request, the history result is empty.
var universeHistory = History(_universe, 30, Resolution.Daily); foreach (var sentiments in universeHistory) { foreach (BrainSentimentIndicatorUniverse sentiment in sentiments) { Log($"{sentiment.Symbol} 7-day sentiment at {sentiment.EndTime}: {sentiment.Sentiment7Days}"); } }
# DataFrame example where the columns are the BrainSentimentIndicatorUniverse attributes: history_df = self.history(self._universe, 30, Resolution.DAILY, flatten=True) # Series example where the values are lists of BrainSentimentIndicatorUniverse objects: universe_history = self.history(self._universe, 30, Resolution.DAILY) for (_, time), sentiments in universe_history.items(): for sentiment in sentiments: self.log(f"{sentiment.symbol} 7-day sentiment at {sentiment.end_time}: {sentiment.sentiment7_days}")
Historical Universe Data in Research
To get historical universe data in research, call the UniverseHistory
universe_history
method with the Universe
object, a start date, and an end date. This method returns the filtered universe. If there is no data in the period you request, the history result is empty.
var universeHistory = qb.UniverseHistory(universe, qb.Time.AddDays(-30), qb.Time); foreach (var sentiments in universeHistory) { foreach (BrainSentimentIndicatorUniverse sentiment in sentiments) { Console.WriteLine($"{sentiment.Symbol} 7-day sentiment at {sentiment.EndTime}: {sentiment.Sentiment7Days}"); } }
# DataFrame example where the columns are the BrainSentimentIndicatorUniverse attributes: history_df = qb.universe_history(universe, qb.time-timedelta(30), qb.time, flatten=True) # Series example where the values are lists of BrainSentimentIndicatorUniverse objects: universe_history = qb.universe_history(universe, qb.time-timedelta(30), qb.time) for (_, time), sentiments in universe_history.items(): for sentiment in sentiments: print(f"{sentiment.symbol} 7-day sentiment at {sentiment.end_time}: {sentiment.sentiment7_days}")
You can call the History
history
method in Research.
Remove Subscriptions
To remove a subscription, call the RemoveSecurity
remove_security
method.
self.remove_security(self.dataset_7day_symbol) self.remove_security(self.dataset_30day_symbol)
RemoveSecurity(_dataset7DaySymbol); RemoveSecurity(_dataset30DaySymbol);
If you subscribe to Brain Sentiment Indicator data for assets in a dynamic universe, remove the dataset subscription when the asset leaves your universe. To view a common design pattern, see Track Security Changes.
Example Applications
The Brain Sentiment Indicator dataset enables you to incorporate sentiment from financial news sources into your strategies. Examples include the following strategies:
- Buying when the public sentiment for a security is increasing
- Short selling when the public sentiment for a security is decreasing
- Scaling the position sizing of securities based on how many times they are mentioned in financial news articles
- Sector rotation based on news sentiment
Classic Algorithm Example
The following example algorithm buys Apple when the 30-day Brain Sentiment indicator increases. Otherwise, it remains in cash.
from AlgorithmImports import * from QuantConnect.DataSource import * class BrainSentimentDataAlgorithm(QCAlgorithm): latest_sentiment_value = None target_holdings = 0 def initialize(self) -> None: self.set_start_date(2019, 1, 1) self.set_end_date(2021, 7, 8) self.set_cash(100000) # Requesting the processed longer term (30-day) sentiment score data for sentiment trading self.aapl = self.add_equity("AAPL", Resolution.DAILY).symbol self.dataset_symbol = self.add_data(BrainSentimentIndicator30Day, self.aapl).symbol # Historical data history = self.history(self.dataset_symbol, 100, Resolution.DAILY) self.debug(f"We got {len(history)} items from our history request for {self.dataset_symbol}") if history.empty: return # Warm up historical sentiment values, cache for comparing last sentiment score to trade, making it immediately tradable signal previous_sentiment_values = history.loc[self.dataset_symbol].sentiment.values for sentiment in previous_sentiment_values: self.update(sentiment) def update(self, sentiment: float) -> None: # Comparing the last sentiment score and decide to buy if the sentiment increases to ride the popularity if self.latest_sentiment_value is not None: self.target_holdings = int(sentiment > self.latest_sentiment_value) self.latest_sentiment_value = sentiment def on_data(self, slice: Slice) -> None: # Update trade direction based on updated data if slice.contains_key(self.dataset_symbol): sentiment = slice[self.dataset_symbol].sentiment self.update(sentiment) # Ensure we have security data in the current slice to avoid stale fill if not (slice.contains_key(self.aapl) and slice[self.aapl] is not None): return # Buy if sentiment increase, liquidate otherwise to ride on the popularity of the equity if self.target_holdings != self.portfolio.invested: self.set_holdings(self.aapl, self.target_holdings)
using QuantConnect.DataSource; namespace QuantConnect { public class BrainSentimentDataAlgorithm : QCAlgorithm { private Symbol _symbol; private Symbol _datasetSymbol; private decimal? _latestSentimentValue = None; private int _targetHoldings = 0; public override void Initialize() { SetStartDate(2019, 1, 1); SetEndDate(2021, 7, 8); SetCash(100000); // Requesting the processed longer term (30-day) sentiment score data for sentiment trading _symbol = AddEquity("AAPL", Resolution.Daily).Symbol; _datasetSymbol = AddData<BrainSentimentIndicator30Day>(_symbol).Symbol; // Historical data var history = History<BrainSentimentIndicator30Day>(_datasetSymbol, 100, Resolution.Daily); Debug($"We got {history.Count()} items from our history request for {_datasetSymbol}"); // Warm up historical sentiment values, cache for comparing last sentiment score to trade, making it immediately tradable signal var previousSentimentValues = history.Select(x => x.Sentiment); foreach (var sentiment in previousSentimentValues) { Update(sentiment); } } public void Update(decimal sentiment) { // Comparing the last sentiment score and decide to buy if the sentiment increases to ride the popularity if (_latestSentimentValue != None) { _targetHoldings = sentiment > _latestSentimentValue ? 1 : 0; } _latestSentimentValue = sentiment; } public override void OnData(Slice slice) { // Update trade direction based on updated data if (slice.ContainsKey(_datasetSymbol)) { var sentiment = slice[_datasetSymbol].Sentiment; Update(sentiment); } // Ensure we have security data in the current slice to avoid stale fill // Buy if sentiment increase, liquidate otherwise to ride on the popularity of the equity if (slice.Bar.ContainsKey(_symbol) && _targetHoldings == 1 != Portfolio.Invested) { SetHoldings(_symbol, _targetHoldings); } } } }
Framework Algorithm Example
The following example algorithm creates a dynamic universe of US Equities that have been mentioned in an article over the last seven days. It then buys the subset of Equities that have increasing sentiment and forms an equal-weighted portfolio.
from AlgorithmImports import * from QuantConnect.DataSource import * class BrainSentimentDataAlgorithm(QCAlgorithm): def initialize(self) -> None: self.set_start_date(2019, 1, 1) self.set_end_date(2021, 7, 8) self.set_cash(100000) self.settings.minimum_order_margin_portfolio_percentage = 0 self.universe_settings.resolution = Resolution.DAILY # Filter base on sentiment data self.add_universe(BrainSentimentIndicatorUniverse, self.universe_selection) self.add_alpha(BrainSentimentAlphaModel()) self.set_portfolio_construction(EqualWeightingPortfolioConstructionModel()) self.add_risk_management(NullRiskManagementModel()) self.set_execution(ImmediateExecutionModel()) def universe_selection(self, alt_coarse: List[BrainSentimentIndicatorUniverse]) -> List[Symbol]: # Filter for any sentiment on last 7 days to trade on sentiment news return [d.symbol for d in alt_coarse \ if d.SentimentalArticleMentions7Days is not None and d.SentimentalArticleMentions7Days > 0] class BrainSentimentAlphaModel(AlphaModel): symbol_data_by_symbol = {} def update(self, algorithm: QCAlgorithm, slice: Slice) -> List[Insight]: insights = [] for symbol, symbol_data in self.symbol_data_by_symbol.items(): # Update trade direction based on updated data if slice.contains_key(symbol_data.dataset_symbol) and slice[symbol_data.dataset_symbol] is not None: sentiment = slice[symbol_data.dataset_symbol].sentiment symbol_data.update(sentiment) # Ensure we have security data in the current slice to avoid stale fill if not (slice.contains_key(symbol) and slice[symbol] is not None): continue # Buy if sentiment increase, liquidate otherwise to ride on the popularity of the equity if symbol_data.target_direction == InsightDirection.UP != algorithm.portfolio[symbol].invested: insights.append(Insight.price(symbol, timedelta(days=100), symbol_data.target_direction)) return insights def on_securities_changed(self, algorithm: QCAlgorithm, changes: SecurityChanges) -> None: for security in changes.added_securities: symbol = security.symbol self.symbol_data_by_symbol[symbol] = SymbolData(algorithm, symbol) for security in changes.removed_securities: symbol_data = self.symbol_data_by_symbol.pop(security.symbol, None) if symbol_data: symbol_data.dispose() class SymbolData: target_direction = InsightDirection.FLAT _latest_sentiment_value = None def __init__(self, algorithm: QCAlgorithm, symbol: Symbol) -> None: self.algorithm = algorithm # Requesting the processed longer term (30-day) sentiment score data for sentiment trading self.dataset_symbol = algorithm.add_data(BrainSentimentIndicator30Day, symbol).symbol # Historical data history = algorithm.history(self.dataset_symbol, 100, Resolution.DAILY) algorithm.debug(f"We got {len(history)} items from our history request for {self.dataset_symbol}") if history.empty: return # Warm up historical sentiment values, cache for comparing last sentiment score to trade, making it immediately tradable signal previous_sentiment_values = history.loc[self.dataset_symbol].sentiment.values for sentiment in previous_sentiment_values: self.update(sentiment) def dispose(self) -> None: # Unsubscribe from the Brain Sentiment feed for this security to release computational resources self.algorithm.remove_security(self.dataset_symbol) def update(self, sentiment: float) -> None: # Comparing the last sentiment score and decide to buy if the sentiment increases to ride the popularity if self._latest_sentiment_value is not None: if sentiment > self._latest_sentiment_value: self.target_direction = InsightDirection.UP else: self.target_direction = InsightDirection.FLAT self._latest_sentiment_value = sentiment
using QuantConnect.DataSource; namespace QuantConnect { public class BrainSentimentDataAlgorithm : QCAlgorithm { public override void Initialize() { SetStartDate(2019, 1, 1); SetEndDate(2021, 7, 8); SetCash(100000); Settings.MinimumOrderMarginPortfolioPercentage = 0; UniverseSettings.Resolution = Resolution.Daily; // Filter by sentiment data AddUniverse<BrainSentimentIndicatorUniverse>(altCoarse => { // Filter for any sentiment on last 7 days to trade on sentiment news return from d in altCoarse.OfType<BrainSentimentIndicatorUniverse>() where d.TotalArticleMentions7Days > 0m select d.Symbol; }); AddAlpha(new BrainSentimentAlphaModel()); SetPortfolioConstruction(new EqualWeightingPortfolioConstructionModel()); AddRiskManagement(new NullRiskManagementModel()); SetExecution(new ImmediateExecutionModel()); } } public class BrainSentimentAlphaModel : AlphaModel { private Dictionary<Symbol, SymbolData> _symbolDataBySymbol = new Dictionary<Symbol, SymbolData>(); public override IEnumerable<Insight> Update(QCAlgorithm algorithm, Slice slice) { var insights = new List<Insight>(); foreach (var entry in _symbolDataBySymbol) { var symbol = entry.Key; var symbolData = entry.Value; // Update trade direction based on updated data if (slice.ContainsKey(symbolData.datasetSymbol) && slice[symbolData.datasetSymbol] != None) { var sentiment = slice[symbolData.datasetSymbol].Sentiment; symbolData.Update(sentiment); } // Ensure we have security data in the current slice to avoid stale fill if (!(slice.ContainsKey(symbol) && slice[symbol] != None)) { continue; } // Buy if sentiment increase, liquidate otherwise to ride on the popularity of the equity if (symbolData.targetDirection == InsightDirection.Up != algorithm.Portfolio[symbol].Invested) { insights.Add(Insight.Price(symbol, TimeSpan.FromDays(365), symbolData.targetDirection)); } } return insights; } public override void OnSecuritiesChanged(QCAlgorithm algorithm, SecurityChanges changes) { foreach (var security in changes.AddedSecurities) { var symbol = security.Symbol; _symbolDataBySymbol.Add(symbol, new SymbolData(algorithm, symbol)); } foreach (var security in changes.RemovedSecurities) { var symbol = security.Symbol; if (_symbolDataBySymbol.ContainsKey(symbol)) { _symbolDataBySymbol[symbol].dispose(); _symbolDataBySymbol.Remove(symbol); } } } } public class SymbolData { public Symbol datasetSymbol; public InsightDirection targetDirection = InsightDirection.Flat; private QCAlgorithm _algorithm; private decimal? _latestSentimentValue = None; public SymbolData(QCAlgorithm algorithm, Symbol symbol) { _algorithm = algorithm; // Requesting the processed longer term (30-day) sentiment score data for sentiment trading datasetSymbol = algorithm.AddData<BrainSentimentIndicator30Day>(symbol).Symbol; // Historical data var history = algorithm.History<BrainSentimentIndicator30Day>(datasetSymbol, 100, Resolution.Daily); algorithm.Debug($"We got {history.Count()} items from our history request for {symbol}"); if (history.Count() == 0) { return; } // Warm up historical sentiment values, cache for comparing last sentiment score to trade, making it immediately tradable signal var previousSentimentValues = history.Select(x => x.Sentiment); foreach (var sentiment in previousSentimentValues) { Update(sentiment); } } public void dispose() { // Unsubscribe from the Brain Sentiment feed for this security to release computational resources _algorithm.RemoveSecurity(datasetSymbol); } public void Update(decimal sentiment) { // Comparing the last sentiment score and decide to buy if the sentiment increases to ride the popularity if (_latestSentimentValue != None) { targetDirection = sentiment > _latestSentimentValue ? InsightDirection.Up : InsightDirection.Flat; } _latestSentimentValue = sentiment; } } }
Research Example
The following example lists US Equities having the highest 7-day sentiment.
#r "../QuantConnect.DataSource.BrainSentiment.dll" using QuantConnect.DataSource; var qb = new QuantBook(); // Requesting data var aapl = qb.AddEquity("AAPL", Resolution.Daily).Symbol; var symbol = qb.AddData<BrainSentimentIndicator30Day>(aapl).Symbol; // Historical data var history = qb.History<BrainSentimentIndicator30Day>(symbol, 30, Resolution.Daily); foreach (BrainSentimentIndicator30Day sentiment in history) { Console.WriteLine($"{sentiment} at {sentiment.EndTime}"); } // Add Universe Selection IEnumerable<Symbol> UniverseSelection(IEnumerable<BaseData> altCoarse) { return (from d in altCoarse.OfType<BrainSentimentIndicatorUniverse>() orderby d.Sentiment7Days descending select d.Symbol).Take(10); } var universe = qb.AddUniverse<BrainSentimentIndicatorUniverse>(UniverseSelection); // Historical Universe data var universeHistory = qb.UniverseHistory(universe, qb.Time.AddDays(-5), qb.Time); foreach (var sentiments in universeHistory) { foreach (BrainSentimentIndicatorUniverse sentiment in sentiments) { Console.WriteLine($"{sentiment.Symbol} 7-day sentiment at {sentiment.EndTime}: {sentiment.Sentiment7Days}"); } }
qb = QuantBook() # Requesting Data aapl = qb.add_equity("AAPL", Resolution.DAILY).symbol symbol = qb.add_data(BrainSentimentIndicator30Day, aapl).symbol # Historical data history = qb.history(BrainSentimentIndicator30Day, symbol, 30, Resolution.DAILY) for (symbol, time), row in history.iterrows(): print(f"{symbol} sentiment at {time}: {row['sentiment']}") # Add Universe Selection def universe_selection(alt_coarse: List[BrainSentimentIndicatorUniverse]) -> List[Symbol]: return [d.symbol for d in sorted([x for x in alt_coarse if x.SentimentalArticleMentions7Days], key=lambda x: x.SentimentalArticleMentions7Days, reverse=True)[:10]] universe = qb.add_universe(BrainSentimentIndicatorUniverse, universe_selection) # Historical Universe data universe_history = qb.universe_history(universe, qb.time-timedelta(5), qb.time) for (_, time), sentiments in universe_history.items(): for sentiment in sentiments: print(f"{sentiment.symbol} 7-day sentiment at {sentiment.end_time}: {sentiment.SentimentalArticleMentions7Days}")
Data Point Attributes
The Brain Sentiment Indicator dataset provides BrainSentimentIndicatorBase
and BrainSentimentIndicatorUniverse
objects.
BrainSentimentIndicatorBase Attributes
BrainSentimentIndicatorBase
objects have the following attributes:
BrainSentimentIndicatorUniverse Attributes
BrainSentimentIndicatorUniverse
objects have the following attributes: