Universe Filtering with Multiple Parameters

Disclaimer: I'm another Quantopian migrant, and mediocre (at best) programmer.

I'm looking to use several parameters in both the coarse and fine universe functions, and was curious if there was a more 'pythonic' way of doing it. For example, this appears to work:

def CoarseSelectionFunction(self, coarse):
price_filter = [x for x in coarse if float(x.Price) < 5]
sortedByDollarVolume = sorted(price_filter, key=lambda x: x.DollarVolume, reverse=True)
return [x.Symbol for x in sortedByDollarVolume[:50 ]]

def FineSelectionFunction(self, fine):
primaryshare = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001']
nototc = [x for x in primaryshare if x.SecurityReference.ExchangeId != 'OTC']
sortedByPERatio = sorted(nototc, key = lambda x: x.ValuationRatios.PERatio, reverse=True)
return [x.Symbol for x in sortedByPERatio[:10]]

However, I don't like cascading each variable into the next (which requires refactoring the whole function each time I add/remove filters).

On Quantopian, I would expect to be able to do something like:

def make_pipeline(context):
primary_share = IsPrimaryShare()
common_stock = morningstar.share_class_reference.security_type.latest.eq('ST00000001')
not_otc = ~morningstar.share_class_reference.exchange_id.latest.startswith('OTC')
AtMostPrice = (price <= context.MyMostPrice)

tradeable_stocks = (common_stock & not_otc & AtMostPrice)

base_universe = AverageDollarVolume(
window_length=20, mask=tradeable_stocks).percentile_between(10,40)

ShortAvg = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=3, mask=base_universe )

LongAvg = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=45, mask=base_universe )

percent_difference = (ShortAvg - LongAvg) / LongAvg

stocks_worst = percent_difference.bottom(context.MaxCandidates)
securities_to_trade = (stocks_worst)

return Pipeline(columns={'stocks_worst': stocks_worst},screen=(securities_to_trade))

So I have two questions:

  1. Is there a way to use booleans, and aggreations of booleans, in the universe filters? I haven't seen this in any of the documentation or example algos.
  2. How could I print the stocks in the universe to confirm the universe filters are working correctly? (eg. excluding OTC)
Many thanks in advance!
Update Backtest

Hi Thomas,

This is my thought

def FineSelectionFunction(self, fine):
OTCprimaryshare = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001' and
x.SecurityReference.ExchangeId != 'OTC']

sortedByPERatio = sorted(OTCprimaryshare, key = lambda x: x.ValuationRatios.PERatio, reverse=True)
return [x.Symbol for x in sortedByPERatio[:10]]

For the second question, you could Log the symbol value in universe selection like this

self.Log(str([x.Symbol.Value for x in sortedByPERatio[:10]]))

Thanks, Jing. That appears to work - for most trades. 

However, there are trades appearing in my backtests with equities priced above $5, which should be excluded by the coarse universe filter. Any ideas why they are included in the universe?

Here are the selection functions:

class Test(QCAlgorithm):
def Initialize(self):

self.SetStartDate(2015,1,1) #Set Start Date
self.SetEndDate(2018,1,11) #Set End Date
self.SetCash(25000) #Set Strategy Cash

self.UniverseSettings.Resolution = Resolution.Minute
self.UniverseSettings.Leverage = 1
self.coarse_count = 50
self.averages = { }

def CoarseSelectionFunction(self, coarse):
price_filter = filter(lambda x: x.Price < 5.0, coarse)
for cf in price_filter:
if cf.Symbol not in self.averages:
self.averages[cf.Symbol] = SymbolData(cf.Symbol)

avg = self.averages[cf.Symbol]
avg.update(cf.EndTime, cf.Price)

values = self.averages.values()
values.sort(key=lambda x: x.pctdif, reverse = False)

return [x.symbol for x in values[:self.coarse_count]]

def FineSelectionFunction(self, fine):
security_filter = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001'
and x.SecurityReference.ExchangeId != 'OTC'
and x.CompanyReference.IsLimitedPartnership == 0
and x.SecurityReference.IsDepositaryReceipt == 0]

sorting = sorted(security_filter, key = lambda x: x.ValuationRatios.CashReturn, reverse=False)
self.Log(str([x.Symbol.Value for x in sorting[:25]]))
self.Log(str([x.SecurityReference.ExchangeId for x in sorting[:25]]))
self.Log(str([x.ValuationRatios.CashReturn for x in sorting[:25]]))

return [x.Symbol for x in sorting[:25]]

class SymbolData(object):
def __init__(self, symbol):
self.symbol = symbol
self.tolerance = d.Decimal(1.01) = SimpleMovingAverage(3)
self.slow = SimpleMovingAverage(45)
self.pctdif = 01

def update(self, time, value):
if, value) and self.slow.Update(time, value):
fast =
slow = self.slow.Current.Value
self.pctdif = (fast - slow) / slow


Hi Thomas,

The Price property in CourseUniverse Selection is yesterday close price and it is the raw price. It won't change by symbol.SetDataNormalizationMode(DataNormalizationMode.Adjusted). If you would like to select the stocks based on adjusted price, you could filter the symbols after FineUniverse selection. When the securities are being added to your security list, then you could use 

self.Securities[Symbol].Close > 5


Update Backtest


The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


This discussion is closed