Universe Filtering with Multiple Parameters by Thomas Schonig

Thomas Schonig INVESTOR

| Reward Discussion

Disclaimer: I'm another Quantopian migrant, and mediocre (at best) programmer.

I'm looking to use several parameters in both the coarse and fine universe functions, and was curious if there was a more 'pythonic' way of doing it. For example, this appears to work:

    def CoarseSelectionFunction(self, coarse):

        price_filter = [x for x in coarse if float(x.Price) < 5]

        sortedByDollarVolume = sorted(price_filter, key=lambda x: x.DollarVolume, reverse=True)

        return [x.Symbol for x in sortedByDollarVolume[:50 ]]

    

    def FineSelectionFunction(self, fine):

        primaryshare = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001']

        nototc = [x for x in primaryshare if x.SecurityReference.ExchangeId != 'OTC']

        sortedByPERatio = sorted(nototc, key = lambda x: x.ValuationRatios.PERatio, reverse=True)

        return [x.Symbol for x in sortedByPERatio[:10]]

However, I don't like cascading each variable into the next (which requires refactoring the whole function each time I add/remove filters).

On Quantopian, I would expect to be able to do something like:

def make_pipeline(context):

    primary_share = IsPrimaryShare()

    common_stock = morningstar.share_class_reference.security_type.latest.eq('ST00000001')

    not_otc = ~morningstar.share_class_reference.exchange_id.latest.startswith('OTC')

    AtMostPrice    = (price <= context.MyMostPrice)



    tradeable_stocks = (common_stock & not_otc & AtMostPrice)



    base_universe = AverageDollarVolume(

        window_length=20, mask=tradeable_stocks).percentile_between(10,40)





    ShortAvg = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=3, mask=base_universe    )





    LongAvg = SimpleMovingAverage(inputs=[USEquityPricing.close], window_length=45, mask=base_universe    )



    percent_difference = (ShortAvg - LongAvg) / LongAvg





    stocks_worst = percent_difference.bottom(context.MaxCandidates)

    securities_to_trade = (stocks_worst)



    return Pipeline(columns={'stocks_worst': stocks_worst},screen=(securities_to_trade))

So I have two questions:

Is there a way to use booleans, and aggreations of booleans, in the universe filters? I haven't seen this in any of the documentation or example algos.
How could I print the stocks in the universe to confirm the universe filters are working correctly? (eg. excluding OTC)

Many thanks in advance!

Update Backtest

Thomas Schonig

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Jing Wu

242.2k ,

Hi Thomas,

This is my thought

def FineSelectionFunction(self, fine):

    OTCprimaryshare = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001' and 

                                          x.SecurityReference.ExchangeId != 'OTC']



    sortedByPERatio = sorted(OTCprimaryshare, key = lambda x: x.ValuationRatios.PERatio, reverse=True)

    return [x.Symbol for x in sortedByPERatio[:10]]

For the second question, you could Log the symbol value in universe selection like this

self.Log(str([x.Symbol.Value for x in sortedByPERatio[:10]]))

Thomas Schonig

19 ,

Thanks, Jing. That appears to work - for most trades.

However, there are trades appearing in my backtests with equities priced above $5, which should be excluded by the coarse universe filter. Any ideas why they are included in the universe?

Here are the selection functions:

class Test(QCAlgorithm): def Initialize(self): self.SetStartDate(2015,1,1) #Set Start Date self.SetEndDate(2018,1,11) #Set End Date self.SetCash(25000) #Set Strategy Cash self.UniverseSettings.Resolution = Resolution.Minute self.UniverseSettings.Leverage = 1 self.coarse_count = 50 self.averages = { } self.AddUniverse(self.CoarseSelectionFunction,self.FineSelectionFunction) def CoarseSelectionFunction(self, coarse): price_filter = filter(lambda x: x.Price < 5.0, coarse) for cf in price_filter: if cf.Symbol not in self.averages: self.averages[cf.Symbol] = SymbolData(cf.Symbol) avg = self.averages[cf.Symbol] avg.update(cf.EndTime, cf.Price) values = self.averages.values() values.sort(key=lambda x: x.pctdif, reverse = False) return [x.symbol for x in values[:self.coarse_count]] def FineSelectionFunction(self, fine): security_filter = [x for x in fine if x.SecurityReference.SecurityType == 'ST00000001' and x.SecurityReference.ExchangeId != 'OTC' and x.CompanyReference.IsLimitedPartnership == 0 and x.SecurityReference.IsDepositaryReceipt == 0] sorting = sorted(security_filter, key = lambda x: x.ValuationRatios.CashReturn, reverse=False) self.Log(str([x.Symbol.Value for x in sorting[:25]])) self.Log(str([x.SecurityReference.ExchangeId for x in sorting[:25]])) self.Log(str([x.ValuationRatios.CashReturn for x in sorting[:25]])) return [x.Symbol for x in sorting[:25]] class SymbolData(object): def __init__(self, symbol): self.symbol = symbol self.tolerance = d.Decimal(1.01) self.fast = SimpleMovingAverage(3) self.slow = SimpleMovingAverage(45) self.pctdif = 01 def update(self, time, value): if self.fast.Update(time, value) and self.slow.Update(time, value): fast = self.fast.Current.Value slow = self.slow.Current.Value self.pctdif = (fast - slow) / slow

Jing Wu

242.2k ,

Hi Thomas,

The Price property in CourseUniverse Selection is yesterday close price and it is the raw price. It won't change by symbol.SetDataNormalizationMode(DataNormalizationMode.Adjusted). If you would like to select the stocks based on adjusted price, you could filter the symbols after FineUniverse selection. When the securities are being added to your security list, then you could use

self.Securities[Symbol].Close > 5

Thomas Schonig INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

232,035 Quants.

Online Now

Stay Connected