Back

Managing Universe and Trades within Algorithm Framework

Hi,

As I'm slowly getting into the framework many questions keep arising! This is what I'm trying to achieve for my framework:

- Every day before the market opens, it selects 5 stocks based on price and volume

- The selection already implies the stocks are long candidates so buy at market open with market orders and equal weighting.

- At the end of the day, we liquidate all those 5 positions. Either using a scheduled event or OnEndOfDay() which I believe is just a scheduled event for 10 minutes before market closes. I guess using a Schedule function is better as it gives more flexibility.

- The above process repeats the next day from scratch and so on.

So as you can see in the attached backtest, I'm failing to liquidate positions and I think the portfolio is just getting bigger and bigger judging by the Alpha Assets. I was also thinking that maybe the problem is that I'm not cleaning up the Universe from the previous day or something and therefore keeps getting bigger and bigger?

Thank you very much for your help!

Emilio

Update Backtest








 From the trade results of this algorithm, long positions are liquidated at the end of the day. You can use the scheduled event to liquidate the existing holdings.

self.AddEquity("SPY")
self.Schedule.On(self.DateRules.EveryDay("SPY"), self.TimeRules.BeforeMarketClose("SPY", 10), self.rebalance)

def rebalance(self):
self.Liquidate()

 "Alpha Assets" chart includes all assets in the entire life of the algorithm. It doesn't reflect the current asset holdings.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hi Jing,

Many thanks for your help.

I have a question regarding a major change between your algo and mine: you selected self.UniverseSettings.Resolution = Resolution.Minute, whereas I was using Daily. I find it hard to understand the timing of all these events.

- When is the UniverseSelectionModel called? Once a day between trading days?

- Also when calling self.Liquidate from main.py, are the stocks sold using the self.execution(ImmediateExecutionModel()) or with a different execution model? Maybe I should schedule a function within the AlphaModel (which I believe runs every minute) for end of day setting InsightDirection.Flat for all symbols in order to make sure it uses the ImmediateExecutionModel.

- I'm getting a lot of "Backtest Handled Error: The order quantity for POM cannot be calculated: the price of the security is zero". How is that possible if one of the conditions from the Universe Selection is price > 20?

Thank you and sorry for all the questions!

Best,

Emilio

0

UniverseSelectionModel is called every day at midnight 00:00 by default.  

self.UniverseSettings.Resolution determines the requested data resolution for symbols returned by the UniverseSelectionModel. If you use daily resolution here, you only get the daily data for those symbols. Therefore, all market orders will be converted to the MakretOnOpen() orders. When you try to place the market order x minutes before the market close, there's no minute price data because you only request the daily data, you'll get the error message like "The order quantity for XXX cannot be calculated: the price of the security is zero". 

Calling liquidate will bypass the execution model and directly sell the assets in your portfolio; we recommend using a flat insight signal or setting the period of the insight to expire before the end of the day.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jing.

Based on your comments, I modified the ConstantAlphaModel accordingly by changing the timedelta parameter of Insight.Price to expire (become flat) 5 hours from time of emiting the insight. As I can see the insight gets created at market open 9.30 and expires at 14.30.

As you can see in the code I set self.UniverseSettings.Resolution = Resolution.Minute in the main script so I should be getting minute data for all symbols now. However, I keep getting the message "Backtest Handled Error: The order quantity for CEO cannot be calculated: the price of the security is zero". I'm no longer using Liquidate either. The only thing I'm thinking could be self.SetPortfolioConstruction(EqualWeightingPortfolioConstructionModel()), which by default has resolution=Resolution.Minute.

Final question if I may, now that I subscribed to Minute for my Universe, all my calculations in the UniverseSelectionModel are based on minute data! I'll have to consolidate, right?

Thank you so much for your help Jing,

Emilio

0


Hi again Jing!

So based on my previous question, my understanding was that I had to request minute data and then consolidate within the UniverseSelectionModel in order to base my selection on daily data. You can find attached my attempts to do so. I had to comment out all the lines related to the data consolidation as it was giving me errors and could not attach the algo. Error message is Runtime Error: Exception : Please register to receive data for symbol 'A RPTMYV3VC57P' using the AddSecurity() function.....

As you can see I was using something like this within the SymbolData Class:

self.priceSMA10 = SimpleMovingAverage(10)

self.Consolidator = algorithm.ResolveConsolidator(self.Symbol, resolution)

algorithm.RegisterIndicator(self.Symbol, self.priceSMA10, self.Consolidator)

and then simply this within the SelectCoarse function:

for cf in coarse:

if cf.Symbol not in self.calculations:

self.calculations[cf.Symbol] = SymbolData(algorithm, cf.Symbol, self.resolution)

 

Please let me know your thoughts when you have some time.

Thanks!!

Emilio

0


UniverseSelectionModel is set to run once at 00:00 every day by default. You don't need the consolidator here as the resolution of UniverseSelectionModel is daily.  "SelectCoarse(self, algorithm, coarse):" method is called every day. "cf.Price, cf.Volume, cf.DollarVolume" those values are all the daily value. cf.Price(raw price) and cf.AdjustedPrice(adjusted price) are all yesterday's daily price. Therefore, all indicators are updated with the daily value. 

"self.UniverseSettings.Resolution = Resolution.Minute" request the minute data for symbols returned by universe model. It means once the universeSelectionModel finish the selection based on the daily price/volume/fundamental factors, those symbols will be returned in fine universe selection

def SelectFine(self, algorithm, fine):
       return [ x.Symbol for x in sorted_fine[:20]]
#### might be a bug in your algorithm
#### should use sorted_fine instead of filtered_fine

With minute resolution setting in self.UniverseSettings.Resolution, you'll get the minute data in alpha model for those 20 symbols returned in SelectFine(). 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jing. I understood now and corrected the algo accordingly.

The attached is using self.UniverseSettings.Resolution = Resolution.Minute, but I'm still receiving the warning message 

"Backtest Handled Error: The order quantity for DOW cannot be calculated: the price of the security is zero" many many times during the backtest!

Emilio

0


You should add a price check in alpha model to stop sending the insight for symbols with zero price. Otherwise, you'll place orders based on insights combined with ImmediateExecutionModel().

def Update(self, algorithm, data):
insights = []
if self.ShouldEmitInsight:
for security in self.securities:
if algorithm.Securities[security.Symbol].Price != 0:
insights.append(Insight.Price(security.Symbol, self.period, InsightDirection.Up))
# self.ShouldEmitInsight = False should be out of the for loop
self.ShouldEmitInsight = False
return insights
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks!!

One quick question Jing please. How can I warm up those indicators within the UniverseSelectionModel so that I can at some point do paper trading live? I'm having trouble figuring that out!

Thanks again!

Emilio

0

You can request the history price to warm up the indicator manually with indicator.Update(). Please refer to this algorithm 

https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/WarmupHistoryAlgorithm.py

Please be wary of large requests. Once you get over about 120K data points you'll run into timeout issues. For reference, coarse data set is about 7K symbols, so even 30 days at the daily resolution on all symbols is 210K data points. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jing,

I'm trying the below but get TypeError : object is not callable. I thought it'd be something similar to this:

class UniverseSelectionModel(FundamentalUniverseSelectionModel):

    def __init__(self,
                 universeCount = 20,
                 filterFineData = True,
                 universeSettings = None,
                 securityInitializer = None):

        super().__init__(filterFineData, universeSettings, securityInitializer)
        self.universeCount = universeCount
        
        # Holds our coarse fundamental indicators by symbol
        self.calculations = {}

    def SelectCoarse(self, algorithm, coarse):
        
        '''Performs coarse selection based on price and volume'''
        
        # We are going to use a dictionary to refer the object that will keep the calculations
        for cf in coarse:
           if cf.Symbol not in self.calculations:
                self.calculations[cf.Symbol] = SymbolData(cf.Symbol)

          history = algorithm.History([cf.Symbol], 50)
          self.calculations[cf.Symbol].update(cf.EndTime, history, cf.Price, cf.Volume, cf.DollarVolume, cf.HasFundamentalData)

# Class used to improve readability of the coarse selection function
class SymbolData:
    def __init__(self, symbol):
        self.Symbol = symbol
        self.priceWin = RollingWindow[float](5)
        self.priceSMA10 = SimpleMovingAverage(10)
        self.priceSMA50 = SimpleMovingAverage(50)
        self.volSMA = SimpleMovingAverage(20)
        self.is_uptrend = False

    def update(self, time, history, price, volume, dollarvolume, fundamentaldata):
        self.price = price
        self.volume = volume
        self.dollarvolume = dollarvolume
        self.fundamentaldata = fundamentaldata
        
        for index, row in history.loc[self.Symbol].iterrows():
            self.priceSMA10.Update(index, row["close"])
            self.priceWin.Add(row["close"])
            self.priceSMA50.Update(index, row["close"])
            self.volSMA.Update(index, row["volume"])

        if self.priceWin.IsReady and self.volSMA.IsReady and self.priceSMA10.IsReady and self.priceSMA50.IsReady:     
            SMA10price = self.priceSMA10.Current.Value
            SMA50price = self.priceSMA50.Current.Value
            SMAvol = self.volSMA.Current.Value

            # Here we can add the criteria for our universe
            self.is_uptrend = (SMA10price > SMA50price)

0

Hi Jing,

So don't worry about the above question as I believe I sorted that out with the below code. Also, following your advice regarding timeout isses, I modified the code so that it first looks at the whole universe and chooses a few based on basic filters and then at the end calculates de indicators only for those securities.

However, now I keep getting this error Runtime Error: ArgumentException : This is a forward only indicator: SMA5 Input: 2015-12-05 00:00:00Z Previous: 2016-01-01 00:00:00Z

Any idea what I might be doing wrong?

Thanks a lot!

Emilio

 

from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Common")
AddReference("QuantConnect.Indicators")
AddReference("QuantConnect.Algorithm.Framework")

from QuantConnect.Data.UniverseSelection import *
from QuantConnect.Indicators import ExponentialMovingAverage
from Selection.FundamentalUniverseSelectionModel import FundamentalUniverseSelectionModel

import decimal as d

class UniverseSelectionModel(FundamentalUniverseSelectionModel):

def __init__(self,
universeCount = 20,
filterFineData = True,
universeSettings = None,
securityInitializer = None):

super().__init__(filterFineData, universeSettings, securityInitializer)
self.universeCount = universeCount

# Holds our coarse fundamental indicators by symbol
self.calculations = {}

def SelectCoarse(self, algorithm, coarse):

'''Performs coarse selection based on price and volume'''

filtered = [x for x in coarse if x.HasFundamentalData and x.Volume > 500 and x.Price > 50]
algorithm.Debug(len(filtered))

# sort the stocks by dollar volume and take the top 1000
top = sorted(filtered, key = lambda x: x.DollarVolume, reverse = True)[:100]
algorithm.Debug(len(top))

# We are going to use a dictionary to refer the object that will keep the calculations
for cf in top:

algorithm.Debug(cf.Symbol)
history = algorithm.History([cf.Symbol], 20, Resolution.Daily)
algorithm.Debug(history)

if history.empty:
algorithm.Debug('no history')
continue

if cf.Symbol not in self.calculations:

self.calculations[cf.Symbol] = SymbolData(cf.Symbol)

self.calculations[cf.Symbol].update(cf.EndTime, history, cf.Price, cf.Volume, cf.DollarVolume, cf.HasFundamentalData)

# Filter the values of the dict based on chosen criteria.
filtered_coarse = list(filter(lambda x: x.is_uptrend, self.calculations.values()))
algorithm.Debug(len(filtered_coarse))

# Sort the values of the dict: we want those with greater DollarVolume
sorted_coarse = sorted(filtered_coarse, key = lambda x: x.dollarvolume, reverse = True)

# Select Top Stocks (based on universeCount)
return [ x.Symbol for x in sorted_coarse[:self.universeCount] ]


# Class used to improve readability of the coarse selection function
class SymbolData:

def __init__(self, symbol):

self.Symbol = symbol

self.priceWin = RollingWindow[float](5)

self.priceSMA10 = SimpleMovingAverage(5)
self.priceSMA50 = SimpleMovingAverage(10)

self.volSMA = SimpleMovingAverage(5)

self.is_uptrend = False

def update(self, time, history, price, volume, dollarvolume, fundamentaldata):

self.price = price
self.volume = volume
self.dollarvolume = dollarvolume
self.fundamentaldata = fundamentaldata

for index, row in history.loc[str(self.Symbol)].iterrows():
self.priceWin.Add(row["close"])
self.priceSMA10.Update(index, row["close"])
self.priceSMA50.Update(index, row["close"])
self.volSMA.Update(index, row["volume"])

if self.priceWin.IsReady and self.volSMA.IsReady and self.priceSMA10.IsReady and self.priceSMA50.IsReady:


SMA10price = self.priceSMA10.Current.Value
SMA50price = self.priceSMA50.Current.Value
SMAvol = self.volSMA.Current.Value

# Here we can add the criteria for our universe

self.is_uptrend = SMA10price > SMA50price
0

It looks like you update the indicator with history price repeatedly. You should only request the history to initialize the indicator once for newly added symbols. For symbols already exists in the dictionary, the update method should only update the indicator with new bars.

for index, row in history.loc[str(self.Symbol)].iterrows():
self.priceWin.Add(row["close"])
self.priceSMA10.Update(index, row["close"])
self.priceSMA50.Update(index, row["close"])
self.volSMA.Update(index, row["volume"])

The above code to initialize the indicator should be out of the update() method.

For example, you can create a method in SymbolData: class named WarmUpIndicator, and then pass the history price to this method and call this method each time you add a new symbol to the dictionary. Once the indicator is initialized for this symbol, this method should not be called anymore. You only need to update the indicator with the current price/volume data in update() method. Your update() method now passed parameters cf.Price, cf.Volume, cf.DollarVolume but didn't use them at all.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Brilliant thanks! Got it to work :)

Thanks Jing,

Emilio

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed