Discussion Forum - QuantConnect.com

Ab DeWeese INVESTOR

| Reward Discussion

Hi,

New guy here. I am writing a first pass algorythm to learn the platform and test some basic ideas. Is this the right way to initialize my indicator groups?

I have a class called JustSomeIndicators which is a group of indicators. I have a factory method called CreateIndicatorGroup() which creates instances of the JustSomeIndicators class. I iterate over this factory method in a loop and hang on to the resulting objects in a dictionary called IndicatorsForEachSymbol.

The trouble is I keep getting exceptions in the OnData callback when I try to index the IndicatorsForEachSymbol dictionary using the symbol obtained from the Slice obect.

Am I going about this the right way?

PS: I have 300 or so lines of code but have shared what I hope is a relevant subset.

private Dictionary<Symbol, JustSomeIndicators> IndicatorsForEachSymbol = new Dictionary<Symbol, JustSomeIndicators>();



public override void Initialize()

{

   SetWarmUp(TimeSpan.FromDays(Math.Max(SAMPLE_SIZE, 50)));

   SetStartDate(2019, 1, 1);

   SetEndDate(2020, 1, 1);

   SetCash(100000);

   ConfigureUniverse();

   SetExecution(new ImmediateExecutionModel());

   SetPortfolioConstruction(new NullPortfolioConstructionModel());

}



private void ConfigureUniverse()

{

   AddUniverse(CoarseSelectionFunction, FineSelectionFunction);

   UniverseSettings.Resolution = Resolution.Daily;

   UniverseSettings.DataNormalizationMode = DataNormalizationMode.Raw;

}



public IEnumerable<Symbol> CoarseSelectionFunction(IEnumerable<CoarseFundamental> coarse)

{

   return coarse

      .Where(x => x.HasFundamentalData && x.DollarVolume > 10000000 && x.Price > 2)

      .OrderByDescending(x => x.DollarVolume)

      .Select(x => x.Symbol)

      .ToList();

}



public IEnumerable<Symbol> FineSelectionFunction(IEnumerable<FineFundamental> fine)

{

   return fine

      .Where(x => x.MarketCap > 200000000)

      .OrderByDescending(x => x.MarketCap)

      .Select(x => x.Symbol)

      .ToList();

}



public override void OnSecuritiesChanged(SecurityChanges changes)

{

   foreach (var security in changes.AddedSecurities)

   {

      IndicatorsForEachSymbol.Add(security.Symbol, CreateIndicatorGroup(security.Symbol));

   }

   base.OnSecuritiesChanged(changes);

}



public override void OnData(Slice data)

{

   foreach (Symbol symbol in data.Bars.Keys)

   {

      JustSomeIndicators indicators = IndicatorsForEachSymbol[symbol];

      indicators.AddTradeBarToFifo(data.Bars[symbol]);

      bool isHeld = false;

      if (Portfolio.Securities.ContainsKey(symbol))

         isHeld = Portfolio.Securities[symbol].HoldStock;

      if (isHeld)

         ConsiderSelling(symbol, data.Bars[symbol], indicators);

      else

         ConsiderBuying(symbol, data.Bars[symbol], indicators);

   }

}

Update Backtest

Ab DeWeese

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Conor Flynn

2.3k ,

I'd recommend using historical data to warm up the custom indicators. Similar to this:

// note here we assume the length of the indicator is: length_indicator

// note we assume it has an update method which directly affects the output of the indicator

// here is an example which would occur in the OnSecuritiesChanged method for

// each added security



// initialize the indicator

JustSomeIndicator indicator = JustSomeIndicator(security.Symbol);



// get historical data of length_indicator days for the desired resolution

var history = History(security.Symbol, length_indicator, Resolution.Daily);

// loop through each bar in the historical data

foreach(var bar in history) {

  indicator.update(bar.Open, bar.Close, bar.High, bar.Low, bar.Volume);

}

Hopefully that helps :)

Ab DeWeese

151 ,

Thanks for your idea. I incorporated that into my code but unfortunately it does not address the issue at hand.

This morning I am fighting an exception (pasted below) in the OnSecuritiesChanged() callback. Any ideas? It seems there are duplicate securities or duplicate symbols.

I'm not sure I understand what is going on under the hood. Guessing that its something like this: The universe is defined including the coarse and find filters and then AFTER the this.Initialize() method returns some code in the background somewhere populates the this.Securities property using the universe definition I supplied. This then triggers the OnSecuritiesChanged() callback which I am guessing fires a single time (not in some huge loop). I then get a list of symbols I can work with inside trhe method body from changes.AddedSecurities.Select(x=>x.Symbol). I use these IEnumerable<Symbol> to iterate over calls to this guy IndicatorsForEachSymbol.Add(security.Symbol, CreateIndicatorGroup(security.Symbol));

Should I add a .Distinct() call to changes.AddedSecurities.Select(x=>x.Symbol) so its of the form changes.AddedSecurities.Select(x=>x.Symbol).Distinct()

Stacktrace: "System.ArgumentException: An item with the same key has already been added. Key: BRKB R735QTJ8XC9X
at System.Collections.Generic.Dictionary`2[TKey, TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) [0x000ad] in :0
at System.Collections.Generic.Dictionary`2[TKey, TValue].Add (TKey key, TValue value) [0x00000] in :0
at QuantConnect.Algorithm.CSharp.JustTheAlgorithm.OnSecuritiesChanged (QuantConnect.Data.UniverseSelection.SecurityChanges changes) [0x00030] in <4cd3b201199040b7b04766d79f265f63>:0"

Ab DeWeese

151 ,

Its throwing at IndicatorsForEachSymbol.Add(s, CreateIndicatorGroup(s)), yelling at me that "An item with the same key has already been added. Key: BRKB R735QTJ8XC9X" to the collection IndicatorsForEachSymbol.

...after a bit of tseting... it appears the OnSecuritiesChanged() callback is called several times instead of being called once as I assumed. I'm confused why BRKB would be added twice..? Can someone please help me understand the execution model going on behind the scenes?

Here is an updated snipet.

        public override void OnSecuritiesChanged(SecurityChanges changes)

        {

            IEnumerable<Symbol> symbols = changes.AddedSecurities.Select(x => x.Symbol).Distinct();

            foreach (var s in symbols)

                IndicatorsForEachSymbol.Add(s, CreateIndicatorGroup(s));

            IEnumerable<Slice> historicalSlices = History(symbols: changes.AddedSecurities.Select(x => x.Symbol), periods: Math.Max(SAMPLE_SIZE, 50), resolution: Resolution.Daily);

            foreach (Slice slice in historicalSlices)

                UpdateIndicators(slice);

            base.OnSecuritiesChanged(changes);

        }

Ab DeWeese

151 ,

Figured it out! :)

A little Google-foo turned up this helpful thread which explains the execution model. TL/DR: both universe filters AND OnSecurityChanged are called everyday. I assumed they would each be called only once at the beggining. Not so.

https://www.quantconnect.com/forum/discussion/6485/onsecuritieschanged-questions/p1

Here is my snipet updated to only add new securities.

        public override void OnSecuritiesChanged(SecurityChanges changes)

        {

            IEnumerable<Symbol> newSymbols = changes.AddedSecurities.Select(x => x.Symbol).Distinct();

            IEnumerable<Symbol> alreadyHaveTheseSymbols = IndicatorsForEachSymbol.Keys;

            IEnumerable<Symbol> newSymbolsToAddIfAny = newSymbols.Except(alreadyHaveTheseSymbols);

            foreach (var s in newSymbolsToAddIfAny)

                IndicatorsForEachSymbol.Add(s, CreateIndicatorGroup(s));

            IEnumerable<Slice> historicalSlices = History(symbols: newSymbolsToAddIfAny, periods: Math.Max(SAMPLE_SIZE, 50), resolution: Resolution.Daily);

            foreach (Slice slice in historicalSlices)

                UpdateIndicators(slice);

            base.OnSecuritiesChanged(changes);

        }

Ab DeWeese INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

241400 Quants.

Online Now

Stay Connected

Right way to initialize custom indicators when using universe selection

Started By:

Join QuantConnect for Free Sign Up

Platform

SIGN IN

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

241400 Quants.

Online Now

Stay Connected

Right way to initialize custom indicators when using universe selection

Started By:

Join QuantConnect for Free Sign Up