Back

Indicator Approximation for Long Periods (EMA/SMA)

This has been an issue I have been struggling with for a while. There are many well packaged indicators available, who's code appears to be correct, and yet the values do not seem to match any charting software I use. Here is what I have discovered:

There are two types of indicators:
1) Those who's current value is calculated based on its own previous value.
ex: EMA,RSI,ADX,ATR,etc [ema = (1-expConstant) * ema + expConstant * quote;]
2) Those who's current value is not calculated based on its own previous value.
ex: SMA [sum += quote; sma = sum / divisor;]

The problem arises during the iterations when the number of samples is less than the specified period. At this time the values of the indicator will be incorrect, which is why the "ready" bool is added, so that you can wait until it has enough data. For the type 2 indicators listed above, once it is ready it will be 100% accurate. The type 1 indicators on the other hand, will still be basing their values on the previous incorrect values, and so the resultant values will still have some error built in. This bias will be worst at the ready signal, and will approach 0 at infinite sample size.

One solution:

Never re-initialize these indicators at the start of each day, but instead have the values carry over from previous days, and test using a large enough number of days to avoid this bias (based on the largest indicator/timescale you are using).

An example:

Let's assume after 25 periods there would be no bias. (I'm not sure how many periods it would take to reach our .01 accuracy, it would need to be experimentally determined.)
EMA(20) + 1 min scale = 1.3 days until accurate
EMA(20) + 5 min scale = 6.4 days until accurate
EMA(20) + 15 min scale = 19.2 days until accurate
EMA(20) + daily scale = 500 days until accurate
EMA(200) + daily scale = 5000 days until accurate <-- Basically, large indicators cannot be used because of this.

A potential workaround:

If we could input a pre-calculated value when initializing the indicator, it would always be 100% accurate with no need for a "ready" signal delay, at the cost of being more annoying to setup. So potentially this should be an optional field, for those of us who want accuracy and/or no delay. It would go something like this:

ExponentialMovingAverage ema = new ExponentialMovingAverage(200,123.45m);
ema.AddSample(price);
Update Backtest








Hey Kevin, good points. For Type #1 above we can just delay the ready flag for 4-5 periods to quickly solve this issue. The indicators are not reset each day by default, unless you're doing something specific with your code. I think EMA's are always an approximation but you can make them close by initializing with the actual value of price for the first bar.

We've finished the data reshuffle of QC so the "History" functions is only a few weeks away. Once this is done you'll be able to import historical data to make it more accurate. We can even build this History function into the base indicators
1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Yes, at one point I tried to have intraday-only indicators but that did not work well.

But I will try to create a set of no-delay indicators, with initialized values, to see how that goes.

Edit: It just occurred to me, setting this initialized value to 0 would be the equivalent of the current code, so there really is no downside to adding it in.
0

I wrote the current code roughly 6 months ago and its pretty atrocious given what we've learnt in the last 6 months. It will also cause memory overflows as the queue grows forever...

@Michael H's code is better for this and uses a default first value. We're almost finished open sourcing the base code and have already built a suite of indicators into the core so you don't need to do this manually.

If you're in a hurry you can do this manually, otherwise I'd recommend waiting a week for the new QC-Designed & Integrated versions.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Sounds like an early Christmas! But thanks, I'll work on other stuff in the meantime
0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed