# Algorithm Framework

## Algorithm Scoring

### Introduction

The Algorithm Framework performs real-time analysis into the effectiveness of your strategy; calculating various scores on your algorithm. These scores can be used to help you quickly identify areas of weakness to improve your models. There are three key scores calculated based on the **Alpha Model** insights: Direction Score, Magnitude Score and Estimated Alpha Value.

Scoring an algorithm is a balance of an art and a science. Between the start of your Insight signal and the end of your timeframe there are infinite variations in the price movements. To address this we've created scoring functions which assign weight based on time.

### Direction Score

The direction score is a measure of the *directional* accuracy of the predictions of your algorithm. When you create an Insight from an Alpha Module you create a prediction the market will move Up or Down. If your prediction is correct during your insight timeframe, you receive a positive score between [0, 1]. If the asset moves in the wrong direction you will receive a 0-score.

The scoring system only reviews the moment of the insight completion giving you a binary score of 1 or 0 if you correctly predicted the direction. The sum of the insights for the backtest is averaged to give your overall direction score as a percentage.

### Magnitude Score

An insight can optionally set the expected magnitude change of the asset over the insight period. This expected return can be used in the Portfolio Construction model to improve results.

The magnitude score is calculated based on the distribution of the magnitude over the insight period. If your magnitude is correct at the endtime you will receive a high weighting. If in the time leading up to your endpoint you achieve the target magnitude you'll receive weighting towards your score but not as much.

If your insight immediately achieves the target magnitude and remains there for the duration of the period you will receive a perfect score.

### Estimated Alpha Value

To assign an approximate value of the revenue potential for framework algorithms we calculate the mean insight value. When an insight is created and successfully fulfills its expectations there is potential for a profit. If an investor had followed the signal blindly, and exited on completion of the insight period, the resulting gain or loss is the *Insight Value*. The insight value is calculated as:

*Insight Value = Insight Price Change x Volume Depth Available*

The estimated algorithm hypothetical value is the sum of these insight values calculated on a monthly basis. Over time we hope to improve the Insight Value formulas to give you an estimate potential Alpha Streams licensing revenues. It is important to note this is entirely hypothetical and backwards looking estimations.

### Insight Confidence

Insights can optionally be assigned a confidence score. This is an indication of the strength of evidence for a specific insight. Models consuming insight scores can use these confidences to assign more weight to high confidence expectations.

When calculating the confidence of an insight you should try and apply statistical techniques. How frequently has this pattern or input resulted in a successful prediction? If you normalize your signal into a standard distribution, how often is the signal reach this threshold?