Back

Machine Learning - TensorFlow Basic Example

"TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them." - TensorFlow Website

One of my favorite things about QuantConnect is the access to some very powerful Python modules not seen before in other quant platforms. One example is TensorFlow. Although usually associated with Deep Learning, TensorFlow can be used generally for numerical computation and comes with some great tools.

Today, my goal is to share the first TensorFlow tutorial, based on the MNIST For ML Beginners which offers a great and basic walkthrough for people new to Machine Learning and or TensorFlow. 

The change in price, over N steps, is used as input features to determine the change in price from step N to step N+1. Using Opening Price, the goal is to forecast the change from one market open to the next to allow for a daily rebalance to the target class(Cash or Long) for a single asset.

Please note, the attached backtest is more academic in that a softmax regression model is trained and then used to predict probabilities per class(Cash or Long positions). Little is done to attempt to achieve success or manage the training in an intelligent way to ensure a quality results or best practices in Machine Learning; making this a good starting point to explore and learn such things(learning rates, dropout, regularization, mini batch inputs, test train splits, cross validation, log loss vs accuracy, feature scaling, non overlapping features, etc). The "positive" result, I'm guessing, has more to do with the model favoring the long class for an appreciating assets vs any deep(darn a pun, we were so close to the end) insight.

Update Backtest






The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



I Made some mistakes, Please use this new backtest!

The larger History call in made in the algo's init and append to it later on each rebalance with an additional, single day History call. These were intended to BOTH request a Daily resolution as the model is used to forecasted/rebalance at a daily level. Sorry, I had some leftover code from when the algo was pure daily mode. But as E.P. Chan often stresses(and it just makes sense); you want to backtest at a faster resolution than you are generating signals, as it can have a big impact on executions: Beware of Low Freq Data, Really Beware of Low Freq Data

I also changed from a Line to Scatter plot to avoid line smoothing in the probability plots, as that was hiding a great deal of info.

2


The accuracy calculation may be wrong due to my addition of inference (actually making predictions with the model on out of sample data with no label). While the cross_entropy is calculated with its own internal softmax (which can be carefully deduced from the function name: softmax_cross_entropy_with_logits), an output layer(in this case softmax) was needed for us to get nice probabilities from the network outside of training.

While the argmax in self.correct_prediction might act the same in this binary classification case, the main issue here is understanding where the network ends and what the output layer is as to correctly use that tensor for things like metric operations or inference. I ran into this problem when extending the algo to work for multiple assets and noticed my output probabilities had a tiny scaling error(-9000 is NOT in the range 0 to 1).

To fix this I changed:

self.correct_prediction = tf.equal(tf.argmax(self.y,1), tf.argmax(self.y_,1))

to

self.correct_prediction = tf.equal(tf.argmax(self.y_pred,1), tf.argmax(self.y_,1))

There is another discussion to be had on how useless it is to calculate training_accuracy as shown in the algo vs calculating test or validation accuracy. But as mentioned before, these next steps would be great practice for people new to ML now that the ops exist in the algo. 

Such a thing would be as simple as adding a train_test_split to the end of Model.preprocessing, and then using these new arrays to perform runs for accuracy calculations on the test set while training on the train set. 

1

Is there a way to checkpoint a model during backtesting? I (rightfully so) get a permission denied response when I attmept to do so. (just added a saver to your example)

Conversely, is there a way to upload a checkpoint?

1

 Nolan English file io is a current limitation. You might have luck training locally and then downloading from dropbox or any server the weights of a network, but until we can easily pickle things working with ML models will be messy. This is a lower priority todo.

0

Basic TensorFlow Algorithm - Single Asset Classifier

Attached here is my current version of the single asset algo.

I have applied some of the fixes and improvements I mentioned above. Most importantly are the examples of charts for accuracy and loss with a test train split which will be the focus of designing, tuning, and diagnosing network performance going forward(nice feature QC!). Areas of poor test-loss seem to correlate to the drawdown periods, that is rather promising and could act as a filter for engaging the signal more efficiently or directing the duration of training per OnData step. More on the way!

2


Pretty epic update Derek Tishler! Nice to see a full use of the charting library as well =)

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Basic TensorFlow Algorithm - Multi Asset Classifier

What if, instead of forecasting the probability of one asset's Cash or Long classes, we extended the problem to a multi asset case? Currently, we are setup to predict the 1 class to which the asset's input belongs. Instead, lets treat this as a multi-label problem and see which, if any, assets have a probability of being relevant(has an open-to-open percent gain larger than some value) to the input. 

This becomes the same problem as trying to figure out which labels describe and image. Multi label networks exist all over, so let's once again piggy back on some open source efforts. Here is a great tutorial showing how to adjust the Inception network to conform to the same problem we are working on:
Multi-label image classification with Inception net - Radek Bartyzal

This setup uses 20 of the top market cap stocks (as of Nov 14, wow much bias, but easy to work with this universe to prevent busy data handling). The intent is to classify, every day on open, which of these assets are expected to return 0.1% by the next open and re balance accordingly. Once again this is just a technical demo and was not refined to be a trade-able strategy, But for those interested in applying more advanced models/neural networks to the market, this could be a fun place to start! These example algos are slowly building to a more formal tutorial / set of examples.

Note: Multi label classification can be approached in a number of ways. We also could have tried an ensemble of single asset classifiers models, but this seemed more tensor-y.

Note: While the training doesn't appear to be going so well, the result happens to be interesting considering the low beta, number of trades & high turnover, and a high potential capital capacity.

2


Im getting a few questions on debugging. Here are some tips for two common ones:

  1. Using Schedule can be problematic when debugging. It seems to change the way the stack trace reports and you loose most the info, I suggest using OnData to test fire Rebalance if things get weird. You may not get a line number still, but hopefully more info.
  2. These algos use many steps worth of historical data to build a nubmer of samples to train the model with. Changing the start date, or adding assets without ensuring data exists(not just at start date but at start date minus the 4-5 years of requested data!) the algo will start to throw errors (see below) as empty arrays begin to show up.
BacktestingRealTimeHandler.Run(): There was an error in a scheduled event AAPL: EveryDay: AAPL: 0 min after MarketOpen. The error was Python.Runtime.PythonException: IndexError : index 30 is out of bounds for axis 0 with size 30

Also, please note that this thread now contains two separate examples posted just above this comment:
Basic TensorFlow Algorithm - Single Asset Classifier (Based on TesnsorFlow intro MINST classification)
Basic TensorFlow Algorithm - Multi Asset Classifier (Extended slightlly for multi-label classification)

0

Very interesting stuff. You say little is done to attempt to achieve success or manage the training in an intelligent way to ensure a quality results or best practices in Machine Learning, how would you go about making this a more tradeable strategy, improving the training, and not just an exercise

2

DainR that is a great question. I will try to touch on a few points with my opinions:

Achieve success:

Many portfolios/assets/algorithms are usually created or selected based on a set of constraints. Average annual return, annualized share, turnover frequency and statistical relevance, a target alpha and or beta, drawdowns, volatility, and many other possibilities. I addressed some of this by rebalancing each morning and selecting features that resulted in a model that traded often, but is very possibly overfitted as a look-back window corresponding to good returns was searched for.

A breakdown of the single asset strategy shows many problems with these sort of constraints. A high beta. A low sharpe. The number of trades is very interesting considering the result, but toward the end the model hardly trades at all. Despite a nice green market/returns, the model begins to loose statistical relevance here as very little trade data begins to come in. That would be troubling in live trading, especially as the beta begins to explode and alpha drops.

The multi asset strategy has some interesting features, such as making the broker rich. Despite the 10k+ trades it performed on the top 20 market cap assets it achieved a very low beta and some returns. Kinda cool! But position concentration is not considered, the weight vector can easily normalize into fully investing in 1 or few assets. Not ideal for risk management. But easily corrected with a Clip function or different weighting schema.

Additionally, both models are unstable if you move dates or params around and you, for the moment, are unable to persist the model into live trading to easily test currently trained models reliably.

To address this one could try things like: attempt to input more meaningful features and labels, adjust model inference timing, smoothing of input or outputs of network, non overlapping features(in my case the siding window of price changes), and using risk management to prevent losses or capture profits.

Manage training:

This one is a bit broad. Training any model is tricky. You have to consider the input, the architecture of the network, methods the network handles things like over fitting(be it through things like dropout, regularization, etc) and ways in which QC and your executions need to progress to happen at the right time. There is also optimizer selection and learning rate scheduling(this is a big one because the model is constantly learning). It can be a time consuming process to tune any machine learning model to a specific domain or problem.

To bring it back to the algos I have shared, the accuracy and loss charts do not exactly show that healthy training is progressing. By playing with: data, architecture, learning rates, and really just every aspect of the network we can then use these charts to try and diagnose the training.

Ideally, we would live trade a model(or an ensemble of such models) if it shows good convergence to a decent accuracy and performed consistently in and out of sample. We also have the added challenge of continuous training, so steps would have to be taken to ensure a gradient doesn't explode or something else that could suddenly derail the model. One challenge is to keep the code/processes simple to prevent creating additional points of failure.

1

Hi,

Im new to Quantconnect and I find this Machine learning very interesting. I am starting eith the basics of Tensorflow i.e. Installing Anaconda, etc...

I am used to dealing with Forex and not with equities. My question is: Can this be modified and be ported to Forex.

As I said I'm still learning and I will be exploring that avenue. If you have some insights kindly let me know.

Thanks

Jeremie

1

Thanks for this article. I would like to use TensorFlow from C#, since most of my algos are now written in C#. 

Do you know if it's feasible?

1

Jeremie Jean Simonet sure Forex work well for me using TF, same for Crypto. The issue there would be switching to your desired forecast period and not overloading things with a higher frequency setup. It's easy to want to drop into second or minute mode and give a network 100k+ samples...but that much history and the time to train on a large dataset are going to crush the backtest and would be difficult to recreate live considering computational limits. These examples use the market open, and daily historical data, to perform daily forecasts. You would want to fire off the rebalance function every n_minutes or something like that when dealing with a more continuous exchange.

These examples are pretty data agnostic which is a weakness but also a big point mentioned above; I slapped down a trivial TF model when the real value of ML often comes from the exploitation of quality data feature/input selections, preprocessing(including ways to increase or generalize dataset), and the ways in which training is undertaken. These are all things that must be carefully tuned to each problem/network, which can be extremely time consuming.

Erik Bengtson TF does not have an official c# version from what I understand. But, they do have APIs for: Python, C, Java, and Go from the looks of their site. It appears github has a c# wrappers for the c api. This TF c# issue discussion might give much more info and options beyond this one repo. I assume the C API + wrapper could theoretically be added to the cloud in the future, but if you wanted to mess with it soon I think it would require local usage of Lean to achieve c#.

0

Derek, thanks for linking me to that. So far I have tried to call python code from C# code but it works only partially, and I have throught in another option which would be running tensorflow in a webserver, and make the calls through HTTP.

Here is the code and error for the C# to python, for sake of completeness. 

Failed to initialize algorithm: Initialize(): Python.Runtime.PythonException: AttributeError : '1, Culture=neutral, PublicKeyToken=null]]' object has no attribute 'open'

at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in :0 Here is the C# code. The python code is on the link.using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using QuantConnect;
using QuantConnect.Data.Market;

using QuantConnect.Algorithm;
using Python.Runtime;
using QuantConnect.Data;
using System.Net;

namespace Quantability.Algorithm.CSharp.PythonTest
{
public class PythonTest : QCAlgorithm
{
// Url of dynamically loaded Python code
static string moduleUrl = "http://www.mocky.io/v2/5a40be643200000d09ac33fb";
// Python module name
static string moduleName = "TestModule";
// Symbol of Equity
static string ticker = "SPY";
// Reference to dynamically loaded Python module
static dynamic moduleRef;
dynamic model;
public override void Initialize()
{
SetStartDate(2007, 10, 7);
SetEndDate(2007, 10, 15);
AddSecurity(SecurityType.Equity, ticker, Resolution.Daily);

// Run initialize method in Python function
Debug("Loading Python Module");
PythonEngine.Initialize();
PythonEngine.BeginAllowThreads();
var s = this;
using (Py.GIL())
{
moduleRef = PythonEngine.ModuleFromString(moduleName, PythonModuleCode(moduleUrl));
moduleRef.InvokeMethod("create_model");
PythonEngine.RunSimpleString("print('Python Module Execution Successful')");
// IEnumerable<Slice> slices = History(100, Resolution.Daily);
// IEnumerable<TradeBar> bars = slices.Get(ticker);
//IEnumerable<decimal> decimals = slices.Get(ticker,Field.Close);
// moduleRef.SetHistory(bars);
moduleRef.LoadHistory(s,ticker,100,Resolution.Daily);
moduleRef.Train();

}

//
// model.setHistory(this,history.ToFloatArray());
//model.loadHistory(this,ticker);
//model.LoadHistory(this,ticker,100);
}

private static string _pythonModuleCode = "";
// Only retrieve file via Http once.
public static string PythonModuleCode(string url)
{
if (_pythonModuleCode.Length == 0)
{
using (var client = new WebClient())
{
_pythonModuleCode = client.DownloadString(url);
}
}
return _pythonModuleCode;
}

public void OnData(Slice data)
{
if (data.ContainsKey(ticker))
{
//decimal allocation = GetAllocation(data[ticker]);
SetHoldings(ticker, GetAllocation(data[ticker]));
}
}

// Pass current price bar into Python module for entry/exit decisions.
public decimal GetAllocation(TradeBar bar)
{
decimal ret = 0m;
using (Py.GIL())
{
ret = moduleRef.scan_entry(bar.Close);
}
Debug("Return value: " + ret);
return ret;
}

}
}
0

Happy New Year everyone!

Derek, that's some impressive work you did there with TF! One question - how would you pass multi-dimensional inputs to TF, say price + some technical indicator, or a HLOCV bundle? Reading the TF documentation, TF should be able to accept n-dimensional tuples as inputs, but I'm at a loss as to how that can be implemented

0

Daniel Nedelcu TF, and these example, are essentially data agnostic(sorta). Above, each sample used for training is a set of inputs: x1, x2, ..., x_n-1, x_n. In my example, I just tossed in recent n price changes for each time step to create a dataset (not ideal, very noisy and overlapping inputs may have little alpha assuming you are not working with something like a LSTM and temporal domains for time series or something similar). I could have more quickly np.vstack'ed the data or some vector op. but I wanted to play out the building of the datasets to help others drop in their samples later on.

I perform this work in preprocessing where I create the features(should have been called samples) and the corresponding labels. Like I did with the historical price data, you can work to assemble Fundamental or any sort of data for the universe of assets and build a dataset from that.

If you are having trouble experimenting with TF and new datasets, then I highly suggested testing network stuff locally(completely outside Lean and QC). This makes is much easier to focus on debugging and understanding a network. You can pass in zeros(cause only shape matters to 'fire' the network) and get the network architecture set before taking the next steps with your data.

0

I've gotten this sort of ported over for crypto I think someone was talking about.  If someone else wants to take a look at trying to optimize it, make it far better.   Also I have no clue what I'm doing, a lot is copied and pieced together from places on here. But its fun to play with, will try working on random forests next time with a multi-indicator deal.  Bed time lol 

0


Also I'm not sure why it isn't rebalancing with the regression to the ETH, but its probably something to do with it messing up with thinking it has funds to spend.. too late for my brain to think but have fun. 

0


Smoothed out the training a bit just by wacking way up the amount of learning/decreasing the trading time.  Also fixed some other bits in the selling/buying part for escaping gdax fees.  Fun stuff.  Still reading that tutorial to add actual additional pieces rather than just messing around with adding more data to smooth out the lines.  Still freaks me out that you don't know whats going on behind the scenes and different results from each trial due to the random parameters haha.   Thanks a ton Derek. 

1


i was wonder if i will be able to get this running on the offline Lean and then distrubuting the tensorflow work load to multiple GPU calculation?

Anyone tried? Cos i am running the windows version and tensorflow doesn't work on python 2.7 on windows.

0

Here is my attempt to provide tensorflow multiple pieces of info per timepoint - in this case, I train the TF on the changes in high/open/low/close/volume from the previous day, rather than on changes in price alone.

3


Derek Tishler

I am both new to quantconnect and to machine learning when it applies to backtesting but let me ask you this fundamental question, which I don't quite understand yet: Why do you train the machine learning algo in the backtester? 

My current understanding of the backtester is that it will recalculate the profit and the weights of the ML algo one rebalance date at a time. That sounds to me like a big for loop wich could be avoided by just having a big matrix where each sample index is the 'i'th rebalancing date. Doesn't it make more sense to train the machine learning algorithm offline / in the research lab and then once trained, plug in the optimized weights into the algorithm?

Sorry for that newbie question, but I am myself trying to figure out the best way to apply machine learning to trading and would like to understand the logic behind using the backtester instead of the research lab.

0

Custos Sure we would ideally train the model offline and then use it to perform inference in an algo. The only reason a real-time training was done via the backtester was that it is the only way to play with TF in QC right now and see results. The Research or Lab has no way to persist(save) a model as mentioned above so this was just a way to get things going. You will want to use Lean and your own data on a local machine to train and then use a machine learning model in live trading easily as I mentioned to Nolan above.

1

Thank you Derek, that makes things clearer for me.

0

So untill now there's no way to save a model (pickle or similar)? I'm starting with QC but it seems almost impossible to work with Machine Learning without pickling.

0

Wow, very interesting indeed. Thanks Derek Tishler !

0

I'm very new to this. I'm getting this error after I cloned the project in the second post in this thread.

How do I get QuantConnect to know about tensor flow?

0

When I run the strategy. It doesn't work and keep reminding me: Build Error: File: n/a Line:0 Column:0 - import tensorflow as tf. How to solve the problem?

0

Ravinia  and  ForeverYoungNow , since a change in the python environment you now have to be careful in what order you import TensorFlow and (i think) pandas or some other lib. Try importing tf first such as this and rebuild:
 

import tensorflow as tf
import random
import numpy as np
import pandas as pd
1

QC upgraded Python, which broke the original algo. 

I fixed those errors:

Lines 15-19:

import tensorflow as tf
import random
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
 

Line 226:        for i in range(self.n_features_per_asset+1, len(next(iter(all_data.items())))-1):

 

but now I am getting error:

 

BacktestingRealTimeHandler.Run(): There was an error in a scheduled event AAPL: EveryDay: AAPL: 0 min after MarketOpen. The error was ValueError : Cannot feed value of shape (0,) for Tensor 'Placeholder:0', which has shape '(?, 255)'

2

Getting the same error above and could not manage to fix it, I had a look on google and stackoverflow and seems to be a common problem easy to solve but I cannot get my head around it, any help is appreciated...

I assume it has to do with this two lines?

        self.x                  = tf.placeholder(tf.float32, shape=[None, self.n_features])
        self.y_                 = tf.placeholder(tf.float32, shape=[None, self.n_classes])

0

Dear Derek, really great code! I am using AI algorithms since more then 15 years in completely other areas (medicine) and I am new here to QC, thus a few questions to make sure:

1. Did I get it write that the model is on a per minute basis, but you train the network only on the rebalance event one time a day? Did you also tried to retrain at every data slice based on the lets say 200-300 last minutes? you currently just use the last 15 open prices for learning or am I wrong?

2. Is it possible to easily adopt your approach to use not only a vector for learning in tensorflow, but e.g. an array of data (e.g. open price, volume, e.g. some indicators)? I never used tensorflow, but we started to this offline for fun with other software and the results looked promising...

3. I know that is is just an easy template to start with tensorflow and increase the possibilities then. But what are the limitations now in QC? As far as I understood, I could not train the models offline because they cannot be stored and accessed them in backtesting of live mode, correct? But doing like this, what are the current limitations if you train in every ondata event? I still think it could be desirable to use more data points (or different additional informations per time) for training? How many points are allowed in history calls in on data events?

4. The second version of the multi asset approach looks also very interesting. Unfortunately, like other people, due to teh change in python, I cannot make it running (considering also having no debugger/breakpoints for python in QC....). Do you have maybe a version which works in the current environment? Would be so much appreciated!

THANKS and very best wishes from Germany!

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed