High volume orders in backtests

In backtesting mode, the default fill model is ImmediateFillModel that does not take volume into account.
If you want to add a check, ideally, you should create o custom fill model:

Thanks a lot! I know this is not a technical question but to make it as close to reality as possible, what should a custom filler do? obviously volume is a factor. Do we have bid/ask size anywhere in our objects? because I think that is definitely an important factor. A wait period before the order is fulfilled is also important because in a serial high frequency trade a second order (sell) will not be placed until the first order (buy) is complete. I also think in a more complex system, we could somehow simulate large orders taking the lowest bids/asks, then going to the next best bid/ask on the next time bar and continue until the order is completely done.

Any input on this? or other thoughts?

I agree that is the ideal Patrick; but it requires very expensive and difficult to get data. We currently provide Trade data only (not quotes like you're asking about). With trade data you can build approximations which get close. No model will ever be perfect.

You should split your orders up into small share lots over time. In professional trading they never/rarely issue a $100M order, its generally split up over a longer period. If its a high volume asset you can do this after. If its a penny stock it might take weeks to fill completely. 

So for your custom fill model; take a look at the behavior of your asset, see how liquid it is and what volumes it can likely sustain; then return hundreds of partial fills events over the couse of a few minutes trading. We provide an example of this here:

Awesome! I will try to do some coding based on your sample model. My $100m example was just an exaggeration to make the point. I'm (back)testing on a $50k cash but I noticed that most of the symbols that I was testing with had fewer volumes than what was required for a $50k order in the Second resolution. It gets worse as time goes by and the cash grows. I was wondering how HFT algos can work with multi million dollars if there aren't enough bids/asks in that small slice of time. If the algo waits for longer periods then the price will change - and most probably not in favor of the algo!

One method that I thought of to resolve the issue is that instead of trading one asset, we could trade with different assets. But then the algo will get exponentially more complex because it probably needs a multi thread process that constantly monitors the indicators on different assets and triggers the orders independently.

I would appreciate if you share your thoughts on how to solve this new problem :)

It does make it more complicated! But we handle most of the mess for you behind the scenes. If you do some common analysis on multiple symbols you should move all of that into a separate class, then instantiate a class per symbol. This way your logic is written just once with minimal mess. 

LEAN was built to manage a portfolio of assets so most of the time you don't need to do anything special. Just use good coding principles and it will work out (DRY code; do not repeat yourself) etc. 

Ex Algorithm 

->> 10x Analysis Class
->> Update Analysis Class
->> Get results, orders to execute
->> Execution Manager ->> Break up orders and fill at the best times.

Jared,

I have to admit right now my code is not in a good shape because I'm trying to make it work first and then do the clean up and create classes, apply SoC, etc.

My next questions

1. if I set the order to use Synchronous, in runtime will that make that line of code wait until the order is filled?
2. can we set a timeout to cancel an order if it takes too long?
3. with IB do you pay a fee for cancelling an order?

Thanks!

A synchronous will make the line of code wait until the order is filled.
However, after 5 seconds it will be cancelled (timeout). 
You can find information about fees for cancelling/modifying orders here.

To elaborate on cancellation fee: As far as I could tell in IB paper trading, you will pay the order fee minimum ($1) but no additional fee will be added for cancelling it or updating it.

In general, QC data is fine for trading highly liquid assets, e.g. SPY. In that case, you can be sure that:

• A) Last price is very close to bid/ask.
• B) Bid/ask spread and thus slippage is fairly predictable.

On the other hand, if you try trading a thinly traded asset, e.g. low volume ETF, it's very easy to accidentally write bad algorithms. Beware of ImmediateFillModel in particular, some issues for thinly traded assets:

• A trade triggered by the last trade's price (but not necessarily immediately on that trade) will be filled at last trade's price, which is an issue if the last trade was a long time ago since the bid/ask spread might have moved since then! Effectively, your algo might be picking great value trades in hindsight in this manner...
• Fill price is calculated as e.g. for buy: fill.FillPrice = Math.Min(prices.High, order.LimitPrice); This fails to model a situation which will often happen where a limit order gets filled at a bad time as price moves through it, resulting in immediate slippage. On a new trade much later after the previous one, we get a new bar with new high/low in a thinly traded asset, meaning limit order gets much better prices than in reality with this fill model.
• Volume is unaccounted for. A quick to implement compromise for small orders is to ignore trades below some threshold, but requiring trades to be of same size as a large order is too draconian.

I've probably wasted more time due to my own buggy fill/slippage models than anything else when trying to create intraday algos. Even algos that trade once a day are in severe danger of being optimistic for assets that aren't the most liquid.

TL;DR: The only way you can reliably tell if a frequently trading algo is actually going to work is to run it on broker paper account ASAP.

Perhaps I'm misunderstanding but I thought our limits fills are pesimistic not optimistic.. "here a limit order gets filled at a bad time as price moves through it, resulting in immediate slippage" - do you mind explaining this a little?

Limit buy order will never fill at a price higher than your limit price (otherwise whats the point :) right!); How is it possible for you to get immediate slippage? If the price is below your limit -- great, a better fill; if its above your limit, it won't be filled.

Hi Jared. Let's discuss the buy case ony. Of course order will never fill above the limit price. The problem is your passive order will be filled at the limit price, at a point where the market really wants the price to be lower than your limit. So what happens in reality is your order fills at the limit price, whereas in the backtest simulation you may get a better fill at a lower price because the next bar happens to start with its high point below the limit.

I should add I observed this behavior in IB paper trading where typically your limit orders will be at a loss immediately after being filled. HFT traders such as market makers (but also your broker) love your limit orders since they can fill them at the worst possible time for you.

I would say the expression fill.FillPrice = Math.Min(prices.High, order.LimitPrice); is only valid for the current bar (in a liquid asset), fill.FillPrice = Math.Max(prices.High, order.LimitPrice); is a more accurate expression when your limit order is staying passive without being updated... And yeah, we're talking intraday bars here (daily bars work more like ImmediateFillModel would suggest).

Oh, and note that fill.FillPrice = Math.Max(prices.High, order.LimitPrice); simplies to fill.FillPrice = order.LimitPrice; since order will never fill above limit, I chose to invert to Max to highlight the symmetry.

Edit: Please give me somwhat longer time to edit posts. ;)

In the other thread I was referring to this discussion. So now I know that although LEAN uses a pesimistic approach to fill the limit orders but when you please an order at the market price, it fills the order at the close price which is not always pesimistic but in fact my two models (one with market price orders and another with limit orders) prove that even for a very liquid asset this is rather optimistic!