Back

Am I #1? Funding, Modeling Issue

Here's a screenshot of my backtest. Is it supposed to say #1? it says I came in at #0.


I don't want to make misleading statements, the actual return might not be as high as the market may not be able to handle transaction sizes as large as those QuantConnect puts through. Does QuantConnect limit those to stay realistic? I have to check and see still. The algorithm does seem to work, will likely turn a profit, and confirms what my own custom backtester seems to have been suggesting.

I am interested in funding and/or selling the algorithm.
Update Backtest








I added some code to limit the amount spent per trade to more realistic values. The results aren't quite as impressive but I can still make the top 1%. Everything is realistic now as best as I know.
0

That looks extremely impressive indeed. I can't help but feel hesitant to trust those results due to the following observations:

- The return is more than 292,000%. I'm not sure which stock(s) you are trading, but individual stocks don't even move that much in total over a period of 2 years (cumulative absolute return). So even if you could predict the movement of stocks with an accuracy of 100%, you still could barely see such results. Of course you could implement a big leverage in your strategy but that would make it all the less realistic.

- The only possibility where such a return would be feasible is if you somehow play the small- and micro-caps. Mind you, that for small- and micro-caps the amount of trading volume that can be absorbed is extremely limited. For some of these stocks, you would become a major holder by buying even $100k worth of shares.

- I once had returns like this and it was 100% due to an implicit look-ahead bias that was accidentally part of my strat. It could be the case that you also accidentally included some sort of look-ahead bias.

I'm curious to find out what's going on here. Also the $65,000,000 in fees is very strange. That would imply an average fee of $59,000 per trade; seems odd.
0

The backtest was run for 2015 and 2016 which is a fair while. If you don't include code to limit the amount spent per trade QuantConnect allows unrealistic entries, so as time goes on the results get more and more unrealistic. In fact the amount you can make from any stock is finite.

Here's a screenshot of the same algorithm with realistic purchases. Run through 2016, starting with $25000.
0

I see daily performance of 5% - 20% and an average of 16 trades per day. I'm not sure how long your holding period is, but if you are trading anything lower than mid cap, the bid-ask spread might also significantly decrease your profits. Also, the stock that you are trading; does the algorithm choose it by itself? Otherwise, if you selected the stock yourself there might be an implicit stock-selection bias.
0

I used two stock selection methods. One, I picked out my favorite long/short pairs of etf's by hand, the one usually goes up while the other goes down so the results of that should be fairly unbiased (e.g. SQQQ/TQQQ). Two, I used Finviz to screen for the most volatile stocks that aren't too low volume, that should give me a pretty even hand of gainers and losers there.

I'ld have thought QuantConnect would have handled the order properly to prevent any bid/ask problems.
0

Using Finviz might introduce a look-ahead bias: you only know that a stock has recently been volatile after the move already took place. Therefore if you select on recent big gainers/losers, you wouldn't be able to replicate your strategy in real life. If you select on the most volatile stocks of all time instead, opposed to some recent time window, you might introduce a lower bias. Nevertheless, both methods will most probably yield stocks that are at the lower end of your volume spectrum (mid cap at least). Simply because stocks with a lower market capitalization or usually more volatile too. Hence volume absorption is still a concern.

QC doesn't use the bid-ask spread to fill your orders. It proxies fills by using the last occurred trade. This is a decent assumption if your trade has approximately the same size as that last trade. If your trade is much larger, however, you will inevitably move the market and therefore get a fill that might be quite a few basis points away from the fill of the last trade. This is called "slippage" and is especially important when trading mid to low-cap stocks.
0

In the 'realistic' version my cap on the amount investable in a trade is 1/10 of the volume*close of the last minute. That should take care of the one problem.

Finviz will only give you the most volatile over the last month or week, I used the month list. I'll check online to see if I can get a longer time period somewhere.

I could move to limit orders and that should help with the bid/ask problem, if I recall the problem correctly from writing my own backtester. QuantConnect should be able to put limit orders through based upon the lows and highs of the TradeBars.
0

I've generally been careful not to overfit.
0

Hi Warren, the LEAN engine provides basic models which use last trade data and assume the order fills without slippage. You can update these to make your backtest more realistic. See an example in the QC University "How Do I Create Custom Transaction Models?"

Our limit orders fill based on the OHLC for the last tradebar.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thanks Jared. Nice site.
0

Warren,

Just a heads-up: be very careful about results like this. JPB makes very good points about possible ways that bias can leak into your algorithm.

On my end, I spent several months coding up a Universe Selection algorithm that got results very much like the ones you're showing. I even put in filters to ensure I was trading highly liquid securities, and put very conservative slippage estimates in.

I ended up cross-checking the algo with another 3rd program and POOF the results vanished. It turned out the indicator I was using was erroneously reading the next day's price rather than the price of the "day of". It has since been fixed, so make sure you're always testing on the latest master branch, rather than a stable tag.

Someone else on this forum also quoted the problem of taking positions in micro or small caps, where low liquidity can cause the bid/ask spread to be quite large; the backtester will always assume that your requested orders will always go through.
0

Thanks Stephen. It's too bad there isn't a way to more rigorously rank the algorithms. Kaggle.com has competitions for coders that pay cash, which is an interesting concept that QuantConnect could consider. The investors would put up the cash for the competition and gain access to the winning algorithms. Controls could be put in place to eliminate cheating. E.g. limited purchase sizes. Rules could also be used, e.g. no external data sources that could bias the competition. The winning algorithms could be inspected by QuantConnect to ensure all rules have been followed. It would be a great service for both programmers and investors. As is it's going to be difficult for investors to determine which algorithms are best.
0

The distribution curve can be used as a very loose guide for backtests. We'll launch actual algorithm competitions soon and rankings will be based on forward trading only.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Even with forward trading, you'll have to close the hole seen here with purchase sizes on lower volume stocks. Either by coding in limits or requiring them by a rule. Limiting stock selection to high volume stocks could also be done, but you'll be bypassing potential profits. This bid/ask issue likely would need to be addressed as well.
0

On a more positive note, the biggest problem I've found so far is exponential growth. With a successful trading algorithm, the amount you have to invest can compound multiple times per day, leading to very rapid exponential growth of your capital. The result is that your bankroll quickly dwarfs the market when you go to try and invest it all. You may well see this problem quite quickly in a competition, even with large volume stocks, as you'll have the most competitive algorithms at the top. Limitations on the amount investable based on volume and price should probably be required in any competition.
0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed