Back

Serious Testing

OK, so we all know this problem: You re-run your backtest, and because of a miniscule change in conditions (minor algorithm difference, or randomness) you get an entirely different outcome. In algorithms where decisions are impacted by current portfolio holdings, it even worse as then you will get a domino effect from the first difference. Chaos theory in action I suppose.

The way I dealt with this outside QC in a crude manner is re-running tests a lot of times. QC doesn't currently support that functionality aside from manually doing so, or if you somehow hack it into your algorithm via e.g. simulating trading inside the simulation (messy and defeats the purpose of QC). It would also be naturally more CPU-intensive (but hey, I am willing to pay more for something that solves my problems out of the box and saves me time).

I think QC would also benefit from an optional high level buy/sell signal framework that allows signals to be tested in isolation. Again, because testing signals without impact stateful portfolio holdings is important to avoid the domino effect. I'm not sure how far one can go in customizing the QC test output today (I think this is basically possible already with some work), but again convenience applies.

As for parameter optimzation (e.g. walkforward), and machine learning, it's necessary to hack around system limitations in a rather awkward manner. I'm curious what the future holds here for QC, there are definitely great possibilities to do more things in the cloud than on my local computer's limited CPU.

Overall I'm wondering what other people's thoughts are about these things, because I regard them as my biggest problems with working in QC right now, some other purely technical issues aside.

Update Backtest








This feature we're broadly calling optimization - its a work in progress and we've very exited by it :) For now we only have the parameter specification in the left menu, but we'll have optimization soon!

2

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Cheers!
0

How can I show an intra-day chart?  Zoom only allows day/week/month?

I need to be able to zoom down to the minute level. 

0

I'm a Noob here.   I have a ton of questions so this is going to be scattered.

I'm successfully coding using quantconnect.lean-master in Visual Studio.  I need IntelliSense and auto code completion.

I want to become a paying customer...  I can see that Lean-master is an excellent codebase.

I wish QCStudioPlugin wasn't deprecated.  It looks like exactly what I need.

I *totally* understand that you can't give away the historical data for free due to licensing agreement.

However, I need a method to draw my own intra-day charts.  That's the only way I would be able to become a full customer.

I have other questions but this is the main problem for me at the moment.

 

1

Hi Keith; in the future please try and start a new thread in the forums instead of commenting on an existing thread:

  • We give away alot of data at quantconnect.com/data
  • You can use the custom charting code to draw charts you need for specific times. This will let you get limited periods of high resolution data for your debugging. Check out the documentation under charting for this.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Great to hear more stuff is coming!

QC is already today a tremendous time saver overall, hope you don't mind me venting about the less good parts hehe. :)

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed