trophy

Long Only, ETF Alpha Streams Competition with $27,500 Prize Pool

Back

How to compare backtest variations - API?

Hi

I have an algo with a number of parameters (which could be coded in) with which I would like to automate backtesting variations.

For example, changing the call delta or days to expiration, stop loss, etc.

I have read a bit about an API on the forums and blog post, but I can't seem to find any documentation because the link from the blog to API is a 404. Was the API deprecated?

Are there recommendations for the best way to programmatically launch and compare parameter/variable performance for this situation, if it is even possible?

Thanks!

Update Backtest







0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Interesting. I've used excel spreadsheet to gather and compare outputs of my algo with various parameters (momentum period, resolution, risk threshold). I'm sure there is better way of doing this, but had to postpone this topic alltogether.

0

What you're describing is called optimization and isnt supported by

QuantConnect quite yet.
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Pawel Kowalke are you just recording the changes & results manually?

 

Jared Broad I understand. Do you have insight as to the API that has been mentioned on the forums? Is it still around?

0

Jean - any optimization, manually or api/otherwise, is not supported for now. It's just not sustainable and creates massive load on the system causing deterioration of service for everyone. We are almost ready with dedicated servers for people to do this type of testing.

Just as a quick survey: If there was an optimization feature available, would you be comfortable covering the computing costs? (i.e. direct pass-through of AWS rates). Running 500 backtests for 1 hour would probably cost ballpark $50+. 

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jean Lepisto I do just that. It would be interesting and useful at some point, to automate that. Just think of possibilites such as using the results of backtests within algorithm! :)

0

Jared Broad I think its fair to cover our own cost. Whether that is an instance paid for monthly similar to the paid live server (ex: paid private backtest server) or covering the costs of the runs.

What I personally would prefer is a predictable rate. So whether it is a flat fee or a calculated estimate once a build has been completed, or something similar so costs don't surprise me and add up.

My preference would be to have a flat monthly rate and be subject to the limits of the instance capacity. So if I can only handle 20 concurrent backtests, or it takes 2x as long then so be it, and I can upgrade/downgrade the instance based on my performance needs at a given time.

That being said, would I pay $50 to start to optimize an algo? Sure, when I'm reasonably confident that I am somewhat close to finalizing. If it runs for 10 hours would I pay $500? Probably not until I had proved it further (paper trading or otherwise)

From my perspective as the algo is developed and backtested that's where the most amount of time, effort and likely server resources are consumed. Especially for us noobs. We have a system that worked somewhere and are developing it while learning code. 

In my case, we have gone through a number of iterations not only to debug but also to try to emulate the strategy performance from visual backtesting tools (ex: optionstack) because these tools aren't completely transparent in how they're processing the strategy, slippage, fills, etc. 

So this means lots of backtest runs for debug and trying to reverse engineer the exact details. I'm sure that means us newbies are using up much more than others. Having a paid instance to do this faster and without affecting others would be very fair in my opion.

Another benefit would be faster processing for longer backtests. A few days or a month is quick, but multiple years can take a lot longer. I find it invaluable to test long term across many market conditions instead of just snapshots of selected timeframes/conditions.

Further, I'd gladly pay for a feature that would allow for import of strategy from a tool like Optionstack vs struggling with the code even with a programmer's help. That to me and many other non-programmers would be gold.

Perhaps you got more response than you asked for, but I appreciate you asking and reading.

 

 

 

0

Thank you Jean for the detailed answer -- appreciate it. We'll do our best to factor these concerns into the design.

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


As a note on this. My strong prefernce would be to have Optimization run using something like spot instances... Rather than dedicated VM time.

0

Even with spot instances Chris it will cost "tens of dollars" per optimization. If people are willing to cover it we can implement this feature but we're not sure of the demand at this time. 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jared Broad "tens of dollars" to verify you have the best performing algo out of the variables you can change sounds like a great deal to me. I don't know why anyone wouldn't want to either confirm they had the best performance for their strategy for that kind of $.

To be fair, I trade options so there's a lot more variables than just stocks for example, and that may be influencing my opinion disproportionately and the additional factors may require more resources than tens of dollars would provide?

0

Hi Jean,

We appreciate your suggestion, and will take it into consideration. Thanks for your support!

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed