Back

Quantopian Live Shut Down

Hi,

I am new to the community and I am about to be followed by thousands of people. You see Quantopin just announced with only 30 days warning that they are killing off their entire live trading feature.

https://www.quantopian.com/posts/phasing-out-brokerage-integrations

This means all of active live traders will be flocking anywhere and everywhere to migrate their trading systems. I am already noticing a problem trying to migrate to Quant Connect, as the ram usage seems to me a major factor with using any non trivially sized universe. On quantopian my live algos universe started with 1500 or 3000 every morning before filtering down.

Does anyone have any suggestions or relevant experiences for people like me whoa are trying to migrate over from Quantopian with minute level, large universe algos who does not want to see my costs go from $0.0 on quantopian to well over $100 per month on Quant Connect once I am done adding on ram? This is at least good news for QC and other competing services as they are about to get a great many desperate, paying customers...

Update Backtest






The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.



0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


How is the disparity between the 512 mb live trading server and the ram limit in backtesting? Do they have different ram usages or is an institutional plan required for most any algos with medium-large universes to be run live?

0

Also, the data sets available on quantopian are very useful - and not available on QuantConnect...

Does quantconnect have plans to expand available services?

0

My initial experiences live trading with Quantconnect nearly a year ago honestly wasn't great. The live trading connection for IB Paper glitched for the first week of my subscription and while out of market hours worked on the backtester, the order type could not be submitted on live trading unless I paid for QC to hire a contractor and implement the feature. Fortunately the owner, Jared, is very responsive and active.

With so many people leaving Quantopian, this is a really great opportunity for Quantconnect to step up.

 

1

Welcome all to QC!

Derek Tishler  -- the limits are guides only. Your universe can be as you like but if you run out of RAM the system will shut down your algorithm. How much ram you use will depend what you're doing. Are you seeing RAM issues in your backtests? The base line subscription is only $20 and gives you 4GB ram -- basically a 100% mark up on a Google/AWS for providing a live data stream + Equinix hosted algorithm with 0.1ms latency.

Jacob M Shrum -- paid datasets can be imported by coding to the individual company API's. You would need to use the Custom data section to learn more about this.

Sofyan Saputra -- sorry you had a glitch. Its a hard problem to solve but we're in a "period of stability" now. About 80% of the instability comes from undocumented IB API changes. Re: Hiring a contractor -- yes you need to be a programmer to use QC.

 

4

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


@Original topic: Hah I would expect something like this April 1st, back to oligopoly practices it seems.

0

As a guesture to welcome all Quantopian users the new baseline free tier is 8GB Ram allocation! Enjoy!

13

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Does QuantConnect ever plan on selling "data packages" or something similar? A sentiment package would be nice (psychSignal, Sentdex, etc.) I'd imagine QuantConnect could lower the price of these data sets (economies of scale) and profit off of them.

1

Hi Jared. Another Quantopian user here :) Just signed up to look around. Thanks for the nice gesture. I'm sure there's a lot of information already available in the forums, but given the short amount of time we all have to move things over, perhaps you can create a quick post to help Quantopian newcomers with the switch. It's hard for us to get to know QuantConnect well enough in a short period of time, and I'm sure you've already done your competitive analysis on them. So it would be easier for you to do a cheat sheet of what to look for when moving over.

For me specifically, I'm definitely looking to live trade with IB. I'm curious as to whether 512 MB for live trading is realistically sufficient. Also, would be nice to have pricing avaliable upfront for, say 1GB and 2GB before going into a custom quote scenario.

Much appreciated.

0

6

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Welcome Sari Louis! 512mb is sufficient depending on what you're doing. You could manage a few hundred securities easily assuming you're not building massive vectors of data.

QP users will want to check out how we do universe selection. Its slightly different to Quantopian. The best repo of python specific examples is here:

https://github.com/QuantConnect/Lean/tree/master/Algorithm.Python

 

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hello everyone,
If you look around at this forum, you will see that we have had a lot of recent questions about python algorithms in QuantConnect. We have been able to answer all those questions and help people bring their algorithm from other platforms. On the other hand, we are working hard on our documentation, especially the tutorials. Please check them out.
In order to help Quantopian users, I will give my best to learn its API and make a post to show what changes you need to make.

We are really excited about this. Our community will be much stronger with you.

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Hey, in the same boat here.  Spent quite a lot of time the past few months learning Quantopian, researching and experimenting... Now trying to figure out how to salvage that investment of time and make use of the alpha sources I've discovered (that unfortunately don't have $10m hedge-fund capacity).  I would love a Quantopian to QuantConnect migration walk-through from anybody who has insight.  I've found comparison overviews before, but 2 or 3 years old, they might not be acurate any more.

I tried to backtest the sample python algorithm that just buy-and-hold SPY and hit the memory limit and died.  :/  I assume that was a fluke?  I ran it again and it completed.  Is there a rule of thumb how many securities in your universe per MB ram to estimate capacity?

I assume that just as with Quantopian our algorithms are secret and safe from prying eyes?

One difference is Quantopian supported Robinhood brokerage integration.  Any intention to support a free brokerage such as Robinhood or Matador?

Quantopian has a research platform.  Anything similar here?  I saw something called Jupyter in the docs, but it seemed the documentation was missing.

Universe selection capabilities look good -- nice to see all the Morningstar data.  Comparible to Quantopian's pipeline.

I'm not married to Python.  I'm assuming C# has non-insignificant speed advantages?  Enough to switch over?  Python is so sloooooow.

My PC has 64gb of ram, 6 cores, yadayadayada. LEAN can be run locally, but I'll have to pay for all the data I need? From the figures I saw, that quickly becomes prohibitively expensive.  Bandwidth must be astonomical as well?  So running locally, even just for live-trading (not back-testing) is probably not viable or faster? or what? (assuming a course universe of ~500 somewhat liquid but not necessarily S&P500 securities)

0

Quantopian Migrant here. I'd be interested to know the increase in accounts you all experienced today? lol.

1

I came here from Quantopian. Does this site integrate with Robinhood?

0

It looks like the Morningstar data on Quantconnect has a few differences from the data at Quantopian, a few metrics are available on one site and not the other but most of the fundamental data is available on both sites. Also wondering if anyone's using fundamental data from prior quarters in Quantconnect algorithms.

0

Hi,

I am also from QuantOpian. I've registered here more than one year ago. Since I am a Python prgrammer and at that time the Python platform here seems not so strong as that by QuantOpian, I moved to QuantOpian.

Now the QuantOpian is going to shut down and I come back to look for an alternative. I am not sure if QuantConnect has improved its Python enviroment a little bit? :-/

I have to following questions:

1.

I have several linked accounts by IB and on each account I have a live algo running. I am not sure if I can do the same here by QuantConnect? How about the costs?

2.

By live trade by IB, I am not sure if I can get the realtime quotes? Or the 15m-delay ones?

3.

In one of my algos I access the CSV-files from CBOE. I am not sure if I can do the same here?

Cheers

 

0

Derek Tishler - Live requires a fracation of the RAM, in backtesting we spin up multiple threads and cache a few thousand bars of data.

 - 

  • Yes your IP is your own; only a few internal team members have algorithm access; when you ask for support we assign a named team member access to your account to help.
  • We don't have Robinhood integration but once they publish their API documentation we'll add it.
  • Research - yes; it should be released today :)
  • C# is roughly 2-3x faster for backtesting in QC. We get about 10-15k points/sec with python. We are mostly running at the theoretical limits though and running locally won't help. Its bottle necked by the single thread speed (synchronizing all the requested data).
  • If running locally you'll need to procure your own data; our data vendor agreements prohibit downloading data (of course). 
  • Running locally for live trading is possible and easy as we open source local data feed connections. Its just expensive and annoying to maintain.

Andrew Kunkel - We don't yet but will do when they release the API.

Thomas Chang - 

  • Yes you can run multiple parallel algorithms. Its $10/mo for a server at the moment. If we get IB to sponsor the servers they can be made free.
  • You can select to use our data (default) or IB's datafeed which is whatever you're paying them for.
  • Yes you can download external data to your algorithm in live. In backtesting I'd recommend using Custom data instead as it would be too slow.

 

 

 

4

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Jared Broad thank you for expanding on the ram usage, both literally and figuratively. I look forward to testing my algos out on the platform.

You mentioned the performance benefit of c# in backtesting. Is there any major benefit(aside from current platform stability as python is new) to using c# over python in live trading? I am reminded of Ernest Chan's books on algoriithnic trading where he described his operation of prototyping in a scripting language, such as matlab(or python in our case), and then using a programming language like c# for live trading. Chan also mentioned colocation and execution speed. Does that play into language selection for different types of algos on your platform, or are things like IB's execution speeds more limiting than the cloud network/location and code performance? 

0

I'd recommend selecting the language you're most comfortable in. Python is a first class citizen in QC with half the team dedicated to making it better and expanding on the documentation, tutorials, feature set etc (4 people -- Alex, Xiang, Xiaowei & Jing).

The LEAN engine is in C# as we can do the heavy data work quickly and maintain a single kernal across multiple languages. The bridge to python is done with PythonNet.

Realistically IB's execution time is far greater than any language difference. The only real speed difference you'll see is in backtesting where C# is 2-3x faster -- this might only matter for options/futures algorithms.

Our backtesting is done on dedicated hardware -- overclocked Intel i7-7700K OC machines running at 4.7-5GHz with NVme disks. Optimizations to the C# stack welcome to make the python go faster! 

1

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Quantopian really shot themselves in the foot on this one. I think cutting brokerage integration was a huge mistake. I'll definitely be using Quantconnect more and Quantopian less.

1

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed