Hello All!

After a mammoth 11 months of engineering, we've finished the deployment of L1 data to production. It has been in backtesting for approximately 7 months, but we encountered some pretty difficult engineering challenges with getting this through to live-trading for the community at this scale.

L1 equity data is about 10x the trade-only live feed; multiplied by the sheer number of community strategies we hit bandwidth and physical limitations of hardware we were operating. This required new racks of equipment and physically upgrading switches and routers to handle the new traffic. We had some brand new engineering leaps on the distribution of the data. Fun parts of the project included: 

  • Upgrading the live racks to 10G fiber internet connections.
  • Hitting the limits of RAM bandwidth and having to optimize the code to shave off the milliseconds.
  • Tracking and setting the perfect exchange opening-auction market ticks to all our aggregated bars.

This upgrade brings:

  • Instant universe selection on deploying your algorithm. You no longer need to wait for the next day to do a universe selection. This is a change of behavior!
  • Spread in live trading. New alphas and paper trading fill will be filled on the quote spread just in backtesting.
  • New live servers; more modern CPUs, RAM, and more efficient live code giving meaning you get more of your allocated node capacity.  
  • New crypto 6+ character ticker changes. Crypto users will need to swap a few tickers in their backtesting and live trading, effective immediately. (ALO -> ALGO, NIO -> ANIO, BSV -> BCHSV, DSH -> DASH, IOT -> IOTA, LIK -> LINK, LOM -> LOOM, MNA -> MANA, GOT -> PKGO, QTM -> QTUM, UST -> USDT, YYW -> YOYOW)

One thing to note -- previously even if you subscribed to minute data, your algorithm charts would update every second. We had to drop this functionality for efficiency but now your chart will update at least every minute even with daily data for your algorithms. We're fairly confident the technology is stable as it has been deployed for more than 2 months to hundreds of alphas working out edge cases and issues. If you have issues please let us know via a support ticket. We're watching this very closely over the next few days. 

Best

Team @ QuantConnect