Projects

Data Storage

Introduction

The ObjectStore is a project-specific key-value storage location to save and retrieve data in QuantConnect's cache. Similar to a dictionary or hash table, a key-value store is a storage system that saves and retrieves objects by using keys. A key is a unique string that is associated with a single record in the key-value store and a value is an object being stored. Some common use cases of the ObjectStore include the following:

  • Transporting data between the backtesting environment and the research environment
  • Training machine learning models in the research environment before deploying them to live trading

Supported Types

The ObjectStore has helper methods to store strings, JSON objects, XML objects, and bytes.

The ObjectStore has helper methods to store strings and bytes.

ObjectStore.Save(stringKey, stringValue);
ObjectStore.SaveJson<T>(jsonKey, jsonValue);
ObjectStore.SaveXml<T>(xmlKey, xmlValue);
ObjectStore.SaveBytes(bytesKey, bytesValue);
self.ObjectStore.Save(string_key, string_value)
self.ObjectStore.SaveBytes(bytes_key, bytes_value)

To store an object that is in a different format, you need to encode it to one of the supported data types. For instance, if you train a machine learning model and it is in binary format, encode it into base 64 before saving it.

The ObjectStore also has helper methods to retrieve the stored objects.

var stringValue = ObjectStore.Read(stringKey);
var jsonValue = ObjectStore.SaveJson<T>(jsonKey);
var xmlValue = ObjectStore.SaveXml<T>(xmlKey);
var bytesValue = ObjectStore.SaveBytes(bytesKey);
string_value = self.ObjectStore.Read(string_key)
bytes_value = self.ObjectStore.ReadBytes(bytes_key)

For complete examples of using the ObjectStore, see Object Store.

Storage Sizes

All organizations get 500MB of free storage in the ObjectStore. Organizations on the Team tier or higher can subscribe to more storage space. The following table shows the cost of the supported storage sizes:

Storage Size (GB)Monthly Cost ($)
0.50
110
220
550
10100

Research to Live Considerations

When you deploy a live algorithm, the contents of the ObjectStore are copied to the live trading node. If you change the contents of the ObjectStore from the Research Environment after deployment, it does not affect the ObjectStore stored on the live trading node.

Monitor Usage

The Resources page shows the total storage used in your organization and the storage used by individual projects so that you can easily manage your storage space. To view the page, log in to the Algorithm Lab and then, in the left navigation bar, click Organization > Resources.

Edit Storage Plan

You need storage billing permissions within an organization on the Team, Trading Firm, or Institution tier to edit the size of the organization's ObjectStore.

Follow these steps to edit the amount of storage available in your organization's ObjectStore:

  1. Log in to the Algorithm Lab.
  2. In the left navigation bar, click Organization > Resources.
  3. On the Resources page, scroll down to the Storage Resources section and then select a storage plan.
  4. Click Update.

Delete Storage

To free up storage space, delete the key-value pairs stored in the ObjectStore or delete projects that have data in the ObjectStore.

To delete a key-value pair stored in the ObjectStore, call the Delete method with a key.

ObjectStore.Delete(key);
self.ObjectStore.Delete(key)

To keep your code while also deleting projects, clone the project and then delete the original project.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: