Back

ObjectStore: Deleting all saved data in a project?

I have been tinkering with saving data (lists of backtested float values, not machine learning weights) into the ObjectStore. My understanding is that each project created in QC has its own ObjectStore "bin" which allows different backtests within the same project to read/write/delete data in its respective ObjectStore.

Is there any way to delete all the saved key/value pairs in an object store? I think this is referred to as a flush command in similar applications. I could just as easily clone a new project and start with a blank ObjectStore but was curious if this was possible.

Update Backtest







0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


I think I remember seeing some helper methods with the ObjectStore in the source code that can reset everything.

Alternatively, what I do is manage the ObjectStore via Research like:

qb = QuantBook()

import os
from datetime import datetime
def ObjectStoreOverview(storage_limit=50, file_limit=1000):
"""
Some basic ObjectStore information.
"""

keys = [str(j).split(',')[0][1:] for _, j in enumerate(qb.ObjectStore.GetEnumerator())]
sizes = [os.path.getsize(qb.ObjectStore.GetFilePath(key)) for key in keys]

# Last modified times (Not working)
#timestamps = [
# datetime.fromtimestamp(os.path.getmtime(qb.ObjectStore.GetFilePath(key))).strftime("%Y-%m-%d %H:%M:%S")
# for key in keys
#]
remaining = storage_limit - sum(sizes)/1e6
print(f'Remaining storage: {remaining} MB.')
print(f'Remaining file limit: {file_limit-len(keys)}')

# For human readability
converted_sizes = []
for size in sizes:
if size < 1000:
s = f'{size} bytes'
elif size < 1e6:
s = f'{size/1000} KB'
else:
s = f'{size/1e6} MB'
converted_sizes.append(s)

print('--')
print('Key, Size')
for file in zip(keys, converted_sizes,): #timestamps):
print(' '.join(file))
return keys


# Delete everything
keys = ObjectStoreOverview()
for key in keys:
qb.ObjectStore.Delete(key)

 

0

Hi CAPOCAPTIAL,

We can clear the ObjectStore with this method:

def clear_ObjectStore():
keys = [str(j).split(',')[0][1:] for _, j in enumerate(qb.ObjectStore.GetEnumerator())]
for key in keys:
qb.ObjectStore.Delete(key)

Best,
Derek Melchin

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Thank you both for sharing the above ObjectStore methods, very helpful! I was saving every item in a list in a separate string to the ObjectStore but was able to pass the entire list as a json5 which worked much better.

0

Hi Craig,

We can use the Newtonsoft.Json framework to serialize a list of C# objects. For example

var dataPack = new MyObjectStoreDataPack(8);
var dataPack2 = new MyObjectStoreDataPack(10);
var lst = new List<MyObjectStoreDataPack>() { dataPack, dataPack2 };
Log(JsonConvert.SerializeObject(lst));

See the attached research notebook for a demonstration of this.

Best,
Derek Melchin

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed