Share New Research 
I thought I'd post how to save DataFrames to the ObjectStore and restore them since it took me a while to get it working (not obvious).
The only way that the round-trip seems to work with is to have the orientation set to "records":
# Save DataFrame data into object store
json_string = stock_history.to_json(orient='records')
qb.ObjectStore.Save("Last_Stock_History", json_string)
# Read back data from object store
json_string = qb.ObjectStore.ReadString("Last_Stock_History")
stock_history_new = pd.read_json(json_string, orient='records')The main problem here is that the indexes are lost. You will need to reset_indexes and then recover them after:
# Flatten DataFrame (move all indexes into column data)
# Need to do this before converting to json_string
stock_history = my_dataframe.reset_index()
# Once you have loaded the data from the store you can restore the indexes
# Need to know what the original indexes were though
my_dataframe = stock_history.set_index(['symbol','time'])
Gregory Szczeszynski
One update: the date-time column is not converted properly, so the following will clear that up:
# Save DataFrame data into object store tmp = stock_history.reset_index() # Flatten indexes into data columns json_string = tmp.to_json(orient='records', date_unit='ns') qb.ObjectStore.Save("Last_Stock_History", json_string) # Read back data from object store json_string = qb.ObjectStore.ReadString("Last_Stock_History") stock_history_new = pd.read_json(json_string, orient='records') # Restore the "date" columns - in this example only 'time' is a date stock_history_new['time'] = pd.to_datetime(stock_history_new['time']) # Restore indexes stock_history_new.set_index(['symbol', 'time'], inplace=True)Derek Melchin
Hi Gregory,
Thanks for sharing. We also need to recreate the Symbol in the DataFrame index. For completeness:
# Save to ObjectStore qb.ObjectStore.Save("data", history.reset_index().to_json(date_unit='ns')) # Read back data from object store restored_history = pd.read_json(qb.ObjectStore.Read("data")) # Restore the indices restored_history['time'] = pd.to_datetime(restored_history['time']) restored_history['symbol'] = restored_history['symbol'].apply(lambda x: qb.Symbol(x)) restored_history.set_index(['symbol', 'time'], inplace=True)See the attached notebook for reference.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Gregory Szczeszynski
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!