Importing Data
Bulk Downloads
Recommended Use Cases
The batch import technique is outside of the LEAN's awareness or control, so it can't enforce good practices. However, the batch import technique is good for the loading the following datasets:
- Trained AI Models
- Well-defined historical price datasets
- Parameters and setting imports such as
Symbol
lists
Download Files
The Download
method downloads the content served from a local file or URL and then returns it as a string.
Basic Usage
var file = Download("<filePathOrURL>");
file = self.Download("<filePathOrURL>") # If your file is in CSV format, convert it to a DataFrame with the `read_csv` method. from io import StringIO import pandas as pd df = pd.read_csv(StringIO(file)) # If your file is in JSON format, parse it with the `loads` method. import json data = json.loads(file) # If your file is in XML format, parse it with the `fromstring` method. import xml.etree.ElementTree as ET root = ET.fromstring(file)
Download Method Arguments
The Download
method can accept header settings, a username, and a password for authentication.
Argument | Data Type | Description | Default Value |
---|---|---|---|
address | string str | A string containing the URI to download | |
headers |
IEnumerable<KeyValuePair<string, string>>
Dict[str,str] | Defines header values to add to the request | Enumerable.Empty<KeyValuePair<string, string>>() dict() |
userName | string str | The user name associated with the credentials | null None |
password | string str | The password for the user name associated with the credentials | null None |
Download Request Headers
var headers = new Dictionary{ { "1", "1" } }; Download(address, headers); Download(address, headers, userName, password);
header = { "1": "1" } self.Download(address, headers) self.Download(address, headers, user_name, password)
Transport Binary Data
Follow these steps to transport binary files:
- Add the following imports to your program:
- Serialize your object.
- Save the string representation of your object into the Object Store or one of the supported external sources.
- Load the string representation of your object into your trading algorithm.
- Restore the object.
import pickle import base64
pickle_bytes = pickle.dumps(my_object) base64_str = base64.b64encode(pickle_bytes).decode('ascii')
base64_bytes = base64_str.encode('ascii') model = base64.b64decode(base64_bytes) restored_model = pickle.loads(model)