Defining Data Sources

Trading Data

Introduction

This page explains how to import alternative data from CSV format into the OnData method. This type of data is meant to inform trading decisions on an existing universe of securities.

Data Format

The following snippet shows an example CSV file:

1997-01-01,905.2,941.4,905.2,939.55,38948210,978.21
1997-01-02,941.95,944,925.05,927.05,49118380,1150.42
1997-01-03,924.3,932.6,919.55,931.65,35263845,866.74
...
2014-07-24,7796.25,7835.65,7771.65,7830.6,117608370,6271.45
2014-07-25,7828.2,7840.95,7748.6,7790.45,153936037,7827.61
2014-07-28,7792.9,7799.9,7722.65,7748.7,116534670,6107.78

The data in the file must be in chronological order.

Define Data Type

Follow these steps to define the data source class:

  1. Open the Lean.DataSource.<vendorNameDatasetName>/<vendorNameDatasetName>.cs file.
  2. Follow these steps to define the properties of your dataset:
    1. Duplicate lines 32-36 for as many properties as there are in your dataset.
    2. Rename the SomeCustomProperty properties to the names of your dataset properties (for example, Destination).
    3. If your dataset is a streaming dataset like the Benzinga News Feed, change the argument that is passed to the ProtoMember members so that they start at 10 and increment by one for each additional property in your dataset.
    4. If your dataset isn't a streaming dataset, delete the ProtoMember members.
    5. Replace the “Some custom data property” comments with a description of each property in your dataset.
  3. Define the GetSource method to point to the path of your dataset file(s).
  4. Set the file name to the security ticker with config.Symbol.Value. An example output file path is /output/alternative/xyzairline/ticketsales/dal.csv.

  5. Define the Reader method to return instances of your dataset class.
  6. Set Symbol = config.Symbol and set EndTime to the time that the datapoint first became available for consumption.

  7. Define the following methods of your dataset class:

Example

The following table provides some example data source classes for linked and unlinked datasets:

Data Source ClassLink Type
QuiverWallStreetBetsLinked
CBOEUnlinked

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: