Securities
Filtering Data
Introduction
Unfiltered raw data can be faulty for a number of reasons, including invalid data entry. Moreover, high-frequency traders can deploy bait-and-switch strategies by submitting bait orders to deceive other market participants, making raw data noisy and untradeable. To avoid messing up with our trading logic and model training, you can filter out suspicious raw data with a data filter.
Set Models
To set a data filter for a security, call the SetDataFilter
property on the Security
object.
// In Initialize var spy = AddEquity("SPY"); spy.SetDataFilter(new SecurityDataFilter());
# In Initialize spy = AddEquity("SPY") spy.SetDataFilter(SecurityDataFilter())
You can also set the data filter model in a security initializer. If your algorithm has a universe, use the security initializer technique. In order to initialize single security subscriptions with the security initializer, call SetSecurityInitializer
before you create the subscriptions.
// In Initialize SetSecurityInitializer(new MySecurityInitializer(BrokerageModel, new FuncSecuritySeeder(GetLastKnownPrices))); // Outside of the algorithm class class MySecurityInitializer : BrokerageModelSecurityInitializer { public MySecurityInitializer(IBrokerageModel brokerageModel, ISecuritySeeder securitySeeder) : base(brokerageModel, securitySeeder) {} public override void Initialize(Security security) { // First, call the superclass definition // This method sets the reality models of each security using the default reality models of the brokerage model base.Initialize(security); // Next, overwrite some of the reality models security.SetDataFilter(new SecurityDataFilter()); } }
# In Initialize self.SetSecurityInitializer(MySecurityInitializer(self.BrokerageModel, FuncSecuritySeeder(self.GetLastKnownPrices))) # Outside of the algorithm class class MySecurityInitializer(BrokerageModelSecurityInitializer): def __init__(self, brokerage_model: IBrokerageModel, security_seeder: ISecuritySeeder) -> None: super().__init__(brokerage_model, security_seeder) def Initialize(self, security: Security) -> None: # First, call the superclass definition # This method sets the reality models of each security using the default reality models of the brokerage model super().Initialize(security) # Next, overwrite some of the reality models security.SetDataFilter(SecurityDataFilter())
Default Behavior
The following table shows the default data filter for each security type:
Security Type | Default Filter |
---|---|
Equity | EquityDataFilter |
Option | OptionDataFilter |
Forex | ForexDataFilter |
Index | IndexDataFilter |
Cfd | CfdDataFilter |
Others | SecurityDataFilter |
None of the preceding filters filter out any data.
Model Structure
Data filtering models should implement the ISecurityDataFilter
interface. Extensions of the ISecurityDataFilter
interface must implement the Filter
method, which receives Security
and BaseData
objects and then returns a boolean
object that represents if the data point should be filtered out.
Data filtering models must implement a Filter
method, which receives Security
and BaseData
objects and then returns a boolean
object that represents if the data point should be filtered out.
public class MyDataFilter : ISecurityDataFilter { public override bool Filter(Security vehicle, BaseData data) { return true; } }
class MyDataFilter: def Filter(self, vehicle: Security, data: BaseData) -> bool: return True