Machine Learning
Training Models
Introduction
Algorithms usually must process each timeslice within 10 minutes, but the Traintrain method allows you to increase this time to train machine learning models. The length of time you can train depends on your training quotas.
Train Models
To train models immediately, call the Traintrain method and pass in the name of your training method.
Train(MyTrainingMethod);
self.train(self.my_method)
Immediate training is most useful for training your model when you first deploy your strategy to production or when the model's performance begins to degrade.
Schedule Training Sessions
You can schedule model training sessions in a similar way to a Scheduled Event. To schedule a training session do this, pass in a DateRules and TimeRules argument to the Traintrain method.
// Set TrainingMethod to be executed at 8:00 am every Sunday Train(DateRules.Every(DayOfWeek.Sunday), TimeRules.At(8, 0), MyTrainingMethod);
# Set TrainingMethod to be executed at 8:00 am every Sunday self.train(self.date_rules.every(DayOfWeek.SUNDAY), self.time_rules.at(8,0), self.my_training_method)
We recommend you schedule your training sessions for when the market is closed to get the best compute allocation. While the market is open, your CPU is occupied with processing incoming tick data and handling other LEAN events.
Training Quotas
Training resources are allocated with a leaky bucket algorithm where you can use a maximum of n-minutes in a single training session and the number of available minutes refills over time. This design gives you burst allocations when you need them and recharges the allowance to prepare for the next training.
Cloud Quotas
If you execute algorithms in QuantConnect Cloud, see Training Quotas for more information about the training quotas.
Local Quotas
If you execute algorithms locally, the following table shows the default settings for the leaky bucket algorithm:
| Setting | Value |
|---|---|
| Capacity (minutes) | 120 |
| Time interval (minutes) | 1440 |
| Refill amount (minutes per time interval) | Capacity / 7 |
To allow virtually unlimited training for local algorithms, add the following key-value pairs to your Lean / Launcher / config.json file:
"scheduled-event-leaky-bucket-capacity" : 99999999, "scheduled-event-leaky-bucket-time-interval-minutes" : 1, "scheduled-event-leaky-bucket-refill-amount": 999999,
Check Model Readiness
In backtests, the Traintrain method is synchronous, so it blocks your algorithm execution while the model trains. In live trading, the Traintrain method is asynchronous, so ensure your model is trained before you continue the algorithm execution. Training occurs on a separate thread, so set a boolean flag to notify your algorithm of the model state. A semaphore is a thread-safe flag you can use to synchronize program operations across different threads.
class SemaphoreTrainingAlgorithm(QCAlgorithm):
# Model Object
model = None
# Model State Flag
model_is_training = False
def initialize(self) -> None:
self.train(self.my_training_method)
def my_training_method(self) -> None:
self.model_is_training = True
# Perform Work
self.model_is_training = False
def on_data(self, slice: Slice) -> None:
# Do not use model while it is being trained.
if self.model_is_training:
return
# Once training is complete; use the model safely.
result = self.model.predict()
public class SemaphoreTrainingAlgorithm : QCAlgorithm
{
// Model Object
private MachineLearningModel _model;
// Model State Flag
private bool _modelIsTraining;
public override void Initialize()
{
Train(MyTrainingMethod);
}
private void MyTrainingMethod()
{
_modelIsTraining = true;
// Perform Work
_modelIsTraining = false;
}
public override void OnData(Slice slice)
{
// Do not use model while it is being trained.
if (_modelIsTraining)
{
return;
}
// Once training is complete; use the model safely.
var result = _model.Predict();
}
}
You can also see our Videos. You can also get in touch with us via Discord.
Did you find this page helpful?
