Jared,
Now I'm getting the following error on an algo that was executing Back Tests just fine about 30 minutes ago:
"Unhandled exception breaking past controls and causing collapse of algorithm node. This is likely a memory leak of an external dependency or the underlying OS terminating the LEAN engine."
https://github.com/QuantConnect/Lean/blob/master/Launcher/Program.cs
Here is the code that I found that outputs the message (lines 103-115):
// if the job version doesn't match this instance version then we can't process it
// we also don't want to reprocess redelivered jobs
if (VersionHelper.IsNotEqualVersion(job.Version) || job.Redelivered)
{
Log.Error("Engine.Run(): Job Version: " + job.Version + " Deployed Version: " + Globals.Version + " Redelivered: " + job.Redelivered);
//Tiny chance there was an uncontrolled collapse of a server, resulting in an old user task circulating.
//In this event kill the old algorithm and leave a message so the user can later review.
leanEngineSystemHandlers.Api.SetAlgorithmStatus(job.AlgorithmId, AlgorithmStatus.RuntimeError, _collapseMessage);
leanEngineSystemHandlers.Notify.SetAuthentication(job);
leanEngineSystemHandlers.Notify.Send(new RuntimeErrorPacket(job.AlgorithmId, _collapseMessage));
leanEngineSystemHandlers.JobQueue.AcknowledgeJob(job);
return;
}