Share New Research 
Hi everyone,
I'm using Lean to run my algorithm on my computer.
The algorithm's core is written in F#, and the shell that Lean interacts with is written in C#.
When a one-minute candle is created, the C# code calls the core algorithm to get signals. Running the core algorithm 1,440 times (one full day of minute candles) takes roughly one second.
Everything runs smoothly at first, but after a while, performance drops drastically. The same one-day computation that initially takes one second can suddenly take up to 10 minutes. Then, after about 50 days, performance returns to normal, only for the cycle to repeat.
I've tested my algorithm in every way I could think of, but its performance seems consistent. I also monitored my computer's CPU and memory usage—one of my CPU cores is fully utilized, and 33% of my memory is in use. It seems like something inside the Lean code is periodically affecting execution speed.
Any clues on how to fix this issue would be greatly appreciated!
Here’s part of my log—CPU usage is always over 100%, whether the algorithm is running smoothly or when the issue occurs.(I log a message every time a daily candle is consolidated from the minutely candles)
20250308 09:27:27.914 TRACE:: Daily bar created: 4/10/2019 12:00:00AM
20250308 09:27:28.756 TRACE:: Daily bar created: 4/11/2019 12:00:00AM
20250308 09:27:29.334 TRACE:: Daily bar created: 4/12/2019 12:00:00AM
20250308 09:27:29.921 TRACE:: Daily bar created: 4/13/2019 12:00:00AM
20250308 09:27:30.530 TRACE:: Daily bar created: 4/14/2019 12:00:00AM
20250308 09:27:31.140 TRACE:: Daily bar created: 4/15/2019 12:00:00AM
20250308 09:27:31.785 TRACE:: Daily bar created: 4/16/2019 12:00:00AM
20250308 09:27:32.394 TRACE:: Daily bar created: 4/17/2019 12:00:00AM
20250308 09:27:33.332 TRACE:: Daily bar created: 4/18/2019 12:00:00AM
20250308 09:27:34.052 TRACE:: Daily bar created: 4/19/2019 12:00:00AM
20250308 09:27:34.659 TRACE:: Daily bar created: 4/20/2019 12:00:00AM
20250308 09:27:35.275 TRACE:: Daily bar created: 4/21/2019 12:00:00AM
20250308 09:28:01.887 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1424, Sample: 2903, App: 4571, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:28:29.771 TRACE:: Daily bar created: 4/22/2019 12:00:00AM
20250308 09:29:01.896 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1414, Sample: 2348, App: 4572, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:30:02.404 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1393, Sample: 834, App: 4572, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:31:02.412 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1543, Sample: 3359, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:32:02.421 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1463, Sample: 1371, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:33:02.428 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1404, Sample: 625, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:34:02.436 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1391, Sample: 1896, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:35:02.445 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1435, Sample: 1365, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:36:02.452 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1410, Sample: 1815, App: 4573, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:37:02.460 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1417, Sample: 1604, App: 4574, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:38:02.470 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1407, Sample: 2260, App: 4574, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:39:02.478 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1430, Sample: 112, App: 4574, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:40:02.488 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1375, Sample: 2165, App: 4574, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:41:02.495 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1395, Sample: 906, App: 4574, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:42:02.501 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1494, Sample: 1185, App: 4575, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:43:02.510 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1491, Sample: 1852, App: 4575, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:44:02.517 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1425, Sample: 222, App: 4575, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:44:50.771 TRACE:: Daily bar created: 4/23/2019 12:00:00AM
20250308 09:45:02.526 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1429, Sample: 1448, App: 4576, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:46:02.534 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1367, Sample: 1207, App: 4577, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:47:02.540 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1441, Sample: 1286, App: 4577, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:48:02.548 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1460, Sample: 504, App: 4577, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:49:02.556 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1452, Sample: 2031, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:50:02.564 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1497, Sample: 1270, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:51:02.573 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1463, Sample: 1952, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:52:02.582 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1436, Sample: 1395, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:53:02.591 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1411, Sample: 654, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:54:02.599 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1438, Sample: 2052, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:55:02.607 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1425, Sample: 307, App: 4578, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:56:02.615 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1414, Sample: 980, App: 4579, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:56:46.722 TRACE:: Daily bar created: 4/24/2019 12:00:00AM
20250308 09:57:02.623 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1401, Sample: 157, App: 4579, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:58:02.630 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1428, Sample: 1208, App: 4579, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 09:59:02.638 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1404, Sample: 2076, App: 4580, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:00:02.647 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1401, Sample: 982, App: 4580, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:01:02.655 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1450, Sample: 2641, App: 4580, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:02:02.664 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1443, Sample: 1367, App: 4576, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:03:02.673 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1433, Sample: 1936, App: 4576, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:04:02.682 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1436, Sample: 1919, App: 4576, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:05:02.689 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1478, Sample: 2132, App: 4576, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:06:02.699 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1464, Sample: 2565, App: 4577, CurrentTimeStepElapsed: 00:00.000. CPU: 101%
20250308 10:07:02.707 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1399, Sample: 502, App: 4577, CurrentTimeStepElapsed: 00:01.000. CPU: 101%
20250308 10:07:59.064 TRACE:: Daily bar created: 4/25/2019 12:00:00AM
20250308 10:07:59.995 TRACE:: Daily bar created: 4/26/2019 12:00:00AM
20250308 10:08:01.313 TRACE:: Daily bar created: 4/27/2019 12:00:00AM
20250308 10:08:02.312 TRACE:: Daily bar created: 4/28/2019 12:00:00AM
20250308 10:08:02.714 TRACE:: Isolator.ExecuteWithTimeLimit(): Used: 1390, Sample: 201, App: 4577, CurrentTimeStepElapsed: 00:00.000. CPU: 102%
20250308 10:08:03.238 TRACE:: Daily bar created: 4/29/2019 12:00:00AM
20250308 10:08:04.228 TRACE:: Daily bar created: 4/30/2019 12:00:00AM
20250308 10:08:05.183 TRACE:: Daily bar created: 5/1/2019 12:00:00AM
20250308 10:08:06.584 TRACE:: Daily bar created: 5/2/2019 12:00:00AM
20250308 10:08:07.565 TRACE:: Daily bar created: 5/3/2019 12:00:00AM
20250308 10:08:08.573 TRACE:: Daily bar created: 5/4/2019 12:00:00AM
20250308 10:08:09.522 TRACE:: Daily bar created: 5/5/2019 12:00:00AM
20250308 10:08:10.599 TRACE:: Daily bar created: 5/6/2019 12:00:00AM
Mia Alissi
The issue you're experiencing with your Lean algorithm, where performance degrades periodically, may be related to how resources are managed during execution. Here’s a structured approach to diagnose and address this problem:
Profile CPU and Memory Usage:
Manage Resources Effectively:
GC.Collect()method judiciously to clear unused memory during less critical phases of the algorithm.Optimize Lean's Isolator:
Isolatorclass's methodexecute_with_time_limitto ensure that your algorithm does not exceed specified memory or time limits, which can overconsume resources on the host machine.memory_capthat is realistic based on your machine's capacity. Default is 1024MB, but can be adjusted according to your observation and needs.Code Optimization:
Monitor and Adjust Execution:
Testing and Simulations:
By applying these suggestions, you should be able to pinpoint the cause of performance drops, enhance your algorithm's efficiency, and maintain consistent run times over extended periods. If necessary, periodic restarting of the algorithm may serve as a temporary mitigation while deeper diagnostics are performed.
Sources: Getting Started - QuantConnect.com, Algorithm Engine - QuantConnect.com, Opening Range Breakout for Stocks in Play, Getting Started - QuantConnect.com
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Mehdi
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!