Back

Backtests are getting stuck after some time

Hi,

Please, help! My algo is getting stuck after running for some time.

Here is the simplified code that causes this:

import itertools

import pandas as pd

STOCKS = ['A', 'AABA', 'AAPL', 'ABT', 'ALGN', 'AMAT', 'AMZN', 'AVGO', 'AVY', 'BA',
'BAC', 'BAX', 'BDX', 'BHGE', 'BKNG', 'C', 'CA', 'CAT', 'CBS', 'CLX',
'CREE', 'CTSH', 'CVX', 'DE', 'DOV', 'DWDP', 'DXC', 'DXCM', 'ELY', 'EMN',
'ERIC', 'ESRX', 'ETN', 'EW', 'GE', 'GM', 'GRMN', 'HAL', 'HIG', 'HMC',
'HON', 'HPQ', 'HRS', 'IBM', 'INTC', 'IP', 'ITW', 'JNJ', 'JPM', 'KMB',
'KO', 'LMT', 'MA', 'MAT', 'MDLZ', 'MDT', 'MET', 'MMM', 'MON', 'MSFT',
'MSI', 'MU', 'NCR', 'NKE', 'NOC', 'ORCL', 'PFE', 'PG', 'PGR', 'PRU',
'PX', 'QCOM', 'ROK', 'RTN', 'SEE', 'SLB', 'SNE', 'SON', 'STX', 'SYK',
'T', 'TEL', 'TER', 'TEVA', 'TGT', 'TXN', 'TXT', 'UNCH', 'UNCH', 'UNH',
'UTX', 'V', 'VRX', 'VZ', 'WHR', 'WRK', 'WY', 'XLNX', 'XOM', 'XRX']

class TestAlgorithm(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2003, 7, 1)
self.SetEndDate(2018, 4, 1)

self.symbols = [self.AddEquity(stock, Resolution.Daily).Symbol \
for stock in STOCKS]

self.Schedule.On(self.DateRules.MonthStart("AAPL"),
self.TimeRules.BeforeMarketClose("AAPL", 60),
Action(self.justdoit))

self.SetRunMode(RunMode.Series)
self.SetWarmUp(300, Resolution.Daily)

def justdoit(self):
self.Debug('Time: %s' % self.Time)
for period1, period2 in itertools.product([21, 42, 63, 84], [168, 189, 210]):
hist1 = self.History(STOCKS, period1, Resolution.Daily).close.unstack(level=0)
hist2 = self.History(STOCKS, period2, Resolution.Daily).close.unstack(level=0)

 

As you can see the code calls History and converts the output to the dataframe. There is nothing special in it except that it's done in a loop for combinations of periods. It works just fine for some time and then it simply stucks without producing any errors. Usually it happens after 16 iterations. It's 100% reproducible.

Any ideas what could be the reason of this?

 

Regards,

Ed

Update Backtest








The code hangs at the same time even if close.unstack(level=0) is not called! So, simply calling History a lot of times makes the backtest to stuck. 

Here is the console log:

```

1 | 16:54:20: Welcome to QuantConnect Algorithm Development Terminal.

2 | 16:54:29: Python autocomplete loaded.

3 | 16:54:38: Build Request Successful for Project ID: 1318844 CompileID: 29dd06f5419aefabedee863f8ed620c3-2febd83774eb37f3c3c2002486ccc913 Lean Version: 2.4.0.1.3549

4 | 16:55:00:Build Request Successful for Project ID: 1318844 CompileID: 0ec197763f6d963b618f78a8c2dc7e97-2dbf2a8dd4af96a077b03d8ef857aa7c Lean Version: 2.4.0.1.3549

5 | 16:55:07:Build Request Successful for Project ID: 1318844 CompileID: f4a419a3eca4ebc9fd5ed10144d4963c-c2f3d878765f4d339ddd4c1768abc1d4 Lean Version: 2.4.0.1.3549

6 | 16:55:07: Backtesting Project ID: 1318844

7 | 16:55:10:Successfully sent backtest request for 'Adaptable Asparagus Buffalo', (Compile Id: f4a419a3eca4ebc9fd5ed10144d4963c-c2f3d878765f4d339ddd4c1768abc1d4)

8 | 16:55:24:Launching analysis for 06bdcbf4e654f6642b607c6b78b40a3e with LEAN Engine v2.4.0.1.3549

9 | 16:55:25:Algorithm-Id: 06bdcbf4e654f6642b607c6b78b40a3e successfully sent to cloud.

10 | 16:55:25:Algorithm warming up...

11 | 16:55:27:Algorithm finished warming up.

12 | 16:56:16:Backtest deployed in 65.445 seconds

13 | 16:56:16:Time: 2003-07-01 15:00:00

14 | 16:56:47:Launching analysis for 06bdcbf4e654f6642b607c6b78b40a3e with LEAN Engine v2.4.0.1.3549

15 | 16:56:48:Algorithm warming up...

16 | 16:56:50:Algorithm finished warming up.

17 | 16:57:37:Time: 2003-07-01 15:00:00

18 | 16:58:23:Time: 2003-08-01 15:00:00

19 | 16:59:10:Time: 2003-09-02 15:00:00

20 | 16:59:57:Time: 2003-10-01 15:00:00

21 | 17:00:47:Time: 2003-11-03 15:00:00

22 | 17:01:33:Time: 2003-12-01 15:00:00

23 | 17:02:23:Time: 2004-01-02 15:00:00

24 | 17:03:09:Time: 2004-02-02 15:00:00

25 | 17:04:00:Time: 2004-03-01 15:00:00

26 | 17:04:46:Time: 2004-04-01 15:00:00

27 | 17:05:36:Time: 2004-05-03 15:00:00

28 | 17:06:27:Time: 2004-06-01 15:00:00

29 | 17:07:14:Time: 2004-07-01 15:00:00

30 | 17:08:07:Time: 2004-08-02 15:00:00

31 | 17:08:52:Time: 2004-09-01 15:00:00

32 | 17:09:44:Time: 2004-10-01 15:00:00

39 | 17:45:21: Runtime Error (Open Stacktrace)

40 | 17:45:37:Launching analysis for 06bdcbf4e654f6642b607c6b78b40a3e with LEAN Engine v2.4.0.1.3549

41 | 17:45:38:Algorithm warming up...

42 | 17:45:39:Algorithm Id:(06bdcbf4e654f6642b607c6b78b40a3e) completed in 2.12 seconds at 22k data points per second. Processing total of 47,469 data points.

```

Notice, that it produced runtime error after 35 minutes of silence, but it didn't even show stacktrace!

0

history examples:

https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/HistoryAndWarmupRegressionAlgorithm.pyhttps://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/HistoryAlgorithm.py

searching self.History here will show you some examples

https://github.com/QuantConnect/Lean/tree/master/Algorithm.Python
0

Thanks for the examples. Does this mean I'm using History incorrectly? Even if I do, why does the algo stuck?

0

yep

ha that is something you have to ask the staff/support.

but it looks like you are requesting tooooo much data.

check the examples!

0

Attaching the algo. Still don't see what I'm doing wrong.

Can anybody help?

0


> but it looks like you are requesting tooooo much data.

If it's the case I'd expect the algo to break during the first iteration. However, it worked for more than 15 iterations and only after that it's stuck.

It looks more like a memory leak in the engine to me.

0

Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed