Open Source

In this tutorial, for those algorithm developers who opt to work offline, we will explore the different methods to debug a python algorithm.

The first step is to install LEAN and get it running with python algorithms.

To do this install:

• Download LEAN from Github.
• Install Visual Studio.
• Open QuantConnect.Lean.sln in Visual Studio.
• Build the solution by clicking Build Menu -> Build Solution.
• Press F5 to run LEAN.

By default, Lean will run the C# version of BasicTemplateAlgorithm. From here the next step is to run the python version of the BasicTemplateAlgorithm successfully.

To run that algorithm:

• Edit config.json with the following paramenters:

      "algorithm-type-name": "BasicTemplateAlgorithm",
      "algorithm-language": "Python",
      "algorithm-location": "../../../Algorithm.Python/BasicTemplateAlgorithm.py",
  

• Build the solution by clicking Build Menu -> Build Solution
• Press F5 to run

Prerequisites

Python Tools for Visual Studio debug (ptvsd) server python library. It can be easily installed using pip:

pip install ptvsd

Attaching the Debugger

The process that we will use to attach the python debugger requires that we run Lean without debugging and attach the process.

First, we will add the ptvsd library, and the following statements:

import ptvsd
ptvsd.enable_attach()
ptvsd.wait_for_attach()

These statements will hold the algorithm execution until Lean is attached to the python debugger.

The following statement sets the breakpoint:

ptvsd.break_into_debugger()

For example, place it in the OnData() method:

import ptvsd
ptvsd.enable_attach()
print(f'''Python Tool for Visual Studio Debugger {ptvsd.__version__}
Please attach the python debugger:
- In Visual Studio, select Debug > Attach to Process (or press Ctrl+Alt+P) to open the Attach to Process dialog box.
- For Connection type, select Python remote (ptvsd)
- In the Connection target box, select tcp://localhost:5678/ and click "Attach" button''')
ptvsd.wait_for_attach()

class BasicTemplateAlgorithm(QCAlgorithm):
    def Initialize(self):
        self.SetStartDate(2013,10, 7)
        self.SetEndDate(2013,10,11)
        self.SetCash(100000)
        self.AddEquity("SPY", Resolution.Second)
        self.Debug(f"numpy test >>> print numpy.pi: {np.pi}")

    def OnData(self, data):
        ptvsd.break_into_debugger()

        close = data["SPY"].Close
        if not self.Portfolio.Invested:
            self.SetHoldings("SPY", 1)

Finally, attach the debugger to the process:

• Run Lean without debugging by clicking on Debug -> Start Without Debugging
• In Visual Studio, select Debug -> Attach to Process (or press Ctrl+Alt+P) to open the Attach to Process dialog box.
• For Connection type, select Python remote (ptvsd)
• In the Connection target box, select tcp://localhost:5678/ and click "Attach" button

Once you attach the process the code execution will stop after ptvsd.break_into_debugger() statement.

This method uses the built in, cross-platform, command line python debugger pdb.

• Edit config.json with the following paramenters:

      "debugging": true,
      "debugging-method": "CommandLine",
  

• Run Lean without debugging by clicking on Debug -> Start Without Debugging
• Lean will hit a first break outside algorithm code
• Start debugging! Example commands:

      break add ../../../Algorithm.Python/BasicTemplateAlgorithm.py:37
      continue
      print(self.Portfolio.Invested)
  

This method uses the Visual Studio Python Development Tools.

Prerequisites

Visual Studio Python development feature. It can be easily installed at Tools -> Get Tools and Features...

• Edit config.json with the following paramenters:

      "debugging": true,
      "debugging-method": "VisualStudio",
  

• Run Lean without debugging by clicking on Debug -> Start Without Debugging
• Lean will stop and wait for the debugger to attach
• Set breakpoints
• In Visual Studio, select Debug -> Attach to Process (or press Ctrl+Alt+P) to open the Attach to Process dialog box.
• For Connection type, select Default
• For Attach to, select Python
• Search for QuantConnect.Lean.Launcher.exe process and attach

Unfortunately, this debugging scenario entails some limitations.

It is not possible to concurrently debug the python algorithm and Lean (C# code). That means that the debugger will not stop at breakpoints in Lean nor we can step into methods defined in Lean (e.g., SetHoldings).

Method 1 ptvsd will always stop after ptvsd.break_into_debugger() call and it not possible to untoggled it. However, we can use conditional statements to prevent its call and avoid unnecessary breaks. In order to resume the algorithm execution without breakpoints, we need to detach the process (Debug -> Detach All).

In this tutorial, we have learned how to debug python algorithms in Visual Studio. It is not quite as smooth as native python yet, but knowing how to debug python algorithms in LEAN can give you a new level of understanding of how they work and allow for easier debugging of your strategy.