Local research is downloading huge amounts of data

When you run the lean research command, it pulls the necessary Docker image for the research environment. This image contains all the necessary dependencies and packages to run the research environment, which can be quite large.

However, the command itself should not download any data unless explicitly instructed to do so. If you're seeing a large amount of data being downloaded, it might be due to the data being requested in your research notebook. If your research notebook is set up to use a large amount of data, the lean research command will attempt to download this data.

If you want to prevent data from being downloaded, you can use the --data-provider Local option when running the lean research command. This will tell LEAN to only use locally available data and not download any new data.

Here is how you can use it:

lean research "crypto3" --data-provider Local

Please note that if the required data is not available locally, your research notebook might not work as expected.

For more information on how to manage data with LEAN, you can refer to the QuantConnect Documentation.

thanks for the fast response. I tried using your suggested command. But it's still downloading data. Any idea why this is the case?

lean research "crypto3" --data-provider Local
Pulling quantconnect/research:latest...
latest: Pulling from quantconnect/research
2b55860d4c66: Pulling fs layer
8e058277d049: Pulling fs layer
cac814e1eefd: Pulling fs layer
a298473404ba: Waiting
cbb32a5e1fa8: Waiting
ec88dd5ac36c: Waiting                                                                                                                          
641e09e0138c: Waiting                                                                                                   
6ffecb7c3a6b: Waiting                                                                                                   
f093b4925221: Waiting
fadd53e74843: Downloading [>                                                  ]  56.94MB/8.068GB

The data being downloaded in the logs you provided is actually the Docker image for the research environment, not the financial data. The quantconnect/research:latest image is necessary to run the research environment locally. This image contains all the necessary dependencies and packages to run the research environment, which can be quite large.

The --data-provider Local option only affects the financial data used by your research notebook, not the Docker image.

If you've previously downloaded the Docker image, it should be cached locally and not need to be downloaded again unless there's an update. If you want to avoid downloading updates to the Docker image, you can use the --no-update flag:

lean research "crypto3" --data-provider Local --no-update

This will use the locally cached Docker image and won't pull any updates.

For more information on how to manage Docker images with LEAN, you can refer to the QuantConnect Documentation.

ok makes sense. Thank you!