Discussion Forum - QuantConnect.com

Shailesh Raval INVESTOR

| Reward Discussion

Now, I am trying to run lean on windows 8.1 without Visual Studio.

I copied Lean master to dive S:\lean2401 on local hard disk.,

Installed Docker Toolbox, Oracle VirtualBox.

After starting Docker and within docker terminal window I have run following commands:



$  cd /s/Lean2401



$ docker build -t quantconnect/lean:foundation -f DockerfileLeanFoundation .

Sending build context to Docker daemon  1.464GB

Step 1/39 : FROM phusion/baseimage:0.9.22

 ---> 877509368a8d

Step 2/39 : MAINTAINER QuantConnect <contact@quantconnect.com>

 ---> Using cache

 ---> 983d17a41448

Step 3/39 : CMD ["/sbin/my_init"]

 ---> Using cache

 ---> f481921e5e9a

Step 4/39 : RUN env TZ=UTC

 ---> Using cache

 ---> d47bb3ccf369

Step 5/39 : RUN apt-get update && apt-get install -y git bzip2 unzip wget python

3-pip python-opengl zlib1g-dev &&     apt-get install -y clang cmake curl xvfb l

ibxrender1 libxtst6 libxi6 libglib2.0-dev libopenmpi-dev libstdc++6 &&     apt-g

et install -y r-base pandoc libcurl4-openssl-dev

 ---> Using cache

 ---> 9b86bfef1068

Step 6/39 : RUN apt-get install -y openjdk-8-jdk openjdk-8-jre

 ---> Using cache

 ---> 6a55c93b350b

Step 7/39 : RUN wget http://cdn.quantconnect.com/interactive/ibgateway-latest-st

andalone-linux-x64-v974.4g.sh &&     chmod 777 ibgateway-latest-standalone-linux

-x64-v974.4g.sh &&     ./ibgateway-latest-standalone-linux-x64-v974.4g.sh -q &&

    wget -O ~/Jts/jts.ini http://cdn.quantconnect.com/interactive/ibgateway-late

st-standalone-linux-x64-v974.4g.jts.ini &&     rm ibgateway-latest-standalone-li

nux-x64-v974.4g.sh

 ---> Using cache

 ---> 735f98b9a45a

Step 8/39 : RUN apt-get update && rm -rf /var/lib/apt/lists/*

 ---> Using cache

 ---> 65b917ca95c6

Step 9/39 : RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key

s 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF

 ---> Using cache

 ---> 67d8f52a4d96

Step 10/39 : RUN echo "deb http://download.mono-project.com/repo/ubuntu stable-x

enial/snapshots/5.12.0.226 main" > /etc/apt/sources.list.d/mono-xamarin.list &&

    apt-get update && apt-get install -y binutils mono-complete ca-certificates-

mono mono-vbnc nuget referenceassemblies-pcl &&     apt-get install -y fsharp &&

 rm -rf /var/lib/apt/lists/* /tmp/*

 ---> Using cache

 ---> 1bf5eab4c862

Step 11/39 : ENV CONDA="Miniconda3-4.5.12-Linux-x86_64.sh"

 ---> Using cache

 ---> df9f1becebcb

Step 12/39 : ENV PATH="/opt/miniconda3/bin:${PATH}"

 ---> Using cache

 ---> 49315541e70e

Step 13/39 : RUN wget https://cdn.quantconnect.com/miniconda/${CONDA} &&     bas

h ${CONDA} -b -p /opt/miniconda3 && rm -rf ${CONDA} &&     ln -s /opt/miniconda3

/lib/libpython3.6m.so /usr/lib/libpython3.6m.so

 ---> Using cache

 ---> 07a4e519f49c

Step 14/39 : RUN conda update -y conda pip

 ---> Using cache

 ---> be6d5b160914

Step 15/39 : RUN conda install -y python=3.6.8

 ---> Using cache

 ---> c5745f12bffc

Step 16/39 : RUN conda clean -y --all

 ---> Using cache

 ---> 71782d578588

Step 17/39 : RUN conda install -y                    cython=0.29.11

         numpy=1.17.3                        pandas=0.23.4

 wrapt=1.11.2

 ---> Using cache

 ---> fe24194f3e93

Step 18/39 : RUN conda install -y                    beautifulsoup4=4.8.0

         dill=0.2.9                          jsonschema=3.0.1

 lxml=4.3.4                          msgpack-python=0.5.6                setupto

ols-git=1.2

 ---> Using cache

 ---> bb54923ab211

Step 19/39 : RUN conda install -y                    blaze=0.11.3

         cvxopt=1.2.0                        gensim=3.8.0

 keras=2.2.4                         lightgbm=2.2.3                      nltk=3.

4.3                          pomegranate=0.10.0                  scipy==1.3.1

                     tensorflow=1.13.1                   xarray=0.13.0

 ---> Using cache

 ---> b9bad8eb84be

Step 20/39 : RUN conda install -y -c conda-forge     fbprophet=0.5

         pulp=1.6.8                          pymc3=3.7

 rauth=0.7.3                         scikit-learn==0.21.3                scikit-

multiflow=0.4.1              scikit-optimize=0.5.2               theano=1.0.4

                     tsfresh=0.12.0

 ---> Using cache

 ---> efcc94d83c89

Step 21/39 : RUN conda install -y -c pytorch         pytorch=1.1.0

         torchvision=0.3.0

 ---> Using cache

 ---> 0f10e6bd279c

Step 22/39 : RUN conda install -y -c fastai          nvidia-ml-py3=7.352.0

         fastai=1.0.54

 ---> Using cache

 ---> aa5bce5f1d74

Step 23/39 : RUN conda clean -y --all

 ---> Using cache

 ---> 89df32b75154

Step 24/39 : RUN pip install --upgrade pip

 ---> Using cache

 ---> e074476f114e

Step 25/39 : RUN pip install gym==0.13.1             arch==4.8.1

         copulalib==1.1.0                    copulas==0.2.1

 creme==0.3.0                        cvxpy==1.0.24

 ---> Using cache

 ---> 7105c64858f5

Step 26/39 : RUN pip install deap==1.3.0             mlfinlab==0.4.1

         pykalman==0.9.5                     pyportfolioopt==0.4.3

 pyramid-arima==0.9.0                pyro-ppl==0.3.3

 ---> Using cache

 ---> 9c35f2585e15

Step 27/39 : RUN pip install sklearn==0.0            stable-baselines==2.6.0

         statistics==1.0.3.5                 tensorforce==0.4.3

 QuantLib-Python==1.15               xgboost==0.90

 ---> Using cache

 ---> 36f45382dfd0

Step 28/39 : RUN pip install --upgrade hmmlearn==0.2.2

 ---> Using cache

 ---> 74c37375d2d4

Step 29/39 : RUN python -m nltk.downloader -d /usr/share/nltk_data punkt

 ---> Using cache

 ---> fcecc99f1048

Step 30/39 : RUN conda clean -y --all

 ---> Using cache

 ---> 87eb01f7d839

Step 31/39 : RUN wget http://cdn.quantconnect.com/ta-lib/ta-lib-0.4.0-src.tar.gz

 &&     tar -zxvf ta-lib-0.4.0-src.tar.gz && cd ta-lib &&     ./configure --pref

ix=/usr && make && make install &&     pip install TA-lib && cd .. && rm -irf ta

-lib

 ---> Using cache

 ---> 0deace9a2660

Step 32/39 : RUN wget https://cdn.quantconnect.com/dx/dx-master-ffe19a6.zip &&

   unzip -q dx-master-ffe19a6.zip && cd dx-master &&     python setup.py install

 && cd .. && rm -irf dx-master

 ---> Using cache

 ---> afba2817265c

Step 33/39 : RUN wget https://cdn.quantconnect.com/py-earth/py-earth-master-b209

d19.zip &&     unzip -q py-earth-master-b209d19.zip && cd py-earth-master &&

 python setup.py install && cd .. && rm -irf py-earth-master

 ---> Using cache

 ---> b5ca27423b30

Step 34/39 : RUN wget https://cdn.quantconnect.com/fastText/fastText-master-9479

e4e.zip &&     unzip -q fastText-master-9479e4e.zip && cd fastText-master &&

 python setup.py install && cd .. && rm -irf fastText-master

 ---> Using cache

 ---> 615e7899223e

Step 35/39 : RUN wget https://cdn.quantconnect.com/odo/odo-master-9fce669.zip &&

     unzip -q odo-master-9fce669.zip && cd odo-master &&     python setup.py ins

tall && cd .. && rm -irf odo-master

 ---> Using cache

 ---> 59152a238f73

Step 36/39 : RUN rm -irf *.zip *.gz

 ---> Using cache

 ---> e5da5a5ac62a

Step 37/39 : RUN conda remove --force-remove -y s3transfer   # Installed by gens

im

 ---> Using cache

 ---> 83e7917fac26

Step 38/39 : RUN conda clean -y --all

 ---> Using cache

 ---> 8e7f133935bf

Step 39/39 : RUN conda list

 ---> Using cache

 ---> 7263734fb76d

Successfully built 7263734fb76d

Successfully tagged quantconnect/lean:foundation

SECURITY WARNING: You are building a Docker image from Windows against a non-Win

dows Docker host. All files and directories added to build context will have '-r

wxr-xr-x' permissions. It is recommended to double check and reset permissions f

or sensitive files and directories.







$ docker build -t quantconnect/lean:algorithm -f Dockerfile .

Sending build context to Docker daemon  1.464GB

Step 1/5 : FROM quantconnect/lean:foundation

 ---> 7263734fb76d

Step 2/5 : MAINTAINER QuantConnect <contact@quantconnect.com>

 ---> Using cache

 ---> 0028b77ec909

Step 3/5 : COPY ./Launcher/bin/Release /root/Lean/Launcher/bin/Release

COPY failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder772229067/Launcher/

bin/Release: no such file or directory

What am I missing or doing wrong?

Thanks

Shailesh.

Update Backtest

Shailesh Raval

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Shailesh Raval

1k ,

I have mono, anaconda, VSCode, Oracle VirtualBox & Docker installed on my Windows 8.1 desktop.

Will it work the way I am trying to make that work? May be I am missing many many 101 courses..

Shailesh Raval

1k ,

output from msbuild command run.

$ msbuild  QuantConnect.Lean.sln

Microsoft (R) Build Engine version 16.5.0-ci for Mono

Copyright (C) Microsoft Corporation. All rights reserved.



Build started 2/6/2020 1:04:25 PM.

Project "S:\lean2401\QuantConnect.Lean.sln" on node 1 (default targets).

ValidateSolutionConfiguration:

  Building solution configuration "Debug|Any CPU".

ValidateProjects:

  The project "QuantConnect.Algorithm.FSharp" is not selected for building in s

  olution configuration "Debug|Any CPU".

  The project "QuantConnect.Algorithm.Python" is not selected for building in s

  olution configuration "Debug|Any CPU".

  The project "QuantConnect.Algorithm.PythonTools" is not selected for building

   in solution configuration "Debug|Any CPU".

  The project "QuantConnect.PythonToolbox" is not selected for building in solu

  tion configuration "Debug|Any CPU".

Project "S:\lean2401\QuantConnect.Lean.sln" (1) is building "S:\lean2401\Launch

er\QuantConnect.Lean.Launcher.csproj" (2) on node 1 (default targets).

Project "S:\lean2401\Launcher\QuantConnect.Lean.Launcher.csproj" (2) is buildin

g "S:\lean2401\Algorithm.CSharp\QuantConnect.Algorithm.CSharp.csproj" (3:2) on

node 1 (default targets).

Project "S:\lean2401\Algorithm.CSharp\QuantConnect.Algorithm.CSharp.csproj" (3:

2) is building "S:\lean2401\Algorithm.Framework\QuantConnect.Algorithm.Framewor

k.csproj" (18:2) on node 1 (default targets).

Project "S:\lean2401\Algorithm.Framework\QuantConnect.Algorithm.Framework.cspro

j" (18:2) is building "S:\lean2401\Algorithm\QuantConnect.Algorithm.csproj" (5:

4) on node 1 (default targets).

Project "S:\lean2401\Algorithm\QuantConnect.Algorithm.csproj" (5:4) is building

 "S:\lean2401\Configuration\QuantConnect.Configuration.csproj" (9:3) on node 1

(default targets).

Project "S:\lean2401\Configuration\QuantConnect.Configuration.csproj" (9:3) is

building "S:\lean2401\Logging\QuantConnect.Logging.csproj" (14:4) on node 1 (de

fault targets).

GenerateTargetFrameworkMonikerAttribute:

Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output fi

les are up-to-date with respect to the input files.

CoreCompile:

  C:\Program Files\Mono\lib\mono\msbuild\15.0\bin\Roslyn\csc.exe /noconfig /now

  arn:1701,1702 /nostdlib+ /errorreport:prompt /warn:4 /doc:bin\Debug\QuantConn

  ect.Logging.xml /define:DEBUG;TRACE /highentropyva+ /reference:S:\lean2401\pa

  ckages\MathNet.Filtering.0.4.0\lib\net40\MathNet.Filtering.dll /reference:S:\

  lean2401\packages\MathNet.Numerics.3.19.0\lib\net40\MathNet.Numerics.dll /ref

  erence:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETF

  ramework\v4.5.2\Microsoft.CSharp.dll" /reference:"C:\Program Files (x86)\Refe

  rence Assemblies\Microsoft\Framework\.NETFramework\v4.5.2\mscorlib.dll" /refe

  rence:S:\lean2401\packages\NLog.4.4.11\lib\net45\NLog.dll /reference:S:\lean2

  401\packages\StringInterpolationBridgeStrong.0.9.1\lib\net40\StringInterpolat

  ionBridge.dll /reference:"C:\Program Files (x86)\Reference Assemblies\Microso

  ft\Framework\.NETFramework\v4.5.2\System.ComponentModel.Composition.dll" /ref

  erence:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETF

  ramework\v4.5.2\System.Core.dll" /reference:"C:\Program Files (x86)\Reference

   Assemblies\Microsoft\Framework\.NETFramework\v4.5.2\System.Data.DataSetExten

  sions.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\

  Framework\.NETFramework\v4.5.2\System.Data.dll" /reference:"C:\Program Files

  (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.2\System.dl

  l" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framewor

  k\.NETFramework\v4.5.2\System.Numerics.dll" /reference:"C:\Program Files (x86

  )\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.2\System.Xml.dl

  l" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framewor

  k\.NETFramework\v4.5.2\System.Xml.Linq.dll" /debug+ /debug:full /filealign:51

  2 /out:\Debug\QuantConnect.Logging.dll /ruleset:..\QuantConnect.ruleset /subs

  ystemversion:6.00 /target:library /utf8output /langversion:6 /features:flow-a

  nalysis /features:flow-analysis /features:flow-analysis /features:flow-analys

  is /features:flow-analysis /analyzer:..\packages\Microsoft.CodeAnalysis.Versi

  onCheckAnalyzer.2.9.3\analyzers\dotnet\Microsoft.CodeAnalysis.VersionCheckAna

  lyzer.dll /analyzer:..\packages\Microsoft.CodeQuality.Analyzers.2.9.3\analyze

  rs\dotnet\cs\Humanizer.dll /analyzer:..\packages\Microsoft.CodeQuality.Analyz

  ers.2.9.3\analyzers\dotnet\cs\Microsoft.CodeQuality.Analyzers.dll /analyzer:.

  .\packages\Microsoft.CodeQuality.Analyzers.2.9.3\analyzers\dotnet\cs\Microsof

  t.CodeQuality.CSharp.Analyzers.dll /analyzer:..\packages\Microsoft.NetCore.An

  alyzers.2.9.3\analyzers\dotnet\cs\Microsoft.NetCore.Analyzers.dll /analyzer:.

  .\packages\Microsoft.NetCore.Analyzers.2.9.3\analyzers\dotnet\cs\Microsoft.Ne

  tCore.CSharp.Analyzers.dll /analyzer:..\packages\Microsoft.NetFramework.Analy

  zers.2.9.3\analyzers\dotnet\cs\Microsoft.NetFramework.Analyzers.dll /analyzer

  :..\packages\Microsoft.NetFramework.Analyzers.2.9.3\analyzers\dotnet\cs\Micro

  soft.NetFramework.CSharp.Analyzers.dll ..\Common\Properties\SharedAssemblyInf

  o.cs CompositeLogHandler.cs CompositeNLogHandler.cs NLogHandler.cs QueueLogHa

  ndler.cs FunctionalLogHandler.cs ConsoleLogHandler.cs FileLogHandler.cs ILogH

  andler.cs ILogHandlerExtensions.cs LogEntry.cs LogType.cs Properties\Assembly

  Info.cs Log.cs RegressionFileLogHandler.cs WhoCalledMe.cs "C:\Users\Shailesh\

  AppData\Local\Temp\.NETFramework,Version=v4.5.2.AssemblyAttributes.cs"

  Using shared compilation with compiler from directory: C:\Program Files\Mono\

  lib\mono\msbuild\15.0\bin\Roslyn

C:\Program Files\Mono\lib\mono\msbuild\15.0\bin\Roslyn\Microsoft.CSharp.Core.ta

rgets(59,5): error MSB6006: "csc.exe" exited with code 1. [S:\lean2401\Logging\

QuantConnect.Logging.csproj]

Done Building Project "S:\lean2401\Logging\QuantConnect.Logging.csproj" (defaul

t targets) -- FAILED.



Done Building Project "S:\lean2401\Configuration\QuantConnect.Configuration.csp

roj" (default targets) -- FAILED.



Done Building Project "S:\lean2401\Algorithm\QuantConnect.Algorithm.csproj" (de

fault targets) -- FAILED.



Done Building Project "S:\lean2401\Algorithm.Framework\QuantConnect.Algorithm.F

ramework.csproj" (default targets) -- FAILED.



Done Building Project "S:\lean2401\Algorithm.CSharp\QuantConnect.Algorithm.CSha

rp.csproj" (default targets) -- FAILED.



Done Building Project "S:\lean2401\Launcher\QuantConnect.Lean.Launcher.csproj"

(default targets) -- FAILED.



Done Building Project "S:\lean2401\QuantConnect.Lean.sln" (default targets) --

FAILED.





Build FAILED.



"S:\lean2401\QuantConnect.Lean.sln" (default target) (1) ->

"S:\lean2401\Launcher\QuantConnect.Lean.Launcher.csproj" (default target) (2) -

>

"S:\lean2401\Algorithm.CSharp\QuantConnect.Algorithm.CSharp.csproj" (default ta

rget) (3:2) ->

"S:\lean2401\Algorithm.Framework\QuantConnect.Algorithm.Framework.csproj" (defa

ult target) (18:2) ->

"S:\lean2401\Algorithm\QuantConnect.Algorithm.csproj" (default target) (5:4) ->

"S:\lean2401\Configuration\QuantConnect.Configuration.csproj" (default target)

(9:3) ->

"S:\lean2401\Logging\QuantConnect.Logging.csproj" (default target) (14:4) ->

(CoreCompile target) ->

  C:\Program Files\Mono\lib\mono\msbuild\15.0\bin\Roslyn\Microsoft.CSharp.Core.

targets(59,5): error MSB6006: "csc.exe" exited with code 1. [S:\lean2401\Loggin

g\QuantConnect.Logging.csproj]



    0 Warning(s)

    1 Error(s)



Time Elapsed 00:00:09.65

Rahul Chowdhury

39.9k ,

Hey Shailesh,

You should try building with Visual Studio. By default, the LEAN executable is stored in bin/Debug. The docker is looking for bin/Release. If you change the docker file to use bin/Debug, it should fix the issue.

Shailesh Raval

1k ,

Hi Rahul,

Why should I try building with Visual Studio when I am taking docker route? When I use docker file, like linux, here too msbuild command from mono should be used (and it is being used).

docker build command has created identical environment, right? And NO, I have not changed any thing in docker file. I am following the same instructions as Usage mentioned in last few lines of dockerfile. Am I missing anything? I am new to docker and missing docker101 or lean101 for sure.



# Usage:

# docker build -t quantconnect/lean:foundation -f DockerfileLeanFoundation .

# docker build -t quantconnect/lean:algorithm -f Dockerfile .

# docker run -v "(absolute to your data folder):/root/Lean/Data" quantconnect/lean:algorithm

I am stuck at second command

# docker build -t quantconnect/lean:algorithm -f Dockerfile .

Thanks

shailesh.

Alexandre Catarino

100.3k ,

Hi Shailesh Raval ,

It looks like you are compiling Lean in Windows:
C:\Program Files\Mono\lib\mono\msbuild\15.0\bin\Roslyn\csc.exe
Why are you using Mono's compiler?

In my Windows 10, I can compile with msbuild that uses:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Roslyn\csc.exe

That's why we suggested using Visual Studio.

You can compile in docker image by uncommenting lines 12 to 22 of the Dockerfile and commenting line 28 and running:

docker build -t quantconnect/lean:algorithm -f Dockerfile .

Shailesh Raval

1k ,

Hi Alexandre Catarino,,

Thank you for looking into my problem. I was away from QC for few days. If I understand correctly, docker gives us minimalistic ubuntu virtual machine and docker files/images are like command scripts to do any thing on that machine. Docker installations ensures that, irrespective of underlying OS or its version, those docker files/images run exactly the same way and give identical results.

Let me try option 1 from dockerfile and see how it goes.. Will come back by week-end and update.

Thanks and regards,

Shailesh.

Shailesh Raval INVESTOR

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

240100 Quants.

Online Now

Stay Connected

Docker build error

Started By:

Join QuantConnect for Free Sign Up

Platform

SIGN IN

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

240100 Quants.

Online Now

Stay Connected

Docker build error

Started By:

Join QuantConnect for Free Sign Up