Dockerfile fails to compile (poetry install, step 27) about scope HOT 9 CLOSED

Hey Daniele,

Unfortunately, due to a large rewrite effort, that fact that the Docker setup was contributed by an external collaborator and also not currently used by us, the existing Dockerfile has been somewhat left behind. We intend to fix this with the next "big" version of SCope, although I realize this doesn't help you right now.

I've tried to also do some debugging, but have come to the same conclusion as you, that for some reason, running the commands manually within the container seems to work, but the build is broken. I managed to overcome the llvm issues, but ran into problems with scipy refusing to install via the wheel.

Although not optimal, is running using a conda environment locally an option for you at the moment? This is how we currently run our main instance. Sorry I haven't been more help.

Cheers,

Kris

from scope.

Dear Kris,

Thanks a lot for your answer.

I have not tried to do install SCope through Conda, do you have a .yaml file I can use? Also, do I need to install some dependency system wide? I was using Docker because I fear that nodejs installation may broke other tools and dependencies.

If you have some instruction to share on your local install that differs from the one listed on the README page, I'll be glad to test them on!

Cheers,

Daniele

from scope.

Hey Daniele,

I don't have a .yaml but poetry should lock the versions of the python packages you need. The benefit of conda here is that you can actually install both python and nodejs within the conda env which will prevent issues with system dependencies.

This conda create command should work for master: conda create -n scope "python=3.7.4" "nodejs=10.13.0", I've just checked and these are the versions we're using on our main deployment. Then just make sure to conda activate scope before you do the install instructions in the README.md.

If you have issues, just let me know and I'll see where I can help out.

Cheers,

Kris

from scope.

Helo Kris,

I have tried following the instructions listed on the site in a conda environment created using your specifications and the npm install command fails.

I attach here the (quite long) error log, it seems to be a problem of some package that is corrupted (at the beginning of the log) that subsequently create the error cascade.

Can you look it up and point me to the right fix?

Thanks!

Cheers,

Daniele

scope_npm_error.log

from scope.

Hey Daniele,

I've seen this sometimes be cause by the package.lock files from npm, can you try removing these two files from the root ofthe repository rm package-lock.json opt/scopeserver/bindserver/package-lock.json and see if that works? You may also need to remove the existing node module files rm -r node_modules opt/scopeserver/bindserver/node_modules.

Cheers,

Kris

from scope.

Hi kris,

After a few workarounds (machine-related issues), I managed to successfully complete the npm-install command.

However, npm run scope fails to launch, apparently it seems that the scope-server is not installed.

I honestly don't know where to look up, I even installed poetry as a dependency and re-run the installation on a separate machine.

I attach here both the npm-install log and the npm-run-scope error logs.

Thanks again,

Daniele
npm_run_scope.log
npm_install.log

from scope.

Hey Daniele,

I'm really sorry about all of these problems - this is of course not the experience we want people to have when trying to run SCope. It looks like due to various updates, certain parts of the install are not working. I have worked out a set of commands here, that from a fresh start will get SCope running for you. Would you mind giving these a go?

git clone https://github.com/aertslab/SCope
cd SCope

# Install miniconda
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -p ./miniconda3 
source miniconda3/bin/activate
conda create -n scope "python=3.7.4" "nodejs=10.13.0"
conda activate scope

# Fix issues with package versions
rm opt/scopeserver/bindserver/package-lock.json
sed -i 's/"bundled": true/"bundled": false/g' package-lock.json

# Install and build client files
npm install
npm run build

# Move client files to a unique directory - prevents leaking files when using python http.server
mkdir host
cd host
ln -s ../assets ./
ln -s ../images ./
ln -s ../index.html ./
ln -s ../index.js ./
ln -s ../src ./

# Use python to host the frontend
python -m http.server 55850

#In another terminal run the backend
cd SCope
source miniconda3/bin/activate
conda activate scope
cd opt
TMPDIR=/tmp DATABASE_URL="sqlite:///scope.sqlite" SCOPE_CONFIG=../config.json poetry run hypercorn --bind 0.0.0.0:55854 main:scope_api

Cheers,

Kris

from scope.

Hi @KrisDavie,

No worries, I understand that these problems may occur, especially with nodejs, and they are quite pesky to resolve.

I followed your instructions carefully but, alas, the npm install still fails to build.

here is the error log:

(scope) caposquared@RainbowUbuntu:~/SCope$ npm install                                                                                                          
> [email protected] preinstall /home/caposquared/SCope            
> npm i shelljs command-exists && node bin/install.js                                                                                                              
npm ERR! code ETARGET                                       
npm ERR! notarget No matching version found for [email protected]       
npm ERR! notarget In most cases you or one of your dependencies are requesting               
npm ERR! notarget a package version that doesn't exist.
                                      
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/caposquared/.npm/_logs/2022-01-20T14_03_56_297Z-debug.log                                                              
npm ERR! code ELIFECYCLE
npm ERR! errno 1      
npm ERR! [email protected] preinstall: `npm i shelljs command-exists && node bin/install.js`
npm ERR! Exit status 1
npm ERR!                                                                              
npm ERR! Failed at the [email protected] preinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm WARN Local package.json exists, but node_modules missing, did you mean to install?            
                           
npm ERR! A complete log of this run can be found in:                                                       
npm ERR!     /home/caposquared/.npm/_logs/2022-01-20T14_03_56_384Z-debug.log

A few things that I tried:

1. wget on all the grb-bus instances from my machine seems to work as a user (thought it may be some incorrect URL, especially this one https://github.com/aertslab/grpc-bus/releases/download/v1.0.2/grpc-bus-1.0.2.tgz since it seems the only matching grpc-bus version
2. removing the package-lock.json and run npm install seems to work, but npm run build fails

I sadly have no confidence with npm so these were workarounds I tried as simple sanity checks, but if you have other suggestions I'll be glad to know them

I attach here the two error logs generated by the npm install command.
2022-01-20T14_03_56_384Z-debug.log
2022-01-20T14_03_56_297Z-debug.log

In case this process becomes too painful to debug, we can also meet and check it together, no problem for me.

Ciao,

Daniele

from scope.

Ciao Kris,

Thanks a lot for your help, your time and for all the precious info that you gave me, now everything is a little bit clearer :)

I attach here a small .md file that sums up all the operations that we did yesterday. I hope that I did not forget something.

Feel free to edit and distribute it as much as you want, hopefully it can be useful for other SCope users as well!

Waiting for the new Scope to unroll, keep up with the good work!

install_scope.md

Ciao,

Daniele

from scope.