Comments (18)
from amptk.
from amptk.
from amptk.
Okay, so following that install:
conda create -n amptk python=3.6 gcc amptk
I get a host of compiler related clobber
issues once the installation is almost done...
SafetyError: The package for amptk located at /mnt/lustre/macmaneslab/devon/.conda/pkgs/amptk-1.2.4-py36r3.4.1_0
appears to be corrupted. The path 'opt/amptk-1.2.4/lib/amptklib.py'
has a sha256 mismatch.
reported sha256: d77dcaf93d7fe79599d734f74c05424591ab9f1be2dc5e023cb0b929346c3875
actual sha256: 831fe2ea3ec4dabbf0740ffa4c97008f3f63d52df41efce2ae44945a415930b5
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libasan.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libatomic.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libatomic.so.1'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgcc_s.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgcc_s.so.1'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgomp.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgomp.so.1'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgomp.so.1.0.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libitm.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libitm.so.1'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libitm.so.1.0.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libquadmath.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libquadmath.so.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libquadmath.so.0.0.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libtsan.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libtsan.so.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libtsan.so.0.0.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'share/info/libgomp.info'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgcc-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'share/info/libquadmath.info'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgfortran-3.0.0-1, conda-forge::libgcc-7.2.0-h69d50b8_2, defaults::gcc-4.8.5-7
path: 'lib/libgfortran.so.3'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgfortran-3.0.0-1, conda-forge::libgcc-7.2.0-h69d50b8_2, defaults::gcc-4.8.5-7
path: 'lib/libgfortran.so.3.0.0'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libgfortran-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libgfortran.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libstdcxx-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libstdc++.so'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::libstdcxx-ng-7.2.0-hdf63c60_3, defaults::gcc-4.8.5-7
path: 'lib/libstdc++.so.6'
And then if I run amptk --version
:
(amptk) [devon@premise ~]$ amptk --version
/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
Traceback (most recent call last):
File "/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/bin/amptk", line 2, in <module>
import amptk
File "/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/opt/amptk-1.2.4/bin/amptk.py", line 15, in <module>
import lib.amptklib as amptklib
File "/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/opt/amptk-1.2.4/lib/amptklib.py", line 14
import edlib-aligner as edlib
from amptk.
I think there was some weird lingering edit that remained in the $HOME//.conda/envs/amptk/opt/amptk-1.2.4/lib/amptklib.py
file where I had substituted:
import edlib-aligner as edlib
I noticed also that edlib
wasn't installed with:
conda create -n amptk python=3.6 gcc amptk
So I did:
conda install edlib
And because of that weird numpy error:
conda update numpy
And now I don't have any issue with running amptk --version
from amptk.
from amptk.
I don't know. I'm just looking at what was installed in the virtual environment bin
, and there wasn't any edlib-aligner
program initially until I manually installed it.
I don't see any python-edlib
at the moment, but could also manually install
from amptk.
For what it's worth, amptk illumina
appears to be running now, once I've made those modifications (installing edlib
and updating numpy
)
from amptk.
from amptk.
Managed to get amptk illumina
to work but now getting stuck on the Dada2 clustering step.
Recall that my virtual environment was created with:
conda create -n amptk python=3.6 gcc amptk
The error in the Rscript.log file is a cluster of things, 1) It can't seem to detect a mirror, and 2) it can't figure out how to load the libraries. A little snippet of that error at the beginning:
Warning: failed to download mirrors file (cannot open URL 'https://cran.r-project.org/CRAN_mirrors.csv'); using local file '/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/R/doc/CRAN_mirrors.csv'
Warning message:
In download.file(url, destfile = f, quiet = TRUE) :
URL 'https://cran.r-project.org/CRAN_mirrors.csv': status was 'Couldn't connect to server'
Loading required package: ShortRead
Loading required package: BiocGenerics
Loading required package: methods
Loading required package: parallel
...
...
Loading required package: BiocParallel
Error: package or namespace load failed for ‘BiocParallel’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/BiocParallel/libs/BiocParallel.so':
/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/R/bin/exec/../../lib/../.././libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/BiocParallel/libs/BiocParallel.so)
Failed with error: ‘package ‘BiocParallel’ could not be loaded’
I figured if the program can't download them, then it can't install them. So I manually went into the virtual environment and tried to do that:
source("https://bioconductor.org/biocLite.R")
biocLite("BiocGenerics")
I also tried things from CRAN:
install.packages("Rcpp", repos='http://cran.us.r-project.org')
The error that I receive in each of these cases is the library loading message:
Error: package or namespace load failed for ‘Rcpp’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/libs/Rcpp.so':
/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/R/bin/exec/../../lib/../.././libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/libs/Rcpp.so)
Error: package ‘Rcpp’ could not be loaded
I tried:
conda install libgcc
... but that didn't work any better
All I can find in the forums is something along the lines of sudo apt-get install libstdc++6
, but I don't have sudo privileges on my compute cluster.
Likewise, I thought maybe updating Conda from the current v4.5.4 to the newest v4.5.11 might help, but I again ran into sudo problems.
Any thoughts? Maybe I should ditch DADA2 and just use UNoise? What's that cryptic line in your docs about Likewise, the output data from UNOISE3 is the same as DADA2 and UNOISE2, although “it works better”….? What clustering algorithm to you trust?
from amptk.
One other thing: I also tried specifying the Conda-specific .libPath
during the install, and it didn't work either:
install.packages("tidyverse", repos='http://cran.us.r-project.org', lib='/mnt/lustre/macmaneslab/devon/.conda/envs/amptk/lib/R/library')
Gives the same error
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/bindrcpp/libs/bindrcpp.so':
/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /mnt/lustre/macmaneslab/devon/R/x86_64-pc-linux-gnu-library/3.4/bindrcpp/libs/bindrcpp.so)
ERROR: lazy loading failed for package ‘dbplyr’
from amptk.
So sounds like the R packages aren’t installed correctly in the conda package. There is an R channel for conda, but if you go to anaconda cloud you can search for all available packages. You need to find the conda packages for those missing R dependencies. Usually they look like r-package or for dada2 they are listed as bioconductor-dada2. Note - I’ve had problems with the R packages on conda as well.... but have let tried in awhile. But these errors seem to be related to the compiler libraries that are missing - ie the package was compiled with a librarynot in your system. Conda theoretically should be managing this, but they switched compilers a few months back that perhaps is still causing some issues.
from amptk.
Right. That's what I was manually trying to do.
The missing R packages are both within CRAN and within Bioconductor. My issue is that I can't install them within R itself.
If I'm hearing you right, it's to try to install those packages through anaconda, rather than manually doing it with R?
I'll give that a shot
from amptk.
I'm still curious though - could I just use UNoise instead? What's the upside/downside of DADA2 vs. UNoise?
from amptk.
Sure. In practice dada2 and unoise3 are very similar. Dada2 is a little bit more aggressive in chimera filtering the last time I compared, but will depend on the versions of each software. Unoise3 is much much faster than dada2. Both ESV pipelines in amptk run an additional 97% clustering step, so user can use either data downstream. Whether you should use ESV or exact sequence variants versus clustering (uparse) depends on the amplicon - there are many where I don’t think the ESVs make biological sense.
from amptk.
I think the bulk of these issues were because the compute nodes on my cluster don't have internet access, so when the R script wants to download/upgrade packages, the program dies.
It was only after going back into R and manually downloading ggplot2
that an error message became obvious: ggplot2
install required another R library, and only after that program was installed could ggplot2
be installed, and only after that could dada2 run.
But after long last, after that manual install, it's working just fine.
Weird.
from amptk.
Okay well good to know. I added those automatic download scripts as a convenience option to download and install missing packages, but sounds like I should just delete that so errors are more simple: ie ggplot2 not installled.....
from amptk.
The "auto-install" in the R scripts was removed. Re-open if this is still an issue.
from amptk.
Related Issues (20)
- Issue installing AMPtk (Mac OS - M1 chip) HOT 2
- getting NoneType vs int error in clustering step
- Error when run quick start HOT 7
- usearch9 not found when generate UTAX database
- VSEARCH error on amptk -filter step
- Support Python 3.8 onwards HOT 3
- SyntaxError in "duplicate ID in mapping file: XXX, exiting"
- Default for -p, --index_bleed documented as 0.005 HOT 1
- Typo "Bjerkandara adusta" --> "Bjerkandera adusta" HOT 1
- Missing species names in amptk_mock1.fa HOT 3
- Missing final new line in amptk_mock1.fa and amptk_synmock.fa HOT 2
- Inconsistent primer trimming sequence in amptk_mock*.fa HOT 5
- Matching MockA, MockB1 and MockB2 to FASTQ filenames HOT 2
- platform.linux_distribution is removed since Python 3.8 HOT 1
- Species names in amptk_mock2.fa and amptk_mock3.fa vs Figure 4
- new users cannot install amptk properly, please help HOT 3
- unoise3 clustering HOT 5
- Problem with TypeError during AMPtk cluster HOT 11
- Saw you started some prelim ONT methods HOT 2
- Problematic unoise3 implementation with VSEARCH HOT 13
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from amptk.