This project forked from multiqc/multiqc
Aggregate results from bioinformatics analyses across many samples into a single report.
Home Page: http://multiqc.info
License: GNU General Public License v3.0
Description of bug:
MultiQC/HTStream module is giving me an error when I try to run it on a set of JSON files generated on a set of samples with ~115m reads. I think that a scientific notation string is being passed to MultiQC code and causing ValueError when it is converted to a float. (e.g. float('6.921012491527816-06')).
MultiQC Error log:
(base) shunter@server:~/work/2021-06-09-FastqQC-HTStream$ multiqc ./tmp/
[WARNING] multiqc : MultiQC Version v1.10.1 now available!
[INFO ] multiqc : This is MultiQC v1.10.dev0
[INFO ] multiqc : Template : default
[INFO ] multiqc : Searching : /home/shunter/work/2021-06-09-FastqQC-HTStream/tmp
Searching ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1
[ERROR ] multiqc : Oops! The 'htstream' MultiQC module broke...
Please copy the following traceback and report it at https://github.com/ewels/MultiQC/issues
If possible, please include a log file that triggers the error - the last file found was:
./tmp/Sample1_stats.json
============================================================
Module htstream raised an exception: Traceback (most recent call last):
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/multiqc.py", line 579, in run
output = mod()
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/modules/htstream/htstream.py", line 106, in __init__
self.process_data(self.data)
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/modules/htstream/htstream.py", line 218, in process_data
section_dict = app.execute(stats_dict, index)
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/modules/htstream/apps/SuperDeduper.py", line 183, in execute
"Table": self.table(stats_json, pe_total_loss, se_total_loss, index),
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/modules/htstream/apps/SuperDeduper.py", line 54, in table
return table.plot(json, headers)
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/plots/table.py", line 53, in plot
return make_table(dt)
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/plots/table.py", line 138, in make_table
c_scale = mqc_colour.mqc_colour_scale(header["scale"], header["dmin"], header["dmax"])
File "/home/shunter/opt/conda2/lib/python3.8/site-packages/multiqc-1.10.dev0-py3.8.egg/multiqc/utils/mqc_colour.py", line 33, in __init__
if float(minval) == float(maxval):
ValueError: could not convert string to float: '6.921012491527816-06'
============================================================
[WARNING] multiqc : No analysis results found. Cleaning up..
[INFO ] multiqc : MultiQC complete
(base) shunter@server:~/work/2021-06-09-FastqQC-HTStream$
[](url)
File that triggers the error:
GitHub doesn't like JSON files for some reason, so this is renamed as a txt:
Sample1_stats.txt
Or unzip:
Sample1_stats.json.zip
MultiQC run details (please complete the following):
multiqc ./tmp/
(base) shunter@server:~/opt/src/MultiQC$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
(base) shunter@server:~/opt/src/MultiQC$ git show
commit d5fd74af3d2441593bc465f6b5d0fe6236fe35ef
Merge: 0d3d3dd 6c0e655
Author: Bradley N. Jenner <[email protected]>
Date: Fri Mar 12 13:42:31 2021 -0800
Merge pull request #4 from ewels/pr_1303_fixes
Fixes for ewels/MultiQC#1303
Linux
Python 3.8.5
I created a conda environment, then from the git clone, used:
pip setup.py install
commas aren't showing up for me for the input read counts
Getting these warnings/errors when trying to install using the --prefix option:
/software/anaconda3/4.5.12/lssc0-linux/lib/python3.6/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '1.9dev' to '1.9.dev0'
normalized_version,
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 2] No such file or directory: '/share/biocore/joshi/projects/VGL_GBS_pipeline/mqc_install/lib/python3.6/site-packages/test-easy-install-53928.write-test'
Hi,
I was previously able to run multiqc with htstream support. However, I have recently upgraded to a new computer (ubuntu 22.04.1) and my new multiqc installation doesn't recognize the "-m htstream" option even after installing it using the latest version from git:
$ git clone https://github.com/s4hts/MultiQC.git
$ cd MultiQC/
$ pip install .
The command I used previously in my pipeline no longer works stating that "htstream" is not in the modules list. However, there is a htstream directory in the source code.
No response
`$ qcdir=qc/
$ multiqc --interactive -m htstream -m fastqc -m fastp -o ${qcdir} ${qcdir}
Usage: multiqc [OPTIONS] [ANALYSIS DIRECTORY]
This is MultiQC v1.15.dev0
For more help, run 'multiqc --help' or visit http://multiqc.info
╭─ Error ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Invalid value for '-m' / '--module': 'htstream' is not one of 'adapterRemoval', 'afterqc', 'anglerfish', 'bamtools', 'bbduk', 'bbmap', 'bcftools', 'bcl2fastq', 'bclconvert', 'biobambam2', 'biobloomtools', │
│ 'biscuit', 'bismark', 'bowtie1', 'bowtie2', 'busco', 'bustools', 'ccs', 'cellranger', 'checkqc', 'clipandmerge', 'clusterflow', 'conpair', 'custom_content', 'cutadapt', 'damageprofiler', 'dedup', │
│ 'deeptools', 'diamond', 'disambiguate', 'dragen', 'dragen_fastqc', 'eigenstratdatabasetools', 'fastp', 'fastq_screen', 'fastqc', 'featureCounts', 'fgbio', 'filtlong', 'flash', 'flexbar', 'gatk', │
│ 'gffcompare', 'goleft_indexcov', 'gopeaks', 'happy', 'hicexplorer', 'hicpro', 'hicup', 'hifiasm', 'hisat2', 'homer', 'hops', 'htseq', 'humid', 'interop', 'ivar', 'jcvi', 'jellyfish', 'kaiju', 'kallisto', │
│ 'kat', 'kraken', 'leehom', 'lima', 'longranger', 'macs2', 'malt', 'methylQA', 'minionqc', 'mirtop', 'mirtrace', 'mosdepth', 'motus', 'mtnucratio', 'multivcfanalyzer', 'nanostat', 'nextclade', 'ngsderive', │
│ 'odgi', 'optitype', 'pangolin', 'pbmarkdup', 'peddy', 'phantompeakqualtools', 'picard', 'porechop', 'preseq', 'prinseqplusplus', 'prokka', 'purple', 'pychopper', 'pycoqc', 'qc3C', 'qorts', 'qualimap', │
│ 'quast', 'rna_seqc', 'rockhopper', 'rsem', 'rseqc', 'salmon', 'sambamba', 'samblaster', 'samtools', 'sargasso', 'sentieon', 'seqyclean', 'sexdeterrmine', 'sickle', 'skewer', 'slamdunk', 'snippy', │
│ 'snpeff', 'snpsplit', 'somalier', 'sortmerna', 'stacks', 'star', 'supernova', 'theta2', 'tophat', 'trimmomatic', 'umitools', 'varscan2', 'vcftools', 'vep', 'verifybamid', 'whatshap'. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
`It would be nice to have a toggle where you could toggle between the actual read counts vs. the log10 read counts for the histogram.
Hi Bradly,
According to the installation instruction in README.md, it seems "If you would like the development version instead, the command is: pip install --upgrade --force-reinstall git+https://github.com/ewels/MultiQC.git".
However, actually with this command, I can only get the original version of MultiQC, but not the version working with HTStream JSON files. I think the correct command should be: pip install --upgrade --force-reinstall git+https://github.com/s4hts/MultiQC.git
Thanks,
Congcong
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.