taxator: incorrect exception message is given when running multi-threaded about taxator-tk HOT 8 OPEN

Error occurring in the following environment:

Distributor ID: Debian
Description: Debian GNU/Linux 8.4 (jessie)
Release: 8.4
Codename: jessie
Linux sebio03 4.3.0-0.bpo.1-amd64 #1 SMP Debian 4.3.5-1~bpo8+1 (2016-02-23) x86_64 GNU/Linux

The error persists with any number of threads > 1

from taxator-tk.

Thanks, I'm adding your findings when run with one core here:

An unrecoverable error occurred: std::exception

Here is some debugging information to locate the problem:
src/fileparser.hh(52): Throw in function FileParser<FactoryType>::RecordType* FileParser<FactoryType>::next() [with FactoryType = AlignmentRecordFactory<AlignmentRecordTaxonomy>; FileParser<FactoryType>::RecordType = AlignmentRecordTaxonomy]
Dynamic exception type: boost::exception_detail::clone_impl<Exception>
std::exception::what: std::exception
[exception_tag_line*] = 2
[exception_tag_taxid*] = 349521
[exception_tag_general*] = bad alignment reference taxon

In all cases (reduced files, normal file, one core, multiple cores) the alignment file sample.alignments.gz is generated but never with more than 1000 lines (looks like the alignment tool is not running properly or there is some buffer limitation).

from taxator-tk.

I'm beginning to remember that the exception you see in single-threaded mode is shaded by the threading exception which happens after. What the exception means is that in alignment line number 2, the taxon 349521 cannot be found in your taxonomy. The solution would be to download a proper (recent) taxonomy from ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/, use a taxonomy version that's closer to the creation date of your data or simply to adjust the taxon in the mapping file to point to a valid taxon in your taxonomy. If your taxonomy is recent, it is often easiest to look the taxon id up at

http://www.ncbi.nlm.nih.gov/Taxonomy/taxonomyhome.html/

and replace it with the shown taxon id in the file mapping.tax (taxator-tk extended version naming convention). Working with and replacing the taxonomy is easy: you need the files nodes.dmp and names.dmp which are text files you can grep for names and taxon ids and drop it into the subfolder ncbi-taxonomy in the refpack folder (again, assuming the extended version). I suppose this will resolve your problems.

I keep this bug open but it means that taxator should display the correct exception message also when run multi-threaded.

from taxator-tk.

I created the mapping.tax file using the eutils and downloaded taxdmp this week, so the taxonomic ids are supposed to be up-to-date.

In the alignment file, line 2 represent the first alignment:
$ zcat sample.alignments.gz | head -n 2
#query ID query begin query end query length reference ID reference begin reference end score E-value identities alignment length alignment code
S|S1|R0/2 1 99 100 NC_007645.1 5914243 5914341 49.0 0.0 74 99

This first reference is contained in the mapping.tax file:
$ grep "NC_007645.1" mapping.tax
NC_007645.1 349521

as well as in the nodes.dmp file:
$ grep "349521" nodes.dmp
349521 | 158327 | no rank | | 0 | 1 | 11 | 1 | 0 | 1 | 1 | 0 ||

So I believe that the taxonomy is not the problem.
I've tried to run with a very small sample of my reads (around 100 reads). The error is still there, but now the tool finishes the analyses (although all reports are empty).

from taxator-tk.

Note that by default the taxonomy is pruned after loading into memory by deleting all nodes that are not on major ranks such as species, genus, family, order, class, phylum or superkingdom. Therefore, if your mapping points at taxa below species level, they cannot be found in the taxonomy. The pruning is the default behavior in the extended package. It could be changed via a command line parameter of taxator, if you need assignments below species level, but the preferred way would be to map the taxa in the mapping file to the species level. Please follow the instructions for building a custom refpack as described under "NCBI taxonomy" in https://github.com/fungs/taxator-tk/blob/master/USAGE.md

I've tried to run with a very small sample of my reads (around 100 reads). The error is still there, but now the tool finishes the analyses (although all reports are empty).

If taxator fails but the pipeline still exits with return status 0 (check shell variable $? after execution) then I would consider this a bug in the extended version pipeline script. Can you please confirm?

from taxator-tk.

Indeed, the problem was in the taxonomy. There were non-major ranks in my mapping.tax file. Following the instructions at https://github.com/fungs/taxator-tk/blob/master/USAGE.md I solved the problem by running:

taxknife -f 2 --mode traverse -r species genus family order class phylum superkingdom < mapping.tax > newmapping.tax

Now it works in single and multi-thread and with the full set of reads.
Thanks for the help.

from taxator-tk.

Glad it works, I'll leave this bug report open until the exception message is clarified and as a reminder for writing a short tutorial how to construct custom refpacks for the taxator-tk binary package.

from taxator-tk.

I meet this error a few days ago by running metaWRAP.
An unrecoverable error occurred: std::exception Here is some debugging information to locate the problem: /home/johdro/projects/taxator-tk_default.git/src/fileparser.hh(52): Throw in function FileParser::RecordType* FileParser::next() [with FactoryType = AlignmentRecordFactory; FileParser::RecordType = AlignmentRecordTaxonomy] Dynamic exception type: boost::exception_detail::clone_impl std::exception::what: std::exception [exception_tag_line*] = 9 [exception_tag_taxid*] = 1283331 [exception_tag_general*] = bad alignment reference taxon
I read the software to find out why. In the python script(bin/metawrap-scripts/prune_blast_hits.py), there is a line that filter many ranks(exclude=["no rank", "subspecies", "species group", "varietas", "forma", "subfamily", "cohort"]). I also don't know why it run with error.
Today I find your post, I think I found the answer.
Thanks.

from taxator-tk.