bluegenes / makemytranscriptome Goto Github PK

update kegg scripts to get (& map to) arbitrary list of KEGG maps
re-integrate into main pipeline
grab KO from annotation table, not simple conversion file
optional user-specified KO map numbers?

check memory usage of random_subset.py; reduce if necessary

Continue to annotate full Trinity transcriptome, but also provide annotation info for the subset of contigs called "good" by transrate: --> just need to grep relevant contig info out of annotation table (or print the subset as part of annotation table script)

*use general database conversion file (ID: Name) to add the 'Name' / stitle field to the blast tabular output from diamond. not necessary if using blast+, as this output is already supported

• join assessment tools into simple virtual pipeline

• make input groups for argparse input options
• hide advanced options from initial help message?

allow user input of pre-downloaded databases

• check best reciprocal blast script with "closest" blast; add to pipeline

move NR "best words" /query coverage calculation --> main annotation table script
calculate query coverage = (alignment length/ query length)
- alignment length = index [3] in the blast file
- query length = from input fasta
- "name" /words can come from index 12 - Tessa will extend the diamond blast output to contain this field

Update masterlog to include parameters for each run

• use bioboxes?

• enable users to provide CSV input when using the assembly tool and quality tool (in addition to "full" and "expression")

(useful for quality tool b/c translate assesses assemblies based on their input reads, if given)

integrate diamond as default in annotator.py

write up the tasks

• edit to include BUSCO & Transrate output
• wrap + integrate into pipeline

• print GFF3 output from annotation_table

When executing manage_tools, it should direct stdout to console not capture and hide it

reduce size footprint

describe how to use database config files

For blastP outputs + parsing (swissprot blastP; PFAM):

• handle cds. on ID as possibility (appears with diamond-blast, not with blast+)
  • if 'cds.' in string: cut it

Download BUSCO Metazoa by default
wget http://buscos.ezlab.org/files/metazoa_buscos.tar.gz
tar zxf metazoa_buscos.tar.gz

• command is already wrapped, but needs to be integrated into pipeline

first, need to check that this version works on ni

change: --JM ---> --max_memory
remove: --bflyopts "-V10 stderr"

Modifications for initial install:
create folder structure if it doesn't exist

Known modules needed to be fixed:
functions_general
manage_tools

remove reliance on Trinity gene-trans map for assembly:

1. allow no gene-trans map (optional input into all downstream steps)
2. allow optional gene-trans-map input from user.

Add PFAM db download:
wget https://data.broadinstitute.org/Trinity/Trinotate_v2.0_RESOURCES/Pfam-A.hmm.gz
gunzip Pfam-A.hmm.gz
hmmpress Pfam-A.hmm

Refactor databases to support user configurable install locations and user defined read locations

get --email working again

current gen_sample_info (in mmt) parses the csv --> generate this sample info differently to allow excel inputs

modify handle_cv function: use pandas to allow excel input

make installation/setup for home-brew & linuxbrew

chmod commands don't normally make new targets meaning they don't work well with the supervisor interface. Needs a fix.
Potential fix : Just chmod using subprocess instead of Task system

The transrate setup task installs dependendencies for transrate, meaning it doesn't have any obvious targets. This results in the task getting skipped as the declared targets are created by downloading transrate, not by running the setup command.

• general cleanup

make the following optional:

signalP
TMHMM
RNAMMER

the license associated with these programs means they can only be used by academic users

Low priority, but imp:

1. change default behavior of manage_databases:
   • change default functionality--> check available databases, install any missing db's
   • make "--update" an option instead (currently default)
2. make manage_db a module, so can call with:
   "mmt databases" instead of having to use ./scripts/manage_databases.py
3. option for which busco db to download (to go with our current option in pipeline.py)

1. in manage_databases:
   • use fastaID2Names to get a tabular lookup dictionary for each fasta
2. In annotator:
   • addStitleToBlastTab.py
     • takes in output of fastaID2Names + diamond blast output; modifies blast output in-place