Let's assume you want to implement some new feature in moshpit. You would fork this repo and then clone the fork to your local machine where you would work on the new features. But before you start coding you need to set up the virtual environment. So you follow the instructions in the wiki and run the following (notice how moshpit is left out of the mamba create
command since you will install your local copy and not the version that is available through mamba).
mamba create -yn test_env \
-c conda-forge -c bioconda -c https://packages.qiime2.org/qiime2/2023.5/tested -c defaults \
q2cli q2-assembly q2-checkm
conda run -n test_env \
pip install --no-deps --force-reinstall git+https://github.com/misialq/quast.git@issue-230
conda activate test_env
cd <path_to_local_moshpit_repo>/q2-moshpit
pip install -e .
So far so good, but then running anything (e.g. qiime dev refresh-cache
) will return this error:
ImportError: cannot import name 'BrackenDBDirectoryFormat' from 'q2_types_genomics.kraken2'
This error comes from q2_types_genomics/kraken2/__init__.py
(and perhaps the other files in q2_types_genomics/kraken2
). The version that gets installed is missing some classes that are used by moshpit.
Installed version:
from ._format import (
Kraken2ReportFormat, Kraken2ReportDirectoryFormat,
Kraken2OutputFormat, Kraken2OutputDirectoryFormat,
Kraken2DBFormat, Kraken2DBDirectoryFormat
)
from ._type import Kraken2Reports, Kraken2Outputs, Kraken2DB
__all__ = [
'Kraken2ReportFormat', 'Kraken2ReportDirectoryFormat', 'Kraken2Reports',
'Kraken2OutputFormat', 'Kraken2OutputDirectoryFormat', 'Kraken2Outputs',
'Kraken2DBFormat', 'Kraken2DBDirectoryFormat', 'Kraken2DB'
]
However, these classes are available in the current version of q2-types-genomics
.
Current GitHub version:
from ._format import (
Kraken2ReportFormat, Kraken2ReportDirectoryFormat,
Kraken2OutputFormat, Kraken2OutputDirectoryFormat,
Kraken2DBFormat, Kraken2DBDirectoryFormat,
BrackenDBFormat, BrackenDBDirectoryFormat
)
from ._type import Kraken2Reports, Kraken2Outputs, Kraken2DB
__all__ = [
'Kraken2ReportFormat', 'Kraken2ReportDirectoryFormat', 'Kraken2Reports',
'Kraken2OutputFormat', 'Kraken2OutputDirectoryFormat', 'Kraken2Outputs',
'Kraken2DBFormat', 'Kraken2DBDirectoryFormat', 'Kraken2DB',
'BrackenDBFormat', 'BrackenDBDirectoryFormat'
]
link to lines of code in q2-types-genomics
So we can fix it by running the following.
pip uninstall q2_types_genomics
git clone [email protected]:bokulich-lab/q2-types-genomics.git
cd <path_to_local_q2-types-genomics_repo>/q2-types-genomics
pip install .
Ok problem solved! Sadly not, apparently q2-types-genomics
is not the only incompatible package. If we run qiime dev refresh-cache
we will get the following error.
TypeError: BLAST6 is not a variant of SampleData.field['type']
The error comes from the q2-types
package. It boils down to the q2_types/feature_data/_type.py
lines 51-53.
Old version (installed with conda/mamba):
BLAST6 = SemanticType('BLAST6',
variant_of=[FeatureData.field['type']])
New version (available in GitHub):
BLAST6 = SemanticType('BLAST6',
variant_of=[FeatureData.field['type'],
SampleData.field['type']])
link to lines of code in q2-types
Again we uninstall the guilty package and clone the new version and install it locally.
pip uninstall q2-types
git clone https://github.com/qiime2/q2-types.git
cd <path_to_local_q2-types>/q2-types
pip install .
Woohoo! We solved it! Almost. Running qiime dev refresh-cache
will return import errors, essentially complaining that some libraries are missing from the environment. Just
pip install xmltodict tqdm
(the libraries that are missing) and you are good to go.