This would be really helpful for alpha, and we should be able to do this using the webbrowser module in Python. I started on this in a local branch, and can finish this one up for alpha.

The codebase is growing and it would be helpful to group all command-related code (builtins and dynamic) under a q2cli.commands package to separate them from modules like cache, util, and handlers. This would be an invisible change to users, it's just code shuffling. Not high priority right now.

Originally posted here.

Artifact/Visualization saving code can likely be simplified. See qiime2/qiime2#135

So calling a workflow w from plugin p would look like qiime p w (where it currently looks like qiime p.w. This could open the door for plugin-specific context parameters.

e.g. commands in the QIIME 2 docs. Not sure if this is fixable or something we have to live with in bash.

The --type option in qiime tools import is free text and requires the user to know the relevant importable type expression ahead of time. It would be useful to display a list of types that are valid inputs to --type (i.e. concrete semantic types that are associated with a directory format).

The set of importable type expressions will be available from the framework via qiime2.sdk.PluginManager.importable_types once qiime2/qiime2#216 is merged.

There are a few routes I can think of to accomplish this:

• Make --type a click.Choice option instead of free text (see here for an example). This renders pretty terribly, especially with | being overloaded:

  

• Keep --type as free text and display choices in help text:

  

• Have an eager option (e.g. similar to --help and --version) that shows the choices and exits. Something like a --show-importable-types option.

• Have a separate command to display importable types.

@gregcaporaso, @ebolyen, @thermokarst, @jakereps: what do you guys think? Any preferences, or maybe a better way to solve this?

Before:

Re-run with --verbose to see debug info.

After:

To learn more about this error, re-run with --verbose (useful when asking for help).

It seems like it is more of an info command as it only side-effects stdout.

Basically calls qiime.sdk.Result.peek.

@ElDeveloper suggested this on the dev call today. Basically do what Sphinx does where stdout/stderr gets logged in a temp directory, unless --verbose is supplied. Having this feature would help with getting verbose output if the user doesn't supply --verbose and the error takes awhile to trigger.

Related to #6.

@ebolyen I'm using q2-feature-classifier as an example for this issue. I'm not sure if I've set it up properly to give a method parameter a default value, so please let me know if I haven't.

Method signature:

def classify(sequences : types.GeneratorType, reference_taxonomy : pd.Series,
        reference_sequences : types.GeneratorType, method : str,
        depth : int = 4) -> pd.Series:

Method registration:

plugin.methods.register_function(
    function=q2_feature_classifier.classify,
    inputs={'sequences' : FeatureData[Sequence],
        'reference_sequences' : ReferenceFeatures[SSU],
        'reference_taxonomy' : ReferenceFeatures[SSU]},
    parameters={'depth': Int,
                'method' : Str % Choices(['naive-bayes', 'svc', 'perfect'])},
    outputs=[('classification', FeatureData[Taxonomy])],
    name='Train and apply feature classifier.',
    description='Train a classifier and apply it to feature data.'
)

Unexpected behaviour:

$ wget https://github.com/qiime2/q2-types/raw/master/q2_types/tests/data/reference-features-ssu.qza
$ wget https://github.com/qiime2/q2-types/raw/master/q2_types/tests/data/feature-data-sequence.qza
$ qiime feature-classifier classify --i-reference-sequences reference-features-ssu.qza --i-reference-taxonomy reference-features-ssu.qza --i-sequences feature-data-sequence.qza --p-method perfect --o-classification classification.qza --p-depth 5
$ qiime feature-classifier classify --i-reference-sequences reference-features-ssu.qza --i-reference-taxonomy reference-features-ssu.qza --i-sequences feature-data-sequence.qza --p-method perfect --o-classification classification.qza
Usage: qiime feature-classifier classify [OPTIONS]

Error: Missing option "--p-depth".

So it fails when I don't give it the depth parameter.

Thanks!

We display the FW version, plus all the plugin versions. It would be useful to indicate the Release, too, so that it is easier for people to figure out what version of the docs they need to consult (since the docs are keyed on release, not version)

Click permits options to be repeated on the CLI and will always use the value of the last specified option (this is intentional, documented behavior). I can't find a way to disallow repeated options in Click but it should be possible to hack something together.

It seems like we'd want q2cli to distinguish between single and repeated options, especially once metadata merging (#140) is supported. Ideally we'll be able to make the CLI error on repeated options unless the option is metadata.

Related to qiime2/qiime2#269

@wasade points out that with a lot of plugins/workflows, building the commands is going to become a bottleneck.

Bug Description
In both of these cases, tab completion stops working after the multi-word value being passed to --input-path.

Screenshots

qiime tools import --input-path 'foo bar' --o-
qiime tools import --input-path "foo bar" --o-

Comments
(This came up in the context of qiime feature-table filter-samples --p-where, but these examples are simpler.)

See @ebolyen's comment. I'll take this one.

The current qiime --version flag has to import QIIME and displays Python, QIIME, and q2cli versions. Since QIIME is imported, the command is slow. This version info (and much more) are available by running qiime info, so it would be nice to use Click's builtin support for version flags to display something like:

$ qiime --version
q2cli 0.0.3
For more version details, run `qiime info`.

• -h is shortcut for --help
• calling subcommands (e.g., qiime tools extract or qiime diversity alpha) invokes help

We should check if uname is Darwin and if so, assume we have a windowed environment.

It'd be great to include a method/visualizer's name and description in the generated help text for the alpha release.

https://forum.qiime2.org/t/qiime-tools-view-on-headless-server/113

I was specifying an outputs in a plugin, and originally had [('foo-bar', Thing)], however, this resulted in a traceback from q2cli as it didn't fully munge the name. The command line argument needed was --o-foo-bar, q2cli was looking for o_foo-bar and throwing a KeyError. Removing the -, and using foobar seemed to unstick this. Full traceback below:

$ qiime humann2 humann2 --i-samples fmt-tutorial-per-sample-fastq.qza --o-gene-families fooa --o-path-abundance foob --o-path-coverage fooc
Traceback (most recent call last):
  File "/Users/mcdonadt/miniconda3/envs/humann2/bin/qiime", line 9, in <module>
    load_entry_point('q2cli', 'console_scripts', 'qiime')()
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/mcdonadt/ResearchWork/software/q2cli/q2cli/commands.py", line 137, in __call__
    outputs, missing_out = self.handle_out_params(kwargs)
  File "/Users/mcdonadt/ResearchWork/software/q2cli/q2cli/commands.py", line 196, in handle_out_params
    outputs.append(handler.get_value(kwargs, fallback=fallback))
  File "/Users/mcdonadt/ResearchWork/software/q2cli/q2cli/handlers.py", line 185, in get_value
    return self._locate_value(arguments, fallback)
  File "/Users/mcdonadt/ResearchWork/software/q2cli/q2cli/handlers.py", line 59, in _locate_value
    v = arguments[click_name]
KeyError: 'o_gene-families'

See biom-format for an example.

Comments
It'd be best to wait for the new provenance support @ebolyen is working on before adding this command.

e.g.:

(q2dev) ☁  moving-pictures  qiime tools peek cm1441/evenness-correlation.qzv
UUID:        e65966a0-a4a7-4490-9900-c8bd491edd85
Type:        Visualization
Data format: Traceback (most recent call last):
  File "/Users/matthew/miniconda3/envs/q2dev/bin/qiime", line 11, in <module>
    load_entry_point('q2cli', 'console_scripts', 'qiime')()
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/matthew/caporasolab/q2cli/q2cli/tools.py", line 80, in peek
    click.secho(metadata.format)
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/termui.py", line 420, in secho
    return echo(style(text, **styles), file=file, nl=nl, err=err, color=color)
  File "/Users/matthew/miniconda3/envs/q2dev/lib/python3.5/site-packages/click/termui.py", line 393, in style
    return ''.join(bits)
TypeError: sequence item 0: expected str instance, NoneType found

It's really slow right now.

This came up at the PHX workshop --- an option to echo the written paths (qzas and qzvs) to stdout would be more illuminating.

We should try to fit this in with the project URLs here:

Also, a statement after the one at the very bottom saying something like: "Also, please remember to cite plugins..."

This is really important for motivating developers to write plugins, and we'll want something like this in all of our interfaces.

Related to qiime2/qiime2#267

• tests of output-dir handling under different circumstances
• tests including parameters (currently only have input and output artifacts)

New linux users periodically end up with the em-dash in their commands, which breaks the script in a opaque way.

Would it be possible to throw an explicit and constructive error or warning any time an em-dash is detected in the input?

(Thank you to @ebolyen for the feedback!)

The help string for import indicates that --input-path can be a file or directory, however it appears only directories are allowed. This contradicts the import example on the wiki as well

$ qiime tools import --type "SampleData[SequencesWithQuality]" --input-path demo.fastq.gz --output-path demo-artifact
Traceback (most recent call last):
  File "/Users/mcdonadt/miniconda3/envs/humann2/bin/qiime", line 9, in <module>
    load_entry_point('q2cli', 'console_scripts', 'qiime')()
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mcdonadt/miniconda3/envs/humann2/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/mcdonadt/ResearchWork/software/q2cli/q2cli/tools.py", line 66, in import_data
    view_type=source_format)
  File "/Users/mcdonadt/ResearchWork/software/qiime2/qiime/sdk/result.py", line 191, in import_data
    return cls._from_view(type_, view, view_type, provenance)
  File "/Users/mcdonadt/ResearchWork/software/qiime2/qiime/sdk/result.py", line 214, in _from_view
    result = transformation(view)
  File "/Users/mcdonadt/ResearchWork/software/qiime2/qiime/core/transform.py", line 56, in transformation
    self.validate(view)
  File "/Users/mcdonadt/ResearchWork/software/qiime2/qiime/core/transform.py", line 113, in validate
    view.validate()
  File "/Users/mcdonadt/ResearchWork/software/qiime2/qiime/plugin/model/directory_format.py", line 163, in validate
    raise ValueError("%r is not a directory." % self.path)
ValueError: InPath('demo.fastq.gz') is not a directory.

Output handling code can likely be simplified. See qiime2/qiime2#131

with option(s) to override this behavior.

Traceback text can be confusing to users because it holds a lot of irrelevant info (from a user's perspective) and can make finding the actual error message difficult. It'd be helpful to 1) only show the error message in red, and 2) show the traceback if --verbose is supplied.

This was recently added to the framework: qiime2/qiime2#116

This came up during some testing I did for the Iceland workshop. If a QIIME 2 plugin fails to load when the cache is being refreshed (and written to disk), the requirements.txt file will be created but state.json will not. Thus, subsequent q2cli commands will fail to detect that a cache refresh is necessary, and installed plugins will not be visible from the CLI.

When I was testing, a plugin relying on matplotlib failed to load due to a matplotlib ImportError (the PyQT conda package bug we've all ran into by now). I believe @nbokulich faced a similar issue recently.

For now, this is easily worked around by fixing the offending plugin's installation error and running qiime dev refresh-cache (or just delete your cache directory).

This is an easy fix, assigning to myself.

In general, the output information about what artifacts are created is very useful (and I really like clean use of colors). However, there are times when it would be nice to not have this output (e.g., scripting). This could be exposed, similar to the --verbose flag as a --silent flag, in which output normally directed to stdout is not shown.

The idea of silencing output and/or increasing verbosity has come up a few times.

Usage: qiime diversity pcoa pcoa [OPTIONS]

Not sure why this is happening.

This is helpful now that the extracted root directory is a UUID, and there could be other extracted UUID directories already, making it hard to find the artifact/visualization you extracted.