Forked from a discussion on #30.

Which lets me notice an additional issue: napoleon indicates that varargs should be documented using

Parameters
----------
*args : ...

i.e. including the star, unescaped (and similarly for google-style), but defopt only recognizes the parameter name without the star).

I'm not sure what the official advice is for Sphinx-style docstrings - I write it as args because PyCharm won't accept it any other way.

If we can find consistent advice across formats, we should stick with that. If not, we'll have to handle both.

This may be easy with Napoleon.

import defopt
def test(*, a):
    """:param str a: test"""
defopt.run(test, argv=['-h'])

usage: do.py [-h] a

positional arguments:
  a           test

optional arguments:
  -h, --help  show this help message and exit

This was an oversight, but my actual words from the documentation are "any optional arguments are converted to flags", so this is presently working as documented, even if not as intended. This means a fix should come with a major version bump.

import defopt
import typing
def foo(*bar: typing.List[int]):
    print(bar)
defopt.run(foo)
# ./test.py --bar 1 2
# (1, 2)

Should either fail or be handled in some sensible way.

I'm getting an exception when running a function with a keyword-only private argument. In this case the arg is called "_sh".

File ".../defopt.py", line 352, in _call_function
arg = getattr(args, name)
AttributeError: 'Namespace' object has no attribute '_sh'

In the code this is what's happening:

    347 def _call_function(func, args):
    348     positionals = []
    349     keywords = {}
    350     sig = _inspect_signature(func)
    351     for name, param in sig.parameters.items():
--> 352         arg = getattr(args, name)
    353         if param.kind in [param.POSITIONAL_ONLY, param.POSITIONAL_OR_KEYWORD]:
    354             positionals.append(arg)
    355         elif param.kind == param.VAR_POSITIONAL:
    356             positionals.extend(arg)
    357         else:

name = '_sh'
param = <Parameter "_sh=<class 'sh.Command'>">
args = Namespace(_func=<function git_repo_info at 0x10443de18>, raw=False, remote='origin', verbose=True, working_dir='.')

So the default value for the argument _sh is present in the value of theparam object, but it's not being picked up. It seems that what should happen is to check if the param has a default value and change line 352 to

arg = getattr(args, name, param.default)

It seems like right now, the best way to achieve something like this (while also ensuring the default shown in the help string is correct) is def main(*, nums: List[int] = [1, 2, 3]), which isn't ideal for its use of a mutable default argument. Maybe adding support for variable-length tuples (e.g., def main(*, nums: Tuple[int, ...] = (1, 2, 3)) would solve this.

This looks like a very promising library for simplifying command line argument handling. However I have a use case which may not be handled. I read the documentation but am not sure if it is possible or not.

I was wondering how to go about calling an intermediate function (named facade) in a library that uses some arguments and then passes back to the project's main function?

eg.

def main(hdx_site: str = "prod", output_failures: bool = False):
    """Main function

    Args:
        hdx_site (str): HDX site
        output_failures (bool): Whetehr to output failures

    Returns:
        None
    """
    print(f"I only use output_failures which is {output_failures}")

if __name__ == "__main__":
    defopt.run(main)

The library has the intermediate function, facade:

def facade(projectmainfn: Callable[[Any], None], **kwargs: Any):
    print(f"I do some setup and use hdx_site which is {hdx_site} then I call the project's main")
    projectmainfn(**kwargs)    

The defopt.run line needs to use main to get the allowed command line arguments, but call facade passing main and the the parsed keyword arguments as parameters to facade. facade does some stuff using some of those arguments then calls main with them (although not all are actually used by main).

Is this possible somehow?

I'd like to propose that

@defopt.run
def main(x: Tuple[int, str]): ...

creates an argument, x, with nargs=2, with each argument converted using the correct type (in the function, x would indeed be a tuple), and that

@defopt.run
def main(x: NamedTuple('T', [('foo', int), ('bar', str)])): ...

does the same but additionally uses (foo, bar) as the metavar for x (specifically, this would check that the annotation is a subclass of tuple and defines __annotation__). (And likewise, when the function is actually called, x would be of type T.)

I've noticed that certain numpy-style docstrings that contain either a Note or Notes sections cause an exception to occur during CLI parsing. In the following example (largely copied from https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html), I've taken their PEP484 example and added a single Notes section from another example on that page. The exception is posted below. If you simply rename the Notes section to be called any other word, things appear to work fine.

#!/usr/bin/env python
import defopt


def run(param1: int, param2: str) -> bool:
    """Example function with PEP 484 type annotations.

    The return type must be duplicated in the docstring to comply
    with the NumPy docstring style.

    Parameters
    ----------
    param1
        The first parameter.
    param2
        The second parameter.

    Returns
    -------
    bool
        True if successful, False otherwise.

    Notes
    -----
    Do not include the `self` parameter in the ``Parameters`` section.

    """


if __name__ == "__main__":
    defopt.run(run)

$ ./test_defopt.py -h
Traceback (most recent call last):
  File "./test_defopt.py", line 31, in <module>
    defopt.run(run)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/defopt.py", line 96, in run
    parser = _create_parser(funcs, **kwargs)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/defopt.py", line 129, in _create_parser
    _populate_parser(funcs, parser, parsers, short, strict_kwonly)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/defopt.py", line 179, in _populate_parser
    doc = _parse_function_docstring(func)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/defopt.py", line 362, in _parse_function_docstring
    return _parse_docstring(inspect.getdoc(func))
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/defopt.py", line 514, in _parse_docstring
    tree.walkabout(visitor)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/docutils/nodes.py", line 178, in walkabout
    if child.walkabout(visitor):
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/docutils/nodes.py", line 170, in walkabout
    visitor.dispatch_visit(self)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/docutils/nodes.py", line 1912, in dispatch_visit
    return method(node)
  File "~/miniconda3/envs/testenv/lib/python3.7/site-packages/docutils/nodes.py", line 1937, in unknown_visit
    % (self.__class__, node.__class__.__name__))
NotImplementedError: <class 'defopt._parse_docstring.<locals>.Visitor'> visiting unknown node type: rubric

All of my commands have some common setup and teardown. One paradigm for this is to have a class with an __init__() and __del__() method, and then have each command by a method of that class. fire supports this, for example. I don't know how difficult this would be, but it would save me a lot of repetition if I could pass a class to defopt.run() instead of a list of functions.

http://www.sphinx-doc.org/en/stable/domains.html#info-field-lists

param, parameter, arg, argument, key, keyword: Description of a parameter.

Currently only param is accepted.

Currently, bullet and enumerated lists in a docstring get deleted in the rendered help. This is because they have an element.tag set to bullet_list and enumerated_list respectively, whereas parse_docstring only handles paragraph and literal_block and silently discards everything else. It should be "relatively" easy to render such xml elements correctly.
More generally, I would also suggest triggering a warning when an xml element gets discarded.

(This relates to the work currently in progress for #13.)

import defopt
def foo(*, bar: int):
    """:param bar: baz"""
    print(bar)
defopt.run(foo)
# ./test.py -h
# usage: test.py [-h] --bar BAR
#
# optional arguments:
#   -h, --help  show this help message and exit
#   --bar BAR   baz (default: None)

These should be suppressed. This should either be done with argparse.SUPPRESS, or with a change to the formatter or argparse itself (since a default for a required argument makes no sense).

A quick suggestion: render bold/italic markup in rst docstrings in the terminal using e.g. colorama, or blessings (which can do underline/italics).

Specific feature request:

I wonder if you could support composition without adding too much complexity. A lot of my CLIs take an optional config argument which takes a path to a YAML file (defaults to ./config.yaml) and loads that YAML file and initializes a logger. I'd love to be able to re-use that code and simply compose it with other scripts. I don't know if that would be possible without sacrificing simplicity though.

Source

This is within the realm of possibility but I'm looking for a clean design. Suggestions are welcome.

Currently, keyword arguments starting with an underscore break defopt. With

@defopt.run
def main(_x: str = "foo"):
    pass

we get

$ python /tmp/foo.py -h
usage: foo.py [-h] [-_ X]

optional arguments:
  -h, --help    show this help message and exit
  -_ X, ---x X

$ python /tmp/foo.py -_ 1
Traceback (most recent call last):
  File "/tmp/foo.py", line 4, in <module>
    def main(_x: str = "foo"):
  File "/usr/lib/python3.6/site-packages/defopt.py", line 101, in run
    return _call_function(args._func, args)
  File "/usr/lib/python3.6/site-packages/defopt.py", line 319, in _call_function
    arg = getattr(args, name)
AttributeError: 'Namespace' object has no attribute '_x'

$ python /tmp/foo.py ---x 1
Traceback (most recent call last):
  File "/tmp/foo.py", line 4, in <module>
    def main(_x: str = "foo"):
  File "/usr/lib/python3.6/site-packages/defopt.py", line 101, in run
    return _call_function(args._func, args)
  File "/usr/lib/python3.6/site-packages/defopt.py", line 319, in _call_function
    arg = getattr(args, name)
AttributeError: 'Namespace' object has no attribute '_x'

(and similarly when there are multiple such arguments).

I think it makes sense to treat such arguments as private and not expose them to the CLI.

I am getting the following error: Unknown directive type "attribute" and have created a minimal example to debug. Any thoughts on how I should debug?

python script.py -h
<string>:3: (ERROR/3) Unknown directive type "attribute".

.. attribute:: name

   the name

<string>:7: (ERROR/3) Unknown directive type "attribute".

.. attribute:: other

   the other
usage: script.py [-h] -w WRAPPER

Some example

optional arguments:
  -h, --help            show this help message and exit
  -w WRAPPER, --wrapper WRAPPER
                        a Wrapper string

import enum
import defopt


@enum.unique
class Other(enum.Enum):
    Foo = "foo"
    Bar = "bar"


class Wrapper:
    """Some Wrapper

    Attributes:
        name: the name
        other: the other
    """

    def __init__(self, value: str) -> None:
        name, other = value.split(':', maxsplit=1)
        self.name: str = name
        self.other: Other = Other(other) 


def main(*, wrapper: Wrapper) -> None:
    """Some example

    Args:
        wrapper: a Wrapper string
    """

    print(wrapper)


if __name__ == '__main__':
    defopt.run(main)

Functions may optionally specify exceptions they raise:

def foo():
    """A function that raises an exception.

    :raises FooError: if something goes wrong
    """

It may be desirable to suppress the traceback in this case and only display the exception text itself. The docstring text could also be displayed, although the language may end up being a bit forced. Something like "This error is raised [text]" to make "This error is raised if something goes wrong".

It sometimes makes sense for an argument to be optional but still specified by position, rather than by an explicit --option (e.g. the second argument to ln). In argparse parlance, this is nargs="?".

A reasonable way to specify such arguments could be e.g.

@defopt.run
def main(x, y=None, *, z=...): ...

i.e. POSITIONAL_OR_KEYWORD arguments that have a default are treated as nargs="?" arguments if the function also defines KEYWORD_ONLY arguments (which would still be treated as --options). If you don't need any --option, you could always use a dummy argument, e.g.

@defopt.run
def main(x, y=None, *, _): ...

(once #34 is fixed, the dummy argument would disappear).

A more radical approach would be to say that only KEYWORD_ONLY arguments are --options, and everything else is positional, but that's probably not viable as long as you want to support Py2 :-) (Yes, the current proposal does not allow creating nargs="?" arguments under Py2 either, but what can you do...)

I would like to support parsing command line options of the type: Optional[List[str]].

import defopt
  
from typing import List
from typing import Optional
import json

def foo(*, params: Optional[List[str]] = None) -> None:
    print(type(params))


if __name__ == '__main__':
    defopt.run(foo)

which gives:

ValueError: unsupported union including container type: typing.Union[typing.List[str], NoneType]

I want to be able to differentiate between an empty list (empty list of values) and None (no list given). This also allows the function being run to be used by other methods as intended. I do not like defaulting it to an empty list either, due to the above as well as it being mutable!

This would be convenient for further customization. Additional points if it takes the ArgumentParser object as first argument (it can always directly set the formatter_class attribute itself).

defopt._call_function would also need to be public so that we can actually make use of the result of the argument-line parsing of course.

It would be nice if types given in the docstring that rely on members of the typing module can be evaluated without having to actually import them from typing (this creates seemingly unused imports, which is both noisy, and not appreciated by linters).

If I declare a positional argument and a list keyword argument:

def run(a: str, *, b: List[str]):
    """
    :param a: a positional field
    :param b: a keyword list field
    """"
    print('a:', a)
    print('b:', b)

Then the help text produced by defopt looks like:
usage: testlist.py [-h] -b [B [B ...]] a

But trying to pass arguments that way, ex. testlist.py -b bar baz foo, doesn't work - the positional argument needs to go first, or it looks like it's part of the list b. Shouldn't the help text look more like:
usage: testlist.py a [-h] -b [B [B ...]]
so that it's more clear the positional argument goes first?

While I can't see a good way of passing arbitrary keyword arguments, users should absolutely be able to pass documented keyword arguments. defopt.run is itself an example of a function that accepts **kwargs for Python 2 compatibility but expects at most argv to be specified.

First let me say I'm a big fan of defopt. I'd like to suggest a new feature related to #7 but simpler. defopt is designed to allow functions to be used as library functions from Python or as CLI programs. To incorporate logging for both cases requires some care. I'd like to suggest some small enhancements that would handle most of it.

1. Have defopt do

if not logging.getLogger().handlers:
  logging.getLogger().addHandler(logging.NullHandler(logging.WARN))

during module loading, as having some null handler avoids generating error messages from logging calls in libraries.

2. Add kwargs to defopt.run() along the lines of:

log_level=logging.WARN
log_file=None
log_handler=None

for specifying those things from the main block to take effect only when functions are run from the defopt CLI.

3. Add a switch argument to defopt.run() like log_args=False that could be set to enable a standard set of CLI args for logging config as if the function had the following in its argument list:

:param str log_level: choice of standard levels 
:param str log_file: path to log file, or special constants 'stderr', 'stdout'

If this were added to defopt then fairly complete logging configuration would be available with minimal effort and w/o compromising the promise that defopt won't interfere with using your functions from Python.

from typing import Optional

def main(*, value: Optional[int] = 2) -> None:
    print(f"{value}")

if __name__ == '__main__':
    defopt.run(main)

I'd like to run python example.py -v None to set the value to None.

argparse natively supports displaying default values. Since we also know the type, it might make sense to add that information too, something like this:

def foo(bar=baz):
    """:param str bar: description'""

FOO    (str) description (default: baz)

MWE

from dataclasses import dataclass

import defopt


def main(
    args: list[str],
):
    pass


def ok(
    *args: str,
):
    pass


@dataclass
class Main:
    args: list[str]


@dataclass
class Ok:
    arg: str


if __name__ == "__main__":
    defopt.run(
        (main, ok, Main, Ok),
        strict_kwonly=False,
    )

resulted in

❯ python example.py ok -h                          
usage: example.py ok [-h] [args ...]

positional arguments:
  args

optional arguments:
  -h, --help  show this help message and exit
❯ python example.py main -h
usage: example.py main [-h] -a [ARGS ...]

optional arguments:
  -h, --help            show this help message and exit
  -a [ARGS ...], --args [ARGS ...]
❯ python example.py Main -h
usage: example.py Main [-h] -a [ARGS ...]

Main(args: list)

optional arguments:
  -h, --help            show this help message and exit
  -a [ARGS ...], --args [ARGS ...]
❯ python example.py Ok -h  
usage: example.py Ok [-h] arg

Ok(arg: str)

positional arguments:
  arg

optional arguments:
  -h, --help  show this help message and exit

Notes

The feature request is to support main(args: list[TypeX], *, ...) to be equivalent in terms of defopt to main(*args: TypeX, ...).

In the function case, main, the user could have written it as the function ok instead.

In the dataclass case however, since Python's dataclass doesn't support variable positional arguments, one cannot defines *args. So the only sensible choice here is to define args: list instead.

Options already implicitly replace underscores in their names by dashes in the CLI. For consistency, it would be appreciated if subcommands did the same. The options I see are

• break backcompatibilty and make this nonconfigurable.
• make this configurable (seems a bit overkill and inconsistent with the behavior for flags)
• hack into add_subparsers to map both ./foo.py bar_baz and ./foo.py bar-baz to the same function, preferably while displaying only one of them in the help string.

Function annotations are officially for type hints (https://www.python.org/dev/peps/pep-0484/). These should be supported.

Thoughts:

• Type annotations should at least override the docstring, and probably error if conflicting types are specified
• Some default translations may need to be applied, e.g. Sequence to list
• May want to support the suggested syntax for Python 2 compatibility (https://www.python.org/dev/peps/pep-0484/#suggested-syntax-for-python-2-7-and-straddling-code), although this isn't really necessary since the docstring will always work

defopt is easily my favorite of the many argument parsers for Python and I know it has a goal to stay fairly small. That said, nested subcommands is one feature that I'd really like to have. Subcommands are already supported, but sometimes a second layer can be useful. Perhaps recursion can be used in the default parser.

I could see the definition of it being something like:

defopt.run([sub1, {'sub2': [subsub1, subsub2]}])

Where these could then be called like

program sub1 --help
program sub2 --help  # not sure where this help would come from though
program sub2 subsub1 --help
program sub2 subsub2 --help

Currently I just use one subcommand depth and use dashes, like this:

defopt.run([sub1, sub2_subsub1, sub2_subsub2])

Which can be used like:

program sub1 --help
program sub2-subsub1 --help
program sub2-subsub2 --help

Consider the code below. I'd like to have the argument skip still act as a normal boolean flag and then program would only get None if neither --skip nor --no-skip is provided. The idea here is that if the user explicitly sets this flag, then it should be followed. But if they don't set the flag explicitly (so it is None), then the CLI would have some logic to decide if the default value would be True or False, potentially depending on some other input, like loops in this dummy example.

from typing import Optional

import defopt


def test(*, loops: int = 5, skip: Optional[bool] = None):
    """This is a dummy function.

    Parameters
    ----------
    skip
        Skip printing something.
    """
    print(f'loops = {loops}')
    print(f'skip  = {skip}')
    _skip = skip if skip is not None else loops > 10
    if _skip:
        print(f'skipping loop printing')
    for i in range(loops):
        if not _skip:
            print(f'long loop iteration: {i+1}')

if __name__ == '__main__':
    defopt.run(test)

But this is now recognized as a variable input instead of a flag.

$ python dummy.py --help
usage: dummy.py [-h] [-l LOOPS] [-s SKIP]

This is a dummy function.

optional arguments:
  -h, --help            show this help message and exit
  -l LOOPS, --loops LOOPS
                        (default: 5)
  -s SKIP, --skip SKIP  Skip printing something.
                        (default: None)
nox > Session dummy was successful.

So specifying nothing works:

$ python dummy.py
loops = 5
skip  = None
long loop iteration: 1
long loop iteration: 2
long loop iteration: 3
long loop iteration: 4
long loop iteration: 5

Specifying 0 or 1 as the input works:

$ python dummy.py --skip 1
loops = 5
skip  = True
skipping loop printing
$ python dummy.py --skip 0
loops = 5
skip  = False
long loop iteration: 1
long loop iteration: 2
long loop iteration: 3
long loop iteration: 4
long loop iteration: 

But of course using just --skip alone (or --no-skip) doesn't work. Note also that, as far as I can tell, you can't manually specify the value of None either:

$ python dummy.py --skip None
usage: dummy.py [-h] [-l LOOPS] [-s SKIP]
dummy.py: error: argument -s/--skip: invalid typing.Optional[bool] value: 'None'

What I would like, is for this to be treated as a flag still. This would be a breaking change since now using the example of python dummy.py --skip 1 would no longer work as instead it would just be python dummy.py --skip or python dummy.py --no-skip. I personally can't imagine any scenario in which using --skip 1 and --skip 0 is better than --skip and --no-skip.

A PR to consider for this is incoming.

It would be nice if keyword arguments which do not share their initial with any other keyword argument could automatically provide a short form, i.e.

def main(foo: str = "foo", bar: str = "bar"): ...

results in the options -f, --foo, -b, --bar.

Basically the only thing that I miss from click, argh, clize, and other cli frameworks is the ability to rename and/or alias function names with friendlier cli names. Alas, those competitors are otherwise far more annoying to use than defopt.

Specifically, it would be great to have functionality like this, or how clize accepts a dict of {cli_name:original_function_name} pairs, available in defopt.

Are there any plans to implement something similar?

(Full disclosure: I haven't looked through defopt's code enough to know how hard it might be to do so)

As it conflicts with the autogenerated -h/--help option. Here's a contrived minimal example, and the resulting traceback

import defopt
def foo(*, hello: str = "Hi!"):
    print(hello)
defopt.run(foo, argv=['-h'])

Traceback (most recent call last):
  File "foo.py", line 8, in <module>
    defopt.run(foo, argv=['-h'])
  File "/usr/local/lib/python3.7/site-packages/defopt.py", line 94, in run
    parser = _create_parser(funcs, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/defopt.py", line 116, in _create_parser
    _populate_parser(funcs, parser, parsers, short, strict_kwonly)
  File "/usr/local/lib/python3.7/site-packages/defopt.py", line 257, in _populate_parser
    _add_argument(parser, name, short, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/defopt.py", line 268, in _add_argument
    return parser.add_argument(*args, **kwargs)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1367, in add_argument
    return self._add_action(action)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1730, in _add_action
    self._optionals._add_action(action)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1571, in _add_action
    action = super(_ArgumentGroup, self)._add_action(action)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1381, in _add_action
    self._check_conflict(action)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1520, in _check_conflict
    conflict_handler(action, confl_optionals)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1529, in _handle_conflict_error
    raise ArgumentError(action, message % conflict_string)
argparse.ArgumentError: argument -h/--hello: conflicting option string: -h

A temporary workaround is to disable the automatic creation of the corresponding short argument: defopt.run(foo, short={"hello": "hello"}, argv=['-h']) but that's kludgy.

Would you consider exposing a bit more of defopt's functionality as public API? Specifically, I had a old (but much used) piece of code which extracted global options from environment variables, using something of the form

@environ_options
def get_option(
    THEFOO: "doc for thefoo" = default_foo,
    THEBAR: "doc for thebar" = default_bar,
):
    pass

which would create a get_option function, allowing one to do get_option("THEFOO") which reads the THEFOO environment variable, converts it using default_foo's type, and defaults, well to default_foo.
This design (in particular, storing the docs in the annotations and using the default's type as type hints) was basically copied from https://pypi.org/project/argh/, which is the CLI helper I was using at that time.

Obviously, this is ripe for rewriting using defopt's approach of parsing the docstring. After doing so using defopt's private API, I think defopt really just needs to expose a single additional API, get_cli_signature(func, parsers) (or however you want to call it), which would 1) call _parse_function_docstring; 2) additionally fill in types extracted from the docstring (so "_Doc" wouldn't be a suitable name for the object it returns -- it contains info from both the signature and the docstring); and 3) add an additional parser entry to the each _Param object.

Thoughts?

While individual paragraphs should continue to line-wrap as per the argparse default, it would be nice to retain paragraph breaks that are present in the original docstring. Additionally, literal blocks should be included and immune from line wrapping.

It would be cool to include a tool that can run any suitably documented function:

$ defopt package.module.function [args]

This will could work well with #9 and https://github.com/python/typeshed (although a lot of the types might be too generic to sensibly handle).

Possible example:

$ defopt round 1.234 --ndigits 2
1.23

Looks like the fallout of PEP560 (in particular) caused real breakage of the test suite.

When installing latest defopt 6.1.0 (and any version 6.*) we have started to see

Collecting defopt (from -r requirements.txt (line 4))
  Downloading https://pypi.counsyl.com/root/pypi/%2Bf/cfe/6ecfb54b1368a/defopt-6.1.0.tar.gz
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-q7m_amiq/defopt/setup.py", line 50, in <module>
        keywords='argument parser parsing optparse argparse getopt docopt sphinx',
      File "/var/lib/go-agent/pipelines/make-ci-10/code/env/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "/var/lib/go-agent/pipelines/make-ci-10/code/env/lib/python3.6/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/var/lib/go-agent/pipelines/make-ci-10/code/env/lib/python3.6/site-packages/setuptools/dist.py", line 444, in __init__
        k: v for k, v in attrs.items()
      File "/var/lib/go-agent/pipelines/make-ci-10/code/env/lib/python3.6/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/var/lib/go-agent/pipelines/make-ci-10/code/env/lib/python3.6/site-packages/setuptools/dist.py", line 732, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/pip-install-q7m_amiq/defopt/.eggs/setuptools_scm-6.1.0-py3.6.egg/setuptools_scm/integration.py", line 26, in version_keyword
        dist.metadata.version = _get_version(config)
      File "/tmp/pip-install-q7m_amiq/defopt/.eggs/setuptools_scm-6.1.0-py3.6.egg/setuptools_scm/__init__.py", line 192, in _get_version
        template=config.write_to_template,
      File "/tmp/pip-install-q7m_amiq/defopt/.eggs/setuptools_scm-6.1.0-py3.6.egg/setuptools_scm/__init__.py", line 94, in dump_version
        version_fields = parsed_version.release
    AttributeError: 'Version' object has no attribute 'release'
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-q7m_amiq/defopt/

The installation command is pip install --no-deps -r .requirements.txt with latest defopt in this requirements file.

Pinning version to lower 5.1.0 resolves this issue, but unfortunately we would like to use latest features.

defopt parsers are a bit annoying to reuse across modules: quite often I introduce a "pseudo-type" solely for the purposes of making a defopt annotation (e.g., comma_separated_int_list with a parser that's def parse_comma_separated_int_list(s): return [*map(int, s.split(","))] if s else [] (*)), but then to reuse the parser in another module I need to import the type annotation and the parser (well, I can cheat by making the type and the parser the same object...) and also pass it explicitly to the defopt.main call.

((*) not just using List[int] because I want List[comma_separated_int_list], in fact...)

It may be useful if it was possible to register parsers with defopt, e.g.
custom_type.py

class foo: ...

@defopt.register_parser(foo)
def parse_foo(s): ...

and then
calling_module.py

from custom_type import foo

def main(arg: foo): ...

defopt.run(main)  # no need to pass foo as parser.

Alternatively, to at least handle the case where the type annotation and the parser are the same object (i.e., an object of class foo can be constructed just from foo(s: str)), when defopt sees an unknown type foo, it could e.g. check whether the signature of foo consists of a single annotated-as-str parameter, and if so, assume that foo is indeed its own parser.
Thoughts?

• Defaults should appear on all arguments - even those without help text
• Defaults should not appear next to variable positional arguments

flags are CLI parameters that take no argument. they usually have an inversion (--no-<name> or +g)

i guess #2 is about customization and short flags/options in general. this is about the automatic version of flag features.

flags with an automatic --no-<name> version would be cool.

import defopt

def my_cmd(legend=True):
    print(legend)

if __name__ == '__main__':
    defopt.run(my_cmd)

$ python my_cmd.py --legend
True
$ python my_cmd.py --no-legend
False

once short options/flags (#2) are there, an automatic flag inversion using +g would also be cool:

$ python my_cmd.py +g
False

This is really simple to implement, but may be tricky to handle if using subcommands that can potentially return different things.

See https://docs.python.org/3/library/argparse.html#action:

'append' - This stores a list, and appends each argument value to the list. This is useful to allow an option to be specified multiple times. Example usage:

>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='append')
>>> parser.parse_args('--foo 1 --foo 2'.split())
Namespace(foo=['1', '2'])

This feature request is to add an option to defopt.run s.t. parser.add_argument(..., action='append') behavior can be toggled.

Currently defopt allows for no customization of the command line. There are some things that could be made more flexible without violating the spirit of defopt if implemented well.

Possibilities:

• Short flags, e.g. -c/--count
• "store_const" flags, e.g. --thing/--not-thing
• Others?

One potential solution is to parse this information out of RST comments so the generated documentation is not polluted.

def func(count=1, up=False):
    """Do something

    :param int count: desc
    :param bool up: desc

    .. count: -c/--count COUNT
    .. up: --up=True --down=False
    """

docutils is unable to correctly parse the following docstring:

    """Example function with :py:class:`enum.Enum` arguments.

    :param Choice arg: Choice to display
    :param Choice opt: Optional choice to display
    """

This produces the following help message:

<string>:1: (ERROR/3) Unknown interpreted text role "py:class".
usage: choices.py [-h] [--opt {one,two,three}] {one,two,three}

Example function with

positional arguments:
  {one,two,three}       Choice to display

optional arguments:
  -h, --help            show this help message and exit
  --opt {one,two,three}
                        Optional choice to display

While the parameters were correctly parsed, the parser stopped as soon as it hit the :py:class:directive.

This problem likely applies to any other markup specific to a Sphinx extension.

The fix should be relatively trivial, except for the need to handle versions of python where pathlib is not present.

The rationale is that paths are quite widely used as command line arguments, of course.