Git Product home page Git Product logo

pyment's Introduction

pyment

Create, update or convert docstrings in existing Python files, managing several styles.

Test Status

Linux: Linux tests (TravisCI)

Windows: Windows tests (Appveyor)


Supported Versions

Supports Python36 Supports Python37 Supports Python38 Supports Python39

Code Coverage

Test coverage (Coveralls)

This Python3 program intends to help Python programmers to enhance inside code documentation using docstrings. It is useful for code not well documented, or code without docstrings, or some not yet or partially documented code, or a mix of all of this :-) It can be helpful also to harmonize or change a project docstring style format.

It will parse one or several python scripts and retrieve existing docstrings. Then, for all found functions/methods/classes, it will generate formatted docstrings with parameters, default values,...

At the end, patches can be generated for each file. Then, man can apply the patches to the initial scripts. It is also possible to update the files directly without generating patches, or to output on stdout. It is also possible to generate the python file with the new docstrings, or to retrieve only the docstrings...

Currently, the managed styles in input/output are javadoc, one variant of reST (re-Structured Text, used by Sphinx), numpydoc, google docstrings, groups (other grouped style).

You can also configure some settings via the command line or a configuration file.

To get further information please refer to the documentation.

The tool, at the time, offer to generate patches or get a list of the new docstrings (created or converted).

You can contact the developer dadel by opening a discussion.

  • install from Pypi
$ pip install pyment
  • install from sources:
$ pip install git+https://github.com/dadadel/pyment.git
or
$ git clone https://github.com/dadadel/pyment.git
$ cd pyment
$ python setup.py install
  • run from the command line:
$ pyment  myfile.py    # will generate a patch
$ pyment -w myfile.py  # will overwrite the file

or

$ pyment  my/folder/
  • get help:
$ pyment -h
  • run from a script:
import os
from pyment import PyComment

filename = 'test.py'

c = PyComment(filename)
c.proceed()
c.diff_to_file(os.path.basename(filename) + ".patch")
for s in c.get_output_docs():
    print(s)

Here is a full example using Pyment to generate a patch and then apply the patch.

Let's consider a file test.py with following content:

def func(param1=True, param2: str = 'default val'):
    '''Description of func with docstring groups style.

    Params:
        param1 - descr of param1 that has True for default value.
        param2 - descr of param2

    Returns:
        some value

    Raises:
        keyError: raises key exception
        TypeError: raises type exception

    '''
    pass

class A:
    def method(self, param1, param2=None) -> int:
        pass

Now let's use Pyment:

$ pyment test.py

Using Pyment without any argument will autodetect the docstrings formats and generate a patch using the reStructured Text format. So the previous command has generated the file test.py.patch with following content:

# Patch generated by Pyment v0.4.0

--- a/test.py
+++ b/test.py
@@ -1,20 +1,22 @@
 def func(param1=True, param2: str = 'default val'):
-    '''Description of func with docstring groups style.
-
-    Params:
-        param1 - descr of param1 that has True for default value.
-        param2 - descr of param2
-
-    Returns:
-        some value
-
-    Raises:
-        keyError: raises key exception
-        TypeError: raises type exception
-
-    '''
+    """Description of func with docstring groups style.
+
+    :param param1: descr of param1 that has True for default value
+    :param param2: descr of param2 (Default value = 'default val')
+    :type param2: str
+    :returns: some value
+    :raises keyError: raises key exception
+    :raises TypeError: raises type exception
+
+    """
     pass

 class A:
+    """ """
     def method(self, param1, param2=None) -> int:
+        """
+
+        :param param1:
+        :param param2:  (Default value = None)
+        :rtype: int
+
+        """
         pass

Let's finally apply the patch with the following command:

$ patch -p1 < test.py.patch

Now the original test.py was updated and its content is now:

def func(param1=True, param2: str = 'default val'):
    """Description of func with docstring groups style.

    :param param1: descr of param1 that has True for default value
    :param param2: descr of param2 (Default value = 'default val')
    :type param2: str
    :returns: some value
    :raises keyError: raises key exception
    :raises TypeError: raises type exception

    """
    pass

class A:
    """ """
    def method(self, param1, param2=None) -> int:
        """

        :param param1:
        :param param2:  (Default value = None)
        :rtype: int

        """
        pass

Also refer to the files example.py.patch or example_numpy.py.patch to see some other results that can be obtained processing the file example.py

If you enjoyed this free software, and want to thank me, you can offer me some bitcoins for a coffee, a beer, or more, I would be happy :)

Here's my address for bitcoins : 1Kz5bu4HuRtwbjzopN6xWSVsmtTDK6Kb89

pyment's People

Contributors

acarapetis avatar acatton avatar amudtogal avatar asettouf avatar bc2297 avatar bryder avatar dadadel avatar eastein avatar emantor avatar eventh avatar inirudebwoy avatar jankatins avatar jgarte avatar maqifrnswa avatar nyoungstudios avatar peterjc avatar rbignon avatar rirze avatar rth avatar tgy avatar timgates42 avatar viveris-villeurbanne avatar wagnerpeer avatar wolph avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyment's Issues

If already numpydoc format, will remove the Raises section

If a function is already described by a Numpydoc docstring, Pyment will remove the raises section when producing Numpydoc output docstring instead of keeping as is.

The following code (from tests/docs_already_numpydoc.py):

def func1(param1):
    """Function 1
    with 1 param

    Parameters
    ----------
    param1 : type
        1st parameter

    Returns
    -------
    string
        a message

    Raises
    ------
    KeyError
        when a key error
    """
    return "huh"

Will produce the patch:

--- a/docs_already_numpydoc.py
+++ b/docs_already_numpydoc.py
@@ -12,10 +12,7 @@
     string
         a message

-    Raises
-    ------
-    KeyError
-        when a key error
+    
     """
     return "huh"

This example is in the test suite test_pyment_cases.

pyment drops units from arguments

Is there a way to get pymentto retain parentheticals? Bracketed expressions are also seem to be elided.

For example, pyment produces the following patch

 def shot_noise(I, Idark, bandwidth, M=1, x=0):
-    """
-    Return the noise current associated with shot/poisson noise
-    Args:
-        I     = current         (A)
-        Idark = dark current    (A)
-        M     = APD Gain factor (--)
-        x     = excess noise (0.3 for Si, 0.7 for InGaAs, 1.0 for Ge APDs)
-    Returns:
-        shot_noise       (A)
+    """Return the noise current associated with shot/poisson noise
+
+    :param I: current
+    :param Idark: dark current
+    :param M: APD Gain factor (Default value = 1)
+    :param x: excess noise (Default value = 0)
+    :param bandwidth: 
+    :returns: shot_noise       (A)
+

Add option for output to STDOUT

It seems that there is currently no option to output to STDOUT. That would be a nice feature in my opinion, since my usual workflow with tools like this is to first get a quick preview of the proposed changes on STDOUT (usually piped to less) and, if satisfied, apply the changes in-place. Of course I can do that using the generated .patch file, but it's more cumbersome. So I'd love to have a CLI parameter to redirect the output to STDOUT.

Wrong Formatting for Input params with default values

Here is the function header generated by pyment

def plot_outlier(inlier, outlier, outlier_rgb = [1, 0, 0]):
    """Plot the outlier of ...

    :param inlier: 
    :param outlier: 
    :param outlier_rgb:  (Default value = [1)
    :param 0: 
    :param 0]: 

    """

I give an default value of the outlier_rgb = [1,0,0]. But it seems like pyment did not recognize it and create two additional param for me.

If already google format, will replace param description by its type and remove raise description

If a function a google docstring describing a parameter with a type, instead of letting it as is when producing google output format, Pyment will replace the param description by the type.
Furthermore, if a raise section is present with an exception described, this description will be remove and the Raises keyword indented.

The following code (from tests/docs_already_google.py):

def func1(param1):
    """Function 1
    with 1 param

    Args:
      param1(type): 1st parameter

    Returns:
      None

    Raises:
      Exception: an exception

    """
    return None

Will produce the patch:

# Patch generated by Pyment v0.3.2-dev4

--- a/docs_already_google.py
+++ b/docs_already_google.py
@@ -3,13 +3,12 @@
     with 1 param

     Args:
-      param1(type): 1st parameter
+      param1: type

     Returns:
       None
-
-    Raises:
-      Exception: an exception
+      
+      Raises:

     """
     return None

This example is in the test suite test_pyment_cases.

doctests incorrectly formatted with reST option

def meaning(subject, answer=False):
    """
    >>> meaning('life', answer=True)
    42
    """

converts to

def meaning(subject, answer=False):
    """>>> meaning('life', answer=True)
    42

    :params subject: 
    ...
    """

which breaks the doctest.

In addition, the reST markup is placed after the doctests, which isn't optimal for Sphinx-generated HTML.

Extra lines, and "Warning: the return type should be provided." when trying numpydoc as output.

test.py contains docstring in the google format:

def test(food, spam, water, n=4):
    """A short description.

    Some more descriptions here.

    Args:
        food: Something to eat.
        spam (string): A type of food.
        water (list, tuple): Multiple types.
        n (int): The number of things. Default is 4.

    Returns:
        The number of things.
    """

When trying to convert to numpydoc, I used: pyment test.py -i "google" -o "numpydoc". It created a patch file, and gave me a warning saying:Warning: the return type should be provided.`

The patch file also contained some errors. These are the contents of test.py.patch.

# Patch generated by Pyment v0.3.1

--- a/test.py
+++ b/test.py
@@ -1,14 +1,28 @@
 def test(food, spam, water, n=4):
     """A short description.
-
+    
     Some more descriptions here.

-    Args:
-        food: Something to eat.
-        spam (string): A type of food.
-        water (list, tuple): Multiple types.
-        n (int): The number of things. Default is 4.
+    Parameters
+    ----------
+    food :
+        Something to eat.
+    spam  : string
+        A type of food.
+    water  : list
+        Multiple types.
+    n  : int
+        The number of things. Default is 4.
+    spam :
+        
+    water :
+        
+    n :
+         (Default value = 4)

-    Returns:
+    Returns
+    -------
+    
         The number of things.
+
     """

This has some errors:

  • There are extra, separate lines for all items that have a type. (The order does not matter, and they are added at the end.)
  • Even though it was specified, it added a (Default value = 4) to n.
  • It added an extra line after the underline for Returns

The first two output issues occurred with reST as the output parameter as well, but there was no warning. The reST patch:

# Patch generated by Pyment v0.3.1

--- a/test.py
+++ b/test.py
@@ -1,14 +1,18 @@
 def test(food, spam, water, n=4):
     """A short description.
-
+    
     Some more descriptions here.

-    Args:
-        food: Something to eat.
-        spam (string): A type of food.
-        water (list, tuple): Multiple types.
-        n (int): The number of things. Default is 4.
+    :param food: Something to eat.
+    :param spam : A type of food.
+    :type spam : string
+    :param water : Multiple types.
+    :type water : list
+    :param n : The number of things. Default is 4.
+    :type n : int
+    :param spam: 
+    :param water: 
+    :param n:  (Default value = 4)
+    :returns: The number of things.

-    Returns:
-        The number of things.
     """

pyment doesn't use my pyment.conf if I don't tell it to do so

When running pyment from the root of my directory which contains the following pyment.conf file

first_line = true
quotes = '''
output_style = numpydoc
input_style = numpydoc
init2class = true

pyment uses the default settings instead.

I don't know if this is expected behavior. On my side I was expecting pyment to look for a pyment.conf in the current directory, because that's what most programs do when they have a configuration file.

I can submit a PR that does that if it helps!

option to write back the files instead of making patches

The way this is done in autopep8 could be a good guide to doing this. There are certainly cases where generating patches is useful, but with everyone using git these days, it's very easy to keep the old version of things and use "git diff" if you want to see the changes before committing them, rather than having to apply them all with patch.

WindowsError: [Error 183] Can not create a file that already exists

When pathing a file on windows (or directly modifying it) python throws "WindowsError: [Error 183] Can not create a file that already exists".

I could fix the problem myself by deleting the orignal file before renaming the temporal file in the write_to_file method. I just added os.remove(self.input_file) in line 320 in pyment.py.

I also encountered an end-of line issue (each line end was doubled in Spyder), but I could fix it with the pull request #25 by yoongu.

Anyhow, thanks for the great package, it works fine for me - at least after those changes.

Comments containing commented multi-line-(doc)strings

For the following class, I would expect to generate an empty docstring if running pyment without parameters:

class Class(object):
    # '''some 
    # multi-line
    # comment
    # string'''
    pass

However the commented multi-line-string gets surrounded by docstring quotes.

Output:

--- a/issue67.py
+++ b/issue67.py
@@ -1,6 +1,9 @@
 class Class(object):
-    # '''some 
+    """# '''some
     # multi-line
     # comment
-    # string'''
+    # string
+
+
+    """
     pass

To me it looks like a similar issue #58

Does not convert existing docstrings

Hi Dadadel,

I am really happy to have come across your package here, a really great idea and feat on your end! I am very excited to start using it, specifically for converting Google docstrings to reST.

But for some reason pyment refuses to convert existing Google docstrings. It just keeps the Google docstrings unchanged in place and adds reST parameters with their default values at the end (In your README you show that existing docstrings will be converted into the new format). I am on a Windows 8, and 32 bit Python 2.7, using pyment's v0.3 release.

I tried many different ways, and I even tried to run the same test example from your README with the following script in the Python IDLE, with the executing code inside a __main__ clause. Am I using pyment in the wrong way or could it be OS related?

def func(param1=True, param2='default val'):
    '''Description of func with docstring groups style.

    Params:
        param1 - descr of param1 that has True for default value.
        param2 - descr of param2

    Returns:
        some value

    Raises:
        keyError: raises key exception
        TypeError: raises type exception

    '''
    pass

class A:
    def method(self, param1, param2=None):
        pass

if __name__ == "__main__":
    import os
    from pyment import PyComment

    filename = __file__
    print filename

    c = PyComment(filename)
    c.proceed()
    c.diff_to_file(os.path.basename(filename) + ".patch")
    for s in c.get_output_docs():
        print(s)

And these are the diff results in the output patch file, which does not convert the Google docstrings the way they do in your README for the same example:

# Patch generated by Pyment v0.3.0

--- a/googledoc.py
+++ b/googledoc.py
@@ -1,22 +1,21 @@
 def func(param1=True, param2='default val'):
-    '''Description of func with docstring groups style.
-
-    Params:
-        param1 - descr of param1 that has True for default value.
-        param2 - descr of param2
-
-    Returns:
-        some value
-
-    Raises:
-        keyError: raises key exception
-        TypeError: raises type exception
-
-    '''
+    '''Description of func with docstring groups style.

Params:
        param1 - descr of param1 that has True for default value.
        param2 - descr of param2

    Returns:
        some value

    Raises:
        keyError: raises key exception
        TypeError: raises type exception

+

+    :param param1:  (Default value = True)

+    :param param2:  (Default value = 'default val')

+

+    '''

     pass

 class A:
+    ''' '''

     def method(self, param1, param2=None):
+        '''

+

+        :param param1: 

+        :param param2:  (Default value = None)

+

+        '''

         pass

AttributeError: 'NoneType' object has no attribute 'split'

I am using Python 3.6.0 and Pyment v0.3.2.

When running pyment -c pyment.conf test.py with

pyment.conf

first_line = true
quotes = '''
output_style = numpydoc
input_style = numpydoc
init2class = true

test.py

def hello_world():
  return 42

I get the following error

Traceback (most recent call last):
  File "/home/toogy/.pyenv/versions/project/bin/pyment", line 11, in <module>
    load_entry_point('Pyment==0.3.2.dev0', 'console_scripts', 'pyment')()
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/pymentapp.py", line 149, in main
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/pymentapp.py", line 98, in run
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/pyment.py", line 333, in proceed
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/docstring.py", line 1942, in generate_docs
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/docstring.py", line 1701, in _set_other
  File "/home/toogy/.pyenv/versions/project/lib/python3.6/site-packages/Pyment-0.3.2.dev0-py3.6.egg/pyment/docstring.py", line 308, in get_raw_not_managed
AttributeError: 'NoneType' object has no attribute 'split'

I think the numpydoc support is broken!

edit: I checked and this is also happening with Python 3.5.2

:raises in reST is incorrectly parsed

Hi,

This is my docstring:

        """
        :raises requests.exceptions.HTTPError: on response error from server
        :raises ValueError: on incorrect JSON

        """

This is patch produced by pyment:

-        """
-        :raises requests.exceptions.HTTPError: on response error from server
+        :raises requests: exceptions.HTTPError: on response error from server
         :raises ValueError: on incorrect JSON

         """

I have omitted parts not relevant for this issue. Returned class is sliced after first "."(dot).
Line should not be altered in my opinion at all.

STDIN support?

Just saw this, and I'd love to incorporate it into plugins like neoformat. It'd be nice if pyment could take in code via STDIN and output the newly formatted code (not a patch but just the updated code) via STDOUT. Essentially, the -w flag without actually overwriting a file.

created patch-file not correct

When trying to convert my handwritten docstrings to something more consisting I ran into a corrupted patch-file. I attached a shortened patch-file and the corresponding original file that reproduce the bug. I was using pyment v0.3.0 with Python 2.7.6 (in a WinPython distribution).

  1. As you can see, the whole docstring is supposed to be changed to convert ''' to """. But only the first line of the new docstring is marked with +, the other 3 lines are not marked, ergo they are treated as context that should be equal in both files. Therefore the line count of the hunk does not fit anymore and the patching fails.
  2. Also note that the last 3 lines got an extra whitespace in the beginning of the line. This is not present in the original file.
  3. As a side-note: pyment creates an a/ and an b/ in the beginning of the paths of source and target of the patch. Note that these are not the actual folders of the files. I executed pyment in the folder of the file. Comparing with patches created from diff, I would guess that you could just omit this behavior.

I do not think it is of importance here, but note that I am using pyment on Windows 7.

patchfile (created running pyment __init__short.py in the same folder as __init__short.py)

# Patch generated by Pyment v0.3.0

--- a/__init__ short.py
+++ b/__init__ short.py
@@ -12,12 +12,7 @@


 def init_test():
-    '''
-    Only working when in WinPython root directory.
-    
-    Returns:
-    maske, data0, data1, no_motion, motion
-    '''
+    """Only working when in WinPython root directory.

    Returns:
    maske, data0, data1, no_motion, motion"""

     maske = read_mask('./MRR/Testdaten/55_mask.bmp')
     display(maske)
     data0 = read_dicom_set('./MRR/Testdaten/188_13-12-10_56_1',

original file:

# -*- coding: utf-8 -*-

__version__ = '1.3.1'
# $Source$

#Some commentary


from .mod1 import *
#import mod2
from . import mod3, mod4


def init_test():
    '''
    Only working when in WinPython root directory.

    Returns:
    maske, data0, data1, no_motion, motion
    '''
    maske = read_mask('./MRR/Testdaten/55_mask.bmp')
    display(maske)
    data0 = read_dicom_set('./MRR/Testdaten/188_13-12-10_56_1',
                               unwrap=True, mask=maske, verbose=False)
    data1 = read_dicom_set('./MRR/Testdaten/188_13-12-10_54_1',
                               unwrap=True, mask=maske, verbose=False)
    no_motion = mean(data0, axis=0)
    motion = mean(data1, axis=0)
    return maske , data0, data1, no_motion, motion

edit: files now reproduce the bug.

Google indentation is set to 2 spaces

Hi,

first of all, congratulations for the amazing tool!

I have seen that converting to google format the indentation is explicitly set to 2 spaces. If I look at the latest Google Style Guide (
http://google-styleguide.googlecode.com/svn/trunk/pyguide.html?showone=Comments#Comments), I see the examples are set in 4 spaces (I think that's the recommended one). Could this be made somehow configurable? Right now I've patched it to add 4 spaces, but I understand there is a rationale behind explicitly setting them to 2...

Add Google docstring generation

Currently Pyment allow to recognize in input docstrings based on Google docs or similar. That category is called groups style.
Pyment should also allow to generate docstring in Google format.

Problem to install on RHEL-7 (with python 2.7)

matej@mitmanek: pyment (master)$ python setup.py develop --user
running develop
running egg_info
writing Pyment.egg-info/PKG-INFO
writing top-level names to Pyment.egg-info/top_level.txt
writing dependency_links to Pyment.egg-info/dependency_links.txt
writing entry points to Pyment.egg-info/entry_points.txt
reading manifest file 'Pyment.egg-info/SOURCES.txt'
writing manifest file 'Pyment.egg-info/SOURCES.txt'
running build_ext
Creating /home/matej/.local/lib/python2.7/site-packages/Pyment.egg-link (link to .)
Adding Pyment 0.3.2.dev0 to easy-install.pth file
Installing pyment script to /home/matej/.local/bin

Installed /home/matej/archiv/knihovna/repos/pyment
Traceback (most recent call last):
  File "setup.py", line 15, in <module>
    'pyment = pyment.pymentapp:main'
  File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.7/site-packages/setuptools/command/develop.py", line 27, in run
    self.install_for_development()
  File "/usr/lib/python2.7/site-packages/setuptools/command/develop.py", line 129, in install_for_development
    self.process_distribution(None, self.dist, not self.no_deps)
  File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 701, in process_distribution
    distreq.project_name, distreq.specs, requirement.extras
TypeError: __init__() takes exactly 2 arguments (4 given)
matej@mitmanek: pyment (master)$ 

Problem with regenerating empty param docstring

First of all, thanks for this great project! I'm seeing undesired behaviors when trying to run pyment twice in a row with the following function:

def fun(a, b):
    pass

The first run generates the following code:

def fun(a, b):
    """
    
    :param a:
    :param b:
    
    """
    pass

Run it again on the same file gives:

def fun(a, b):
    """
    
    :param a: param b:
    :param b:
    
    """
    pass

Whereas I am expecting the second run to not modify anything. However, running it a third time would not generate a patch.

:rtype: is removed from doc comments; :return: loses indentation

https://pythonhosted.org/an_example_pypi_project/sphinx.html#python-cross-referencing-syntax shows that :rtype: is the sphinx way to specify the type or return items, but pyment removes that tag from the docstring. It also seems to de-dent or lose the indentation of the :return: tag:

# Patch generated by Pyment v0.3.0

--- a/constants.py
# Patch generated by Pyment v0.3.0

--- a/constants.py
+++ b/constants.py
@@ -122,8 +124,8 @@

     :param range: A viewer structure range
     :type range: list or tuple
-    :return: A nice representation
-    :rtype: tuple
+:return: A nice representation
+
     """
     vs = VIEWER_STRUCTURE_KEYS
     _type = range[vs.RANGE_TYPE_IDX]

Fails on installation using python 3.6.2

Something is broken when installing in Python 3.6.2

In python 2.7.14 it works...

[humitos@julia:~]$ pyenv virtualenv 2.7.14 test
New python executable in /home/humitos/.pyenv/versions/2.7.14/envs/test/bin/python2.7
Also creating executable in /home/humitos/.pyenv/versions/2.7.14/envs/test/bin/python
Installing setuptools, pip, wheel...done.
Requirement already satisfied: setuptools in /home/humitos/.pyenv/versions/2.7.14/envs/test/lib/python2.7/site-packages
Requirement already satisfied: pip in /home/humitos/.pyenv/versions/2.7.14/envs/test/lib/python2.7/site-packages
(test) [humitos@julia:~]$ pip install pyment
Collecting pyment
  Could not find a version that satisfies the requirement pyment (from versions: )
No matching distribution found for pyment
(test) [humitos@julia:~]$ pip install git+https://github.com/dadadel/pyment.git#egg=pyment
Collecting pyment from git+https://github.com/dadadel/pyment.git#egg=pyment
  Cloning https://github.com/dadadel/pyment.git to /tmp/pip-build-MOA7T_/pyment
Installing collected packages: pyment
  Running setup.py install for pyment ... done
Successfully installed pyment-0.3.2.dev4
(test) [humitos@julia:~]$ 

but in python 3.6.2 I got this error:

[humitos@julia:~]$ pyenv virtualenv 3.6.2 test
Requirement already satisfied: setuptools in /home/humitos/.pyenv/versions/3.6.2/envs/test/lib/python3.6/site-packages
Requirement already satisfied: pip in /home/humitos/.pyenv/versions/3.6.2/envs/test/lib/python3.6/site-packages
(test) [humitos@julia:~]$ pip install git+https://github.com/dadadel/pyment.git#egg=pyment
Collecting pyment from git+https://github.com/dadadel/pyment.git#egg=pyment
  Cloning https://github.com/dadadel/pyment.git to /tmp/pip-build-sorkrbkd/pyment
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-sorkrbkd/pyment/setup.py", line 5, in <module>
        import pyment
      File "/tmp/pip-build-sorkrbkd/pyment/pyment/__init__.py", line 1, in <module>
        from .pyment import PyComment, __version__, __copyright__, __author__, __licence__
      File "/tmp/pip-build-sorkrbkd/pyment/pyment/pyment.py", line 8, in <module>
        from docstring import DocString
    ModuleNotFoundError: No module named 'docstring'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-sorkrbkd/pyment/
(test) [humitos@julia:~]$ python

Thanks!

Doctests get placed strangely in numpydoc format

Using the following input, the paragraphs are all placed together with several intervening lines, and the doctests are all placed at the end.

from __future__ import (absolute_import, division, print_function)

import sys

def _event_type_class_factory():
    """
    A line that describe this function that does some stuff.

        >>> import os
        >>> print(os.getcwd())

    Another line that is between some things.

        >>> import sys
        >>> sys.getdefaultencoding()

    Another filler line.

        >>> print('yes')

    """
    pass

Output using 7b4fd87 via pyment -o numpydoc -t -f False test.py:

from __future__ import (absolute_import, division, print_function)

import sys

def _event_type_class_factory():
    """
    A line that describe this function that does some stuff.


    Another line that is between some things.


    Another filler line.

    Parameters
    ----------

    Returns
    -------

    >>> import os
        >>> print(os.getcwd())

        >>> import sys
        >>> sys.getdefaultencoding()

        >>> print('yes')
    """
    pass

No patch is generated, only numbers are printed

When I run pyment against the provided example.py file, I don't get a patch output but only some numbers:

$ pyment example.py
49 50

The same happens for other files:

$ pyment one/of/my/files.py
3 3
15 17
19 20
23 24

I'm using version 0.3.3:

$ pyment --version
Pyment v0.3.3 - Copyright 2012-2018 - A. Daouzli - GPL3

def statement gets deleted

Running pyment (with fix #31) on this file

def hello_world(a=22, b='hello'):
  return 42

Gives the following patch

# Patch generated by Pyment v0.3.2

--- a/test.py
+++ b/test.py
@@ -1,2 +1,15 @@
-def hello_world(a=22, b='hello'):
+    '''
+
+    Parameters
+    ----------
+    a :
+         (Default value = 22)
+    b :
+         (Default value = 'hello')
+
+    Returns
+    -------
+
+    
+    '''
   return 42

It deletes the first line.

Now I may be wrong but I'm pretty sure this is not the expected output! 😉

Switch to ast module for docstring extraction

At the moment pyment is parsing complete Python files on its own in a very basic way.

However parsing and extracting any information from Python files is naturally a built-in feature of the programming language.
For this I propose to use the ast module and it's features of node visitors. See also:
ast.NodeVisitor
To extract docstrings one can use the function ast.get_docstring().

Another benefit would be the reliable identification of names, types and default values of function, class and method arguments. This would address issue #46

In the long run, switching to the ast module would also allow to use ast.NodeTransformer, which can be used to modify the ast in place, aka. modifying the docstrings in a more direct way.

Extra blank line for docstring with a muli-line description

If a function has no argument and a multi-line description, Pyment will insert two blank lines between the description and the end of the docstring. It should be one blank line.

The following example:

def func():
    """Description
    in two lines

    """
    pass

Will provide the patch:

# Patch generated by Pyment v0.3.2-dev4

--- a/tst.py
+++ b/tst.py
@@ -1,6 +1,7 @@
 def func():
     """Description
     in two lines
+
 
     """
     pass

Errors when output is numpydoc

For many files, I get the following error:

Traceback (most recent call last):
  File "/usr/local/bin/pyment", line 4, in <module>
    __import__('pkg_resources').run_script('Pyment==0.3.0', 'pyment')
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 534, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1445, in run_script
    exec(script_code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/Pyment-0.3.0-py2.7.egg/EGG-INFO/scripts/pyment", line 145, in <module>

  File "/usr/local/lib/python2.7/dist-packages/Pyment-0.3.0-py2.7.egg/EGG-INFO/scripts/pyment", line 142, in main

  File "/usr/local/lib/python2.7/dist-packages/Pyment-0.3.0-py2.7.egg/EGG-INFO/scripts/pyment", line 97, in run

  File "build/bdist.linux-x86_64/egg/pyment/pyment.py", line 286, in proceed
  File "build/bdist.linux-x86_64/egg/pyment/docstring.py", line 1812, in generate_docs
  File "build/bdist.linux-x86_64/egg/pyment/docstring.py", line 1789, in _set_raw
  File "build/bdist.linux-x86_64/egg/pyment/docstring.py", line 1709, in _set_raw_return
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

The command was pyment -i "google" -o "numpydoc" myfile.py.

Add to pypi

This is a super useful tool for updating old codebases to new standards! Thanks for developing it. It would be great to add this package to pypi!

Test cases dont work.

I worked on some improvements of pyments, like pyment pyment ;) and Pep8 adjustments. I added also pytest and travis support in my branch. I realized that some of the test cases fail, in case I dont have time to fix it myself I like to add this as a issue in the main repository.

Multi-line args descriptions are not indented correctly

Here's an example that illustrates the issue.

def foo(baz=1):
    """

    Args:
      baz: long-description breaking into multiple
        lines (Default value = 1)

    Returns:

    """
    pass

Formatting under google style results in:

def foo(baz=1):
    """

    Args:
      baz: long-description breaking into multiple
    lines (Default value = 1)

    Returns:

    """
    pass

where the second line of the arg description is indented incorrectly.

Let NumpydocTools and GoogledocTools inherit from a common base class

As discussed in the abandoned (closed) PR #60, I open up another issue better suited to the changes I propose.
During modifications to the source code to fix issue #49, I realized that there was room for improvement in how the different docstring styles (Numpydoc and Googledoc) were handled. They had a lot of duplicate code and if they would have used a common base class, issue #49 should not have happened, as the GoogledocTools class already contained the solution.

My proposal is to introduce the common base class to prevent future issues arising because of duplicate logic in the now different "DocTools"

list, tuple, dict default param values are not parsed correctly

When a function has parameters with default values that are list, dictionary or tuple, Pyment will just consider several parameters splitting on coma.

The following python code:

def func1(param1=[1, None, "hehe"]):
    pass

def func2(param1=(1, None, "hehe")):
    pass

def func3(param1={0: 1, "a": None}):
    pass

Will produce the patch:

# Patch generated by Pyment v0.3.2-dev4

--- a/issue46.py
+++ b/issue46.py
@@ -1,9 +1,29 @@
 def func1(param1=[1, None, "hehe"]):
+    """
+
+    :param param1:  (Default value = [1)
+    :param None: 
+    :param "hehe"]: 
+
+    """
     pass
 
 def func2(param1=(1, None, "hehe")):
+    """
+
+    :param param1:  (Default value = (1)
+    :param None: 
+    :param "hehe": 
+
+    """
     pass
 
 def func3(param1={0: 1, "a": None}):
+    """
+
+    :param param1:  (Default value = {0: 1)
+    :param "a": None}: 
+
+    """
     pass
 

Bullet point lists of arguments break if multiline

Example, call this test_bullets.py,

"""Silly mathematical example."""

import math

def euclidean_distance(point_a, point_b):
    """Return the Euclidean distance between points A and B.

    Arguments:
     - point_a - point in space as a list, tuple or array.
     - point_b - point in space as a list, tuple or array.
       This should be the same length as point_a.

    Example:

    >>> euclidean_distance((0, 0), (3, 4))
    5.0

    """
    if len(point_a) != len(point_b):
        raise ValueError("Points A and B must have the same length (dimensionality).")
    sum_of_squares = sum((a - b)**2 for (a, b) in zip(point_a, point_b))
    return math.sqrt(sum_of_squares)

if __name__ == "__main__":
    alpha = (1, 1, 1)
    beta =  (1, 4, 5)
    print("Example, distance from %r to %r is %r" %
          (alpha, beta, euclidean_distance(alpha, beta)))

Actual output:

$ pyment -o reST test_bullets.py && more test_bullets.py.patch
9 10
# Patch generated by Pyment v0.3.2

--- a/test_bullets.py
+++ b/test_bullets.py
@@ -3,18 +3,15 @@
 import math
 
 def euclidean_distance(point_a, point_b):
-    """Return the Euclidean distance between points A and B.
+    """
 
-    Arguments:
-     - point_a - point in space as a list, tuple or array.
-     - point_b - point in space as a list, tuple or array.
-       This should be the same length as point_a.
-
-    Example:
+    :param point_a: point in space as a list
+    :param point_b: point in space as a list
+    :param This: should be the same length as point_a
+    :param Example: 
 
     >>> euclidean_distance((0, 0), (3, 4))
     5.0
-
     """
     if len(point_a) != len(point_b):
         raise ValueError("Points A and B must have the same length (dimensionality).")

Or:

"""Silly mathematical example."""

import math

def euclidean_distance(point_a, point_b):
    """

    :param point_a: point in space as a list
    :param point_b: point in space as a list
    :param This: should be the same length as point_a
    :param Example: 

    >>> euclidean_distance((0, 0), (3, 4))
    5.0
    """
    if len(point_a) != len(point_b):
        raise ValueError("Points A and B must have the same length (dimensionality).")
    sum_of_squares = sum((a - b)**2 for (a, b) in zip(point_a, point_b))
    return math.sqrt(sum_of_squares)

if __name__ == "__main__":
    alpha = (1, 1, 1)
    beta =  (1, 4, 5)
    print("Example, distance from %r to %r is %r" %
          (alpha, beta, euclidean_distance(alpha, beta)))

Note the problem line :param This: should be the same length as point_a created from the second line of the bullet point for argument point_b.

Note also the problem line :param Example:

Note also that the periods (full stops) have been removed.

Note also the one-line summary Return the Euclidean distance between points A and B. was removed.

Expected output:

There is some flexibility but the continuation line must be indented, this is a valid reStructuredText docstring:

"""Silly mathematical example."""

import math

def euclidean_distance(point_a, point_b):
    """Return the Euclidean distance between points A and B.

    :param point_a: point in space as a list
    :param point_b: point in space as a list
      This should be the same length as point_a

    Example: 

    >>> euclidean_distance((0, 0), (3, 4))
    5.0
    """
    if len(point_a) != len(point_b):
        raise ValueError("Points A and B must have the same length (dimensionality).")
    sum_of_squares = sum((a - b)**2 for (a, b) in zip(point_a, point_b))
    return math.sqrt(sum_of_squares)

if __name__ == "__main__":
    alpha = (1, 1, 1)
    beta =  (1, 4, 5)
    print("Example, distance from %r to %r is %r" %
          (alpha, beta, euclidean_distance(alpha, beta)))

any upcoming release of pyment?

Hi,

my name is Mike Gabriel and I am a Debian Developer, that is, I can upload software to Debian.

I stumbled over this project when in need for a simply epydoc -> sphinx docstring converter. As I got fond of this nice tool, I consider bringing pyment to Debian.

For bringing pyment to Debian, it would be nice to have a recent release (not one that is 3 years old). Of course, I can package a Git snapshot, but a release tag is much more preferred, so that's why I ask.

Thanks!
Mike

Parameter's default value set in type

When a parameter having a default value was not documented, the generated docstring will contain in the type filed the default value while it should be added formatted in the parameter's field.

Trailing white space introduced

Tools like flake8 will complain about trailing white space (W291), and it seems pyment sometimes introduces white space.

As a specific example, take the test.py example from the README.rst file which produces the following:

        def func(param1=True, param2='default val'):
            """Description of func with docstring groups style.

            :param param1: descr of param1 that has True for default value
            :param param2: descr of param2 (Default value = 'default val')
            :returns: some value
            :raises keyError: raises key exception
	    :raises TypeError: raises type exception

	    """
            pass

        class A:
            """ """
            def method(self, param1, param2=None):
	        """

	        :param param1:
                :param param2:  (Default value = None)

	        """
                pass

There is a trailing space on the line :param param1: in the class A method, both here and in the patch itself.

I have confirmed this running pyments test.py locally using v0.3.2

Other examples (which I have not reduced to a minimal test case) introduce "blank lines" with the default indentation spaces included. I had a look at the code, but was confused by the multiply defined lambda function with_space which seems might be to blame.

Add maximum line length support

At the moment it appears that there's no way to specify a maximum line length, and thus many of the docstrings overhang and need to be manually corrected. It would be great to have this as an option!

existing class docstrings should not get empty Args: and Returns: sections

I tend to document my class with both a class doc string and a separate __init__ docstring for the constructor. Pyment seems to treat class doc strings as if they were method doc strings which is not helpful.

Desired behavior: only create Args: and Returns: sections if there is no separate __init__ doc string. Alternatively: never create empty Args: and Return: sections for classes. Only if there is :param etc statements in the original class doc string, creating these sections would make sense.

Raise block convertion

Hi,

I don't know if it is a problem but when the program finds exceptions without description it crashes. Could be useful if the script warns the user about it or creates an empty string in that place because as it is the error is not explicative.

The error is the following:

Traceback (most recent call last):
File "/usr/local/bin/pyment", line 11, in
load_entry_point('Pyment==0.3.2', 'console_scripts', 'pyment')()
File "build/bdist.macosx-10.12-x86_64/egg/pyment/pymentapp.py", line 150, in main
File "build/bdist.macosx-10.12-x86_64/egg/pyment/pymentapp.py", line 99, in run
File "build/bdist.macosx-10.12-x86_64/egg/pyment/pyment.py", line 344, in proceed
File "build/bdist.macosx-10.12-x86_64/egg/pyment/docstring.py", line 1960, in generate_docs
File "build/bdist.macosx-10.12-x86_64/egg/pyment/docstring.py", line 1937, in _set_raw
File "build/bdist.macosx-10.12-x86_64/egg/pyment/docstring.py", line 1822, in _set_raw_raise
TypeError: cannot concatenate 'str' and 'NoneType' objects

I'm using Python 2.7.14 on MacOS with Pyment v0.3.2 and I was converting from Google doc style to reST.

To reproduce the problem you can use the following code:

def foo():
    """Bar bar bar.

    Raises:
        test
    """
    pass

As you can see if test has no description the program doesn't work, you have to specify something as follow.

def foo():
    """Bar bar bar.

    Raises:
        test: bar
    """
    pass

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.