jbweston / miniver Goto Github PK

I believe the encoding declaration # -*- coding: utf-8 -*- is not needed any more as far as Python 3 is concerned. Do you rely on other tools (like your IDE or text editor) to make use of that information?

In any case it is not used consistently in all files, I would recommend removing it from all files or adding it to all files.

After installing miniver in a new project we will typically get a version unknown until we make a tag.
Miniver could detect this and notify the user, and possibly offer to apply some tags (maybe on the first commit of the repo).

At the moment people need to manually copy a bunch of files and snippets, but most of this can be automated.

The contents of the setup.py cannot be automated, as it would be excessively complicated to parse that file and insert that snippet at the correct location. OTOH everything else can be automated.

We should:
+ add an entrypoint script that grabs the version.py in the installed version of miniver and inserts the necessary lines into __init__.py and .gitattributes. It should then print the necessary lines that need to be copied manually by the user to the terminal and exit
+ add a script that can be curld and piped into bash/python that will do the same thing as the entrypoint script, but it will get its version.py from GitHub instead.

The 0.7.0 release of miniver added support for allowing packages inside of a "src" directory.
While this makes miniver more flexible than before, the current implementation fails when the package is inside a directory with a different name.

When building a package that is part of a native namespace, your repository will look something like this:

.                                 # <-- Root of the repository
├── some_namespace                # <-- namespace directory
│   └── some_package              # <-- distribution package directory
│       ├── some_sourcefile.py    #
│       ├── .gitattributes        #
│       ├── __init__.py           #
│       ├── _static_version.py    #
│       └── _version.py           #
├── README.md
└── setup.py

In this scenario, producing a wheel file (by running python3 setup.py bdist_wheel) will fail because miniver's cmdclass (from _version.py) contains classes that make reference to ./some_package/_static_version.py (instead of ./some_namespace/some_package/_static_version.py).

This could be very easily solved if instead of having a dictionary cmdclass in _version.py, we have a function with a signature get_cmd(prefix_dir="") that can pre-pend any arbitrary directory to the classes contained in the dictionary.
With this proposed approach, in your setup.py you can call either:

• module.get_cmdclass("some_namespace") when your distribution is part of a namespace
• module.get_cmdclass("src") When your distribution is inside of a "src" directory
• module.get_cmdclass() When your distribution is not part of either a namespace nor inside of a "src" directory

NOTE: Solving this problem is related to issue #38, and it's a pre-requisite to having a solution for that issue.

There is a comment in _version.py that reads:

# The top-level directory of the current Git repository is not the same
# as the root directory of the distribution: do not extract the
# version from Git.

I tried looking for documentation elaborating on why this is desirable, but didn't find much.

The problem I'm experiencing is that requiring the distribution package to be at the top level of the repository makes miniver fail whenever we want to version and package two distributions that belong to the same namespace.

Here's the documentation on how to structure several packages so that they are part of the same namespace.

As you can see, this approach requires that your distribution package is contained in a subdirectory inside of another directory with the name of the namespace, like so:

$ tree
.                                 # <-- Root of the repository
├── some_namespace                # <-- namespace directory
│   └── some_package              # <-- distribution package directory
│       ├── some_sourcefile.py    #
│       ├── .gitattributes        #
│       ├── __init__.py           #
│       ├── _static_version.py    #
│       └── _version.py           # <-- miniver will refuse to get the version from git because it's one level removed from the root
├── README.md
└── setup.py

Even though the advantage of miniver is being able to manually tag versions, in order to get things to work nicely we often need to do extra work. e.g. after tagging vX.Y.0 we should immediately create a commit and tag it as vX.(Y+1)-dev. It would be useful to be able to do this with a single command.

IMO we should have a single command miniver that has 2 subcommands: install, which works like install-miniver now, and bump that implements the new functionality.

Perhaps this is how miniver should work, but if there are no tags in the repository, the version will be unknown. This can be fixed by adding a --always flag to git-describe.

Make it so that _static_version.py and _version.py doesn't need to exist in repo.
Use the default versions from miniver package.

Used miniver as replacement for versioneer. Ran into problem:

FileNotFoundError: [Errno 2] No such file or directory: '<package_name>/_version.py'

Fixed the issue by changing:

spec = spec_from_file_location('version', os.path.join(package_name, '_version.py'))

to:

spec = spec_from_file_location('version', os.path.join('src', package_name, '_version.py'))

adding 'src' to path. This information should be included in the documentation. If I'll find time I'll provide a merge request.

In the hpc05 package I use a root/src/hpc05 structure.

Here this check:

    if not os.path.samefile(p.communicate()[0].decode().rstrip("\n"), distr_root):	    # if not os.path.samefile(p.communicate()[0].decode().rstrip("\n"), distr_root):
        # The top-level directory of the current Git repository is not the same	    #     # The top-level directory of the current Git repository is not the same
        # as the root directory of the distribution: do not extract the	    #     # as the root directory of the distribution: do not extract the
        # version from Git.	    #     # version from Git.
        return	

fails and results in a unknown version. If I comment this out like in basnijholt/hpc05@d27b432 then the correct version is displayed.

The installed version of miniver has a modified _static_version.py, which includes the miniver version installed. This is not what we want in the _static_version.py wherever we install miniver

Context

As of the current version (0.7.0), miniver has the following mechanism for detecting a "dirty" state on a tree:

1. Run in a sub-process: git", "describe", "--long", "--always --first-parent or git", "describe", "--long", "--always (if the first one fails). If both these commands fail, don't go any further see code
2. Assuming one of the commands in the previous step succeeded, take the plaintext output, sanitize and extract values for 3 fields: release, dev, git (see code)
3. Figure out the "labels" (such as the "dirty" label) needed to construct a valid PEP440-complient format (see code)

BUG / Explanation

This last step is not properly detecting when a tree is "dirty" in two cases I have observed:

1. When one ore more files have been modified and all modifications have been staged (i.e. when you're ready to commit changes but haven't yet committed them)
2. When a completely new file has been added to the tree

This undesired behaviour happens because the current approach uses git diff --quiet which returns a value of 1 when a tracked file in the last commit has been modified

Proposal

A better mechanism would leverage the output of git describe --dirty by looking at the ending of the string (which has the form v0.0.2rc6-1-g409e8dd-dirty) in order to determine if the tree is dirty or not

Following the instructions from the readme on current master (abe5a59) results in the following:

curl https://raw.githubusercontent.com/jbweston/miniver/master/install-miniver | python - . 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5280  100  5280    0     0  23677      0 --:--:-- --:--:-- --:--:-- 23677
Traceback (most recent call last):
  File "<stdin>", line 170, in <module>
  File "<stdin>", line 134, in main
  File "<stdin>", line 127, in get_args
NameError: name '_miniver_version' is not defined

Currently miniver only supports packages where the package is a direct subdirectory of the project root.
However it is also common to have an src directory, with the package as a subdirectory of that.
It would be good to support this common case also.

Currently we need to miniver ver <package_name. Ideally we would just miniver ver.

I've tried adding miniver here: https://github.com/instagrambot/instabot/pull/578/files

basnijholt-macbook ➜  instabot git:(version)  pip install .
Processing /Users/basnijholt/Downloads/instabot
Installing collected packages: instabot
  Running setup.py install for instabot ... done
Successfully installed instabot-unknown
basnijholt-macbook ➜  instabot git:(version)  python -c 'import instabot; print(instabot.__version__)'
unknown

Do you have any idea why it doesn't work?

In recent versions of git, the command fails if the .git directory is not owned by the user executing the git command. This is the result of a security patch CVE-2022-24765.

Others have seen this problem, too, for example setuptools-scm. Their fix is to specify --git-dir explicitly. This would be straightforward in miniver, but the module _version.py does not know where the .git directory lives (it is commonly located in the parent directory of the python root package, but this is not guaranteed). So I don't have an immediate suggestion for a foolproof fix.

I am trying to modify miniver to not allow installations that do not properly resolve to a known version by raising an exception. I therefore modified all the places where unknown was returned with a new exception. This works fine for pip install -e ., but not for pip install .

This made me dig deeper, and I am not sure I understand how this is ever possible -- in the case where you are doing pip install ., the first thing pip does is copy the repo to a temporary folder, without the hidden folders, which means the .git folder is no longer present.

I am obviously missing something important here, but how can it be possible for miniver to successfully get the git version during such an install? Do I always have to install with pip install -e . if I am trying to do a local install from a repo?

With Python 3.11 around the corner, how about updating the versions of Python explicitly supported and tested?

GitHub actions now support CI.

This would allow us to unify our CI setup into a single file.

Also at the moment every other Travis build fails due to timeouts (unrelated to the code AFAICT) so we could avoid this.

Another issue I came across, was that the version used for the package build by miniver was not based on the latest git tag:

$ git tag
v1.0.1
v1.0.1p
v1.0.1p1
v1.0.2
v1.0.3
v1.1
v2.0.0
v2.0.0rc1

Given these tags, the resulting package name was:

• <package-name>-1.0.3.dev11+ge33b9565.dirty

I have to admit, that the tag names are not that well chosen, but the error still seems strange.

Perhaps a little bit far-stretched, but sys.exit re-defines the builtin exit helper for the interactive shell:

See Re-definition found for builtin function - PYL-W0622.

It's a bit more typing, but perhaps less ambiuguous

Setup.py has been replaced by setup.cfg and is now moving towards pyproject.toml. Probably there aren't many projects on setup.py anymore.

While I was cleaning up an old setup.py & miniver project, migration seemed straight forward (with pypa/setuptools#2570 now implemented).

Except that cmdclass dict values must be module paths to a public, top-level class. Since the change 1c9f8ca, the build command classes can only be created through function calls, which does not work for the declarative setup.cfg.

I tried working around by creating a helper object with lazy properties (so I can preserve the lazy import of super classes), but that is not detected by setuptools and gives an error.

PACKAGE_SOURCE_PATH = os.path.dirname(__file__)

class CmdClass:
    build_py = None
    sdist = None

    def __init__(self, pkg_source_path):
        self._pkg_source_path = pkg_source_path

    @property
    def build_py(self):
        from setuptools.command.build_py import build_py as build_py_orig

        pkg_source_path = self._pkg_source_path

        class _build_py(build_py_orig):
            def run(self):
                super().run()

                src_marker = "".join(["src", os.path.sep])

                if pkg_source_path.startswith(src_marker):
                    path = pkg_source_path[len(src_marker):]
                else:
                    path = pkg_source_path
                _write_version(os.path.join(self.build_lib, path, STATIC_VERSION_FILE))

        return _build_py


    @property
    def sdist(self):
        from setuptools.command.sdist import sdist as sdist_orig

        pkg_source_path = self._pkg_source_path

        class _sdist(sdist_orig):
            def make_release_tree(self, base_dir, files):
                super().make_release_tree(base_dir, files)
                _write_version(os.path.join(base_dir, pkg_source_path, STATIC_VERSION_FILE))

        return _sdist


cmdclass = CmdClass(pkg_source_path=PACKAGE_SOURCE_PATH)


def get_cmdclass(pkg_source_path=PACKAGE_SOURCE_PATH):
    cmds = CmdClass(pkg_source_path=pkg_source_path)
    return dict(sdist=cmds.sdist, build_py=cmds.build_py)

# Unlike expected, setup.cfg cannot access cmdclass.build_py, but the following would work:
# build_py = cmdclass.build_py
# sdist = cmdclass.sdist

[options]
cmdclass =
    build_py = my_package._version.cmdclass.build_py
    sdist = my_package._version.cmdclass.sdist

ModuleNotFoundError: __path__ attribute not found on 'my_package._version' while trying to find 'my_package._version.cmdclass'

In short, first I think it would be good to support setup.cfg, and second, I am not sure whether it is feasible without moving the build command classes to the top level.

This is related to issue #16.

I believe that the default behaviour for numbering releases. For example, if I release version 0.2.0 but don't push the next release tag automatically, versions for the next pushed commits would be 0.2.0.devX, which is incorrect: 0.2.1.devX or 0.3.0.devX or 0.3.devX would make more sense. Probably first would be the most sane default, if dev tag is not pushed explicitly.

If I install miniver into a project, remove build/, and run setup.py install, then the installed package will have version unknown.

This is because we override the build and sdist commands, but not build_py. The install command only runs build_py unless there are extension modules.

Miniver imports setuptools, and therefore may fail in some environments. Is this intentional? Should packages using miniver have setuptools as a requirement?