Git Product home page Git Product logo

mixology's Introduction

Mixology

A generic dependency-resolution library written in pure Python. It is based on the PubGrub algorithm.

Installation

If you are using poetry, it's as simple as:

poetry add mixology

If not you can use pip:

pip install mixology

Usage

Mixology is a dependency resolution algorithm.

In order to start using Mixology you need to initialize a VersionSolver instance with a PackageSource which should be adapted to work with your system.

Then, you need to call VersionSolver.solve() which will return a result with the list of decisions or raise a SolveFailure which will give a detailed explanation of the reason why the resolution failed.

Example

This example is extracted from the test suite of Mixology and uses the poetry-semver library.

First we need to have our own PackageSource class which implements the required methods and a simple Dependency class. Packages will be represented by simple strings.

from semver import Version
from semver import VersionRange
from semver import parse_constraint

from mixology.constraint import Constraint
from mixology.package_source import PackageSource as BasePackageSource
from mixology.range import Range
from mixology.union import Union


class Dependency:

    def __init__(self, name, constraint):  # type: (str, str) -> None
        self.name = name
        self.constraint = parse_constraint(constraint)
        self.pretty_constraint = constraint

    def __str__(self):  # type: () -> str
        return self.pretty_constraint


class PackageSource(BasePackageSource):

    def __init__(self):  # type: () -> None
        self._root_version = Version.parse("0.0.0")
        self._root_dependencies = []
        self._packages = {}

        super(PackageSource, self).__init__()

    @property
    def root_version(self):
        return self._root_version

    def add(
        self, name, version, deps=None
    ):  # type: (str, str, Optional[Dict[str, str]]) -> None
        if deps is None:
            deps = {}

        version = Version.parse(version)
        if name not in self._packages:
            self._packages[name] = {}

        if version in self._packages[name]:
            raise ValueError("{} ({}) already exists".format(name, version))

        dependencies = []
        for dep_name, spec in deps.items():
            dependencies.append(Dependency(dep_name, spec))

        self._packages[name][version] = dependencies

    def root_dep(self, name, constraint):  # type: (str, str) -> None
        self._root_dependencies.append(Dependency(name, constraint))

    def _versions_for(
        self, package, constraint=None
    ):  # type: (Hashable, Any) -> List[Hashable]
        if package not in self._packages:
            return []

        versions = []
        for version in self._packages[package].keys():
            if not constraint or constraint.allows_any(
                Range(version, version, True, True)
            ):
                versions.append(version)

        return sorted(versions, reverse=True)

    def dependencies_for(self, package, version):  # type: (Hashable, Any) -> List[Any]
        if package == self.root:
            return self._root_dependencies

        return self._packages[package][version]

    def convert_dependency(self, dependency):  # type: (Dependency) -> Constraint
        if isinstance(dependency.constraint, VersionRange):
            constraint = Range(
                dependency.constraint.min,
                dependency.constraint.max,
                dependency.constraint.include_min,
                dependency.constraint.include_max,
                dependency.pretty_constraint,
            )
        else:
            # VersionUnion
            ranges = [
                Range(
                    range.min,
                    range.max,
                    range.include_min,
                    range.include_max,
                    str(range),
                )
                for range in dependency.constraint.ranges
            ]
            constraint = Union.of(ranges)

        return Constraint(dependency.name, constraint)

Now, we need to specify our root dependencies and the available packages.

source = PackageSource()

source.root_dep("a", "1.0.0")
source.root_dep("b", "1.0.0")

source.add("a", "1.0.0", deps={"shared": ">=2.0.0 <4.0.0"})
source.add("b", "1.0.0", deps={"shared": ">=3.0.0 <5.0.0"})
source.add("shared", "2.0.0")
source.add("shared", "3.0.0")
source.add("shared", "3.6.9")
source.add("shared", "4.0.0")
source.add("shared", "5.0.0")

Now that everything is in place we can create a VersionSolver instance with the newly created PackageSource and call solve() to retrieve a SolverResult instance.

from mixology.version_solver import VersionSolver

solver = VersionSolver(source)
result = solver.solve()
result.decisions
# {Package("_root_"): <Version 0.0.0>, 'b': <Version 1.0.0>, 'a': <Version 1.0.0>, 'shared': <Version 3.6.9>}
result.attempted_solutions
# 1

Contributing

To work on the Mixology codebase, you'll want to fork the project, clone the fork locally and install the required dependencies via poetry <https://poetry.eustace.io>_.

git clone [email protected]:sdispater/mixology.git
poetry install

Then, create your feature branch:

git checkout -b my-new-feature

Make your modifications, add tests accordingly and execute the test suite:

poetry run pytest tests/

When you are ready, commit your changes:

git commit -am 'Add new feature'

push your branch:

git push origin my-new-feature

and finally create a pull request.

mixology's People

Contributors

sdispater avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mixology's Issues

What's the best way to use install mixology - here or through poetry?

Hey, I noticed that Poetry has a mixology component https://github.com/python-poetry/poetry/tree/master/src/poetry/mixology that looks more up-to-date than this repo (last commit here was a couple of years ago, last commit to that link was a couple of weeks ago).

IIUC you (the maintainer of this project) are the creator of poetry. I'm wondering if you have any advice for me as I'm developing my own package manager for a different programming language and would like to incorporate the mixology and semver sub-packages from Poetry's codebase. Is there a way I can pip/conda install some parts of Poetry's backend to get the up-to-date versions or should I stick to these standalone repos for mixology and poetry-semver that are perfectly functional but no-longer updated?

Thanks, any thoughts much appreciated :) And thanks for open-sourcing this fantastic approach too.

Bug in version solver when using a hashable object to represent a package, not just a string.

I believe it's this line:

changed.add(str(self._propagate_incompatibility(root_cause)))

self._propagate_incompatibility will return the _conflict type which is the hashable type used to represent a package (in my usecase this is not a string). The line converts it to str which is a problem since then it is no longer a valid key in the self._incompatibilities dictionary (line 113 of the same function).

def _propagate(self, package): # type: (Hashable) -> None
"""
Performs unit propagation on incompatibilities transitively
related to package to derive new assignments for _solution.
"""
changed = set()
changed.add(package)
while changed:
package = changed.pop()
# Iterate in reverse because conflict resolution tends to produce more
# general incompatibilities as time goes on. If we look at those first,
# we can derive stronger assignments sooner and more eagerly find
# conflicts.
for incompatibility in reversed(self._incompatibilities[package]):
result = self._propagate_incompatibility(incompatibility)
if result is _conflict:
# If the incompatibility is satisfied by the solution, we use
# _resolve_conflict() to determine the root cause of the conflict as a
# new incompatibility.
#
# It also backjumps to a point in the solution
# where that incompatibility will allow us to derive new assignments
# that avoid the conflict.
root_cause = self._resolve_conflict(incompatibility)
# Back jumping erases all the assignments we did at the previous
# decision level, so we clear [changed] and refill it with the
# newly-propagated assignment.
changed.clear()
changed.add(str(self._propagate_incompatibility(root_cause)))
break
elif result is not None:
changed.add(result)

Need help when port the PubGrub version solver into other language

Hi, @sdispater, thanks for your great work here!

I was confused when I read a piece of your code, see
https://github.com/sdispater/mixology/blob/master/mixology/incompatibility.py#L29
there you remove the root package from generated incompatibilities, since it will always be satisfied, but,

term.is_positive() and term.package != Package.root()

is there a typo here? I think it should be:

term.is_positive() and term.package == Package.root()

I do a double check by reading ruby port of the PubGrub version solver, see https://github.com/jhawthorn/pub_grub/blob/master/lib/pub_grub/incompatibility.rb#L121

Would you mind to make a check and explain? Thanks!

PyPA is seeking for a paid contractor for dependency resolution

I just wanted to let you know that PyPA is looking for someone to do similar work.

This is paid (important) work.

In case you haven't heard about this, it's here: http://pyfound.blogspot.com/2019/11/seeking-developers-for-paid-contract.html

BTW, after having use poetry a little bit, I really like it.

Et puisque tu es visiblement français, sache que poetry me sert pour https://github.com/vpoulailleau/padpo qui sert à la traduction en français de la doc officielle de Python.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.