Git Product home page Git Product logo

lml's Introduction

lml - Load me later. A lazy plugin management system.

https://codecov.io/github/python-lml/lml/coverage.png https://pepy.tech/badge/lml/month https://img.shields.io/github/stars/python-lml/lml.svg?style=social&maxAge=3600&label=Star https://img.shields.io/static/v1?label=continuous%20templating&message=%E6%A8%A1%E7%89%88%E6%9B%B4%E6%96%B0&color=blue&style=flat-square https://img.shields.io/static/v1?label=coding%20style&message=black&color=black&style=flat-square https://readthedocs.org/projects/lml/badge/?version=latest

lml seamlessly finds the lml based plugins from your current python environment but loads your plugins on demand. It is designed to support plugins that have external dependencies, especially bulky and/or memory hungry ones. lml provides the plugin management system only and the plugin interface is on your shoulder.

lml enabled applications helps your customers [1] in two ways:

  1. Your customers could cherry-pick the plugins from pypi per python environment. They could remove a plugin using pip uninstall command.
  2. Only the plugins used at runtime gets loaded into computer memory.

When you would use lml to refactor your existing code, it aims to flatten the complexity and to shrink the size of your bulky python library by distributing the similar functionalities across its plugins. However, you as the developer need to do the code refactoring by yourself and lml would lend you a hand.

[1]the end developers who uses your library and packages achieve their objectives.

Quick start

The following code tries to get you started quickly with non-lazy loading.

from lml.plugin import PluginInfo, PluginManager


@PluginInfo("cuisine", tags=["Portable Battery"])
class Boost(object):
    def make(self, food=None, **keywords):
        print("I can cook %s for robots" % food)


class CuisineManager(PluginManager):
    def __init__(self):
        PluginManager.__init__(self, "cuisine")

    def get_a_plugin(self, food_name=None, **keywords):
        return PluginManager.get_a_plugin(self, key=food_name, **keywords)


if __name__ == '__main__':
    manager = CuisineManager()
    chef = manager.get_a_plugin("Portable Battery")
    chef.make()

At a glance, above code simply replaces the Factory pattern should you write them without lml. What's not obvious is, that once you got hands-on with it, you can start work on how to do lazy loading.

Installation

You can install lml via pip:

$ pip install lml

or clone it and install it:

$ git clone https://github.com/python-lml/lml.git
$ cd lml
$ python setup.py install

lml enabled project

Beyond the documentation above, here is a list of projects using lml:

  1. pyexcel
  2. pyecharts
  3. moban

lml is available on these distributions:

  1. ARCH linux
  2. Conda forge
  3. OpenSuse

License

New BSD

lml's People

Contributors

chfw avatar ayan-b avatar mcepl avatar gitter-badger avatar pgajdos avatar

Stargazers

Pho Hale avatar Andrew Christiansen avatar Alexander Eckert avatar John Walker avatar Joe Hosteny avatar Yuguo Xia avatar Žiga Avsec avatar John Vandenberg avatar github user avatar 玖亖伍 avatar Juan Pino avatar (╯°□°)╯︵ uᴉǝssnH ɐɟɐʇsoW avatar

Watchers

John Vandenberg avatar James Cloos avatar  avatar 玖亖伍 avatar

lml's Issues

to find plugin names with different naming patterns

atm, it only looks for any plugin names that start with a text, or that is white-listed and is not black-listed.

but now, it needs support plugin names that end with a text.

hence, the change will be to support regular expression in search so that prefix and suffix could be specified in one regular expression.

reference:

moremoban/moban#35

License for packaging

I initially created the openSUSE .spec using py2pack, and it injected the following into the spec

License: New BSD (FIXME:No SPDX)

I havent fully investigated this yet, and it isnt mentioned in https://en.opensuse.org/openSUSE:Packaging_Python .

I have found https://github.com/openSUSE/py2pack/blob/master/py2pack/spdx_license_map.p , which contains both New BSD License and BSD-3-Clause.

Interesting that https://packaging.python.org/tutorials/packaging-projects/ doesnt mention using setup(.. license='...', ..) at all.

Semi related, we should at least be including some more Trove classifiers, including a license classifier -- probably this one...?

License :: OSI Approved :: BSD License

No need to rush out a new release for this, as I can manually solve this one in the .spec , but it is an interesting problem that needs to be understood and solved so that running py2pack on a moban powered package 'just works', to make it easier to package them (them = moban / pyexcel / etc).

"Deprecated! since version 0.0.3. Please use scan_plugins_regex!"

Receiving this warning when trying out the v2 example . Any suggestions on fixing it ?
$robotchef_v2 "Portable Battery"
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/lml/loader.py:64: UserWarning: Deprecated! since version 0.0.3. Please use scan_plugins_regex!
"Deprecated! since version 0.0.3. Please use scan_plugins_regex!"
I can cook Portable Battery for robots

Regards

Tests not distributed for use by distro package validation

I am building openSUSE packages for lml & moban at https://build.opensuse.org/project/show/home:jayvdb:soe , and noticed that the tests and docs/source are missing, so there is only one doctest running to validate the package.

If they are not too big, it would be good to include tests and docs/source in the source tarball.

Alternatively, a second tarball including those could be created during the release process, to be used by packagers.

(I expect I will encounter the same problem when polishing the package for moban)

tests_require not complete

On Python 2.7, the tests need mock, and six is also mentioned in tests/requirements.txt but I can not see it mentioned in the source of the tests.

Even with them both installed on openSUSE, there is an interesting error at https://github.com/chfw/lml/blob/master/tests/test_utils.py#L16
import pyexcel_test

======================================================================
ERROR: test_utils.test_do_import
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/abuild/rpmbuild/BUILD/lml-0.0.8/tests/test_utils.py", line 17, in test_do_import
    import pyexcel_test
ImportError: No module named pyexcel_test

and also two others like

======================================================================
ERROR: test_plugin_loader.test_load_plugins
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/home/abuild/rpmbuild/BUILD/lml-0.0.8/tests/test_plugin_loader.py", line 39, in test_load_plugins
    info = CACHED_PLUGIN_INFO["test_io"][0]
IndexError: list index out of range
-------------------- >> begin captured logging << --------------------
lml.loader: DEBUG: scanning for plugins...
lml.utils: ERROR: pyexcel_test is abscent or cannot be imported
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/lml-0.0.8/lml/utils.py", line 42, in do_import
    return _do_import(plugin_module_name)
  File "/home/abuild/rpmbuild/BUILD/lml-0.0.8/lml/utils.py", line 51, in _do_import
    plugin_module = __import__(plugin_module_name)
ImportError: No module named pyexcel_test
lml.loader: DEBUG: pyexcel_test
lml.loader: DEBUG: No module named pyexcel_test
lml.loader: DEBUG: ignored pyexcel_io
lml.loader: DEBUG: scanning done
--------------------- >> end captured logging << ---------------------

https://build.opensuse.org/package/live_build_log/home:jayvdb:soe/python-lml/openSUSE_Tumbleweed/x86_64

It might be related to different versions of six ..?

Listing loaded plugins

Would love to be able to list loaded plugins.
Maybe also displaying some info about them: name, tags, etc.

Load plugins via setuptools entrypoints

See moremoban/moban#191 for the need. coala could also use this, as it loads bears using setuptools, and it would be really nice to have the setuptools dependency/problem managed.

I'm not sure lml is the best option, but it is worth asking the question and having a decision whether this would work.

There are three options, either

  1. wrap setuptools
  2. use an existing library which re-implements setuptools
  3. re-implement this setuptools feature internally within lml

If the first option is taken, there are a lot of versions of setuptools which are broken and need blacklisting, depending on which features is used, so it would be helpful to have a set of tests here which verify all supported versions of setuptools load the hooks correctly.

PyOxidizer support

https://github.com/indygreg/PyOxidizer/ is really interesting, especially as it has decided to not support __file__ (indygreg/PyOxidizer#69).

https://github.com/storyscript/community/issues/8#issuecomment-540956151 has a survey of the other similar tools.

If PyOxidizer works with lml, the others should also work.

The ability to ship a moban standalone binary with all deps, and extensions, included, would be fantastic ;-)

It would be ok even if some lml functionality is degraded because of no __file__.

PyOxidizer can be detected with sys.oxidized, and other similar tools usually set sys._MEIPASS.

Currently tests fail quite badly on PyOxidizer and other similar tools.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.