pyecore / pyecoregen Goto Github PK

View Code? Open in Web Editor NEW

31.0 8.0 6.0 137 KB

Python Model to Text framework for PyEcore, including the Ecore to Python generator

License: BSD 3-Clause "New" or "Revised" License

Python 82.22% Smarty 17.78%

pyecoregen's Introduction

pyecoregen - Python code generation from pyecore models

Contents

Overview
Installation
Usage

Overview

pyecoregen is a code generator, producing Python classes from Ecore models. It can be used at the command line as well as a module. In the latter case the passed model is expected to be an instance of the pyecore metamodel.

After using pyecoregen, you have a Python package representing the classes from the Ecore model. The generated classes are instances of the pyecore metaclasses. Please see pyecore for documentation how to work with them.

Installation

pyecoregen comes in form or a regular Python distribution and can be installed from Github or PyPI with a simple:

$ pip install pyecoregen

The library works with any version of Python >= 3.5.

Usage

Code generation can be done programmatically and directly at the command line.

Command line interface

After installation an executable script pyecoregen has been installed. Assuming library.ecore is your Ecore XMI file, and you want to generate the classes in some/folder, you can do:

$ pyecoregen -vv -e library.ecore -o some/folder

The -vv is optional to raise verbosity to log level DEBUG. You should see output like this:

2017-05-26 08:06:54,303 INFO [multigen.generator] Generating code to '/here/some/folder'.
2017-05-26 08:06:54,304 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x000001DCF3C61E80> --> '/here/some/folder/library/__init__.py'
2017-05-26 08:06:54,363 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x000001DCF3C61E80> --> '/here/some/folder/library/library.py'

The pyecoregen command line interface also allows you to generate the classes from a remote Ecore XMI file if its "path" starts with http(s)://. The usage does not change:

$ pyecoregen -e "http://path/towards/my/ecore" -o some/folder

Programmatic interface

If you need to generate code from an in-memory representation of a pyecore model, you instantiate the EcoreGenerator class and call the generate method. Assuming you have loaded above model and hold it's root package in library_pkg, you would generate with:

generator = EcoreGenerator()
generator.generate(library_pkg, 'some/folder')

Generator options

The end user can control some of the features how the metamodel code is generated. This can be done at the command line as well as via programmatic invocation. A command line parameter --my-param is then turning into a keyword argument my_param.

--auto-register-package (Default: False): If enabled, the generated packages are automatically added to pyecore's global namespace registry, which makes them available during XMI deserialization.
--user-module (Default: None): If specified, the given string is interpreted as a dotted Python module path. E.g. --user-module my.custom_mod will make the generated code import mixin classes from a module my.custom_mod. A generated class with name <name> then derives from a mixin <name>Mixin, which is expected to be part of the user module. If this option is used, the generator also produces a skeleton file which contains all required mixin classes and methods. Usually you copy parts of this template to your own module, which is then checked into version control all your other code.
--with-dependencies (Default: False): If enabled, the generator also generates code from all metamodels that are dependencies of the input metamodel. A metamodel dependency is typically a reference from the input metamodel to another .ecore file. Please note that this option introduces slower code generation as all metamodels must be scanned in order to determine dependencies.

pyecoregen's People

Contributors

Stargazers

Watchers

Forkers

xtutran cfandy fgro93 bmjjr rettenbs lenninst

pyecoregen's Issues

multiple inheritance requires to **kwargs in any case

If using multiple inheritance than the **kwargs need to be generated independent from eSuperTypes

{%- macro generate_class_init_args(c) -%}
    {% if c.eStructuralFeatures | list  %}, *, {% endif -%}
    {{ c.eStructuralFeatures | map(attribute='name') | map('re_sub', '$', '=None') | join(', ') }}
    {%- if c.eSuperTypes %}, **kwargs{% endif %}
{%- endmacro %}

{%- macro generate_super_init_args(c, user_module=False) -%}
    {%- if user_module and c.eStructuralFeatures -%}
    {{ c.eStructuralFeatures | map(attribute='name') | map('re_sub', '(.+)', '\g<0>= \g<0>') | join(', ') }}
    {%- endif %}
    {%- if c.eSuperTypes %}{{', ' if user_module and c.eStructuralFeatures else ''}}**kwargs{% endif %}
{%- endmacro %}

Following use case:

class A
  aattr1
  aattr2

class B 
 battr1
 battr2

class C(A,B)


C(aattr1=1, battr1=2)

A.__init__ looks like this def __init__(self, *, aattr1=None, aattr2=None)
This lead to the problem that A.__init__ is called with a keyword argument (battr1) which is not known by the init function.

BR
Andreas

PYPI tar file doesn't have template files

It appears that the tar file on PYPI (https://pypi.python.org/pypi/pyecoregen) doesn't have the templates included.

I tried to install it from the tar file and ran into errors that said the template files couldn't be found.

I looked through the tar file and didn't see them but when looked through the wheel file I found them.

I reinstalled using the wheel file and everything worked.

I couldn't figure out how to report the issue on PYPI so thought I would post here.

Thanks,
Ben

Prevent generation of invalid identifiers

I am using the generator to create Python classes from a metamodel for Franca IDL. The ecore model defined by the project contains some classes used in an expression language, e.g. there is a meta-class else. Right now, pyecoregen will therefore produce invalid Python code since Python keywords must not be used as identifiers.

This enhancement will add a Jinja filter to ensure conflicting names are adapted during code generation. The Pythonic way of adaptation is to add a trailing underscore, which is what this filter would be doing.

Import error when using freshly installed pyecoregen

After fresh installation of this library, the command line tool is throwing the following error message:

>pyecoregen -h
Traceback (most recent call last):
  File "C:\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Nina\ninaenv\Scripts\pyecoregen.exe\__main__.py", line 5, in <module>
  File "c:\nina\ninaenv\lib\site-packages\pyecoregen\cli.py", line 7, in <module>
    import pyecore.resources
  File "c:\nina\ninaenv\lib\site-packages\pyecore\resources\__init__.py", line 2, in <module>
    from . import xmi
  File "c:\nina\ninaenv\lib\site-packages\pyecore\resources\xmi.py", line 4, in <module>
    from lxml import etree
ImportError: DLL load failed: Die angegebene Prozedur wurde nicht gefunden.

Probably some dependency (lxml) is missing.

@varanega: I am a coworker of Mike...

arguments not forwarded to Mixin

If using Mixin the parameters are not forwarded to the MixinClass.

Documentation generation

Is it possible to generate documentation from an ecore model using this tool? If the feature is currently not available, could you give me some pointers to start?

Generator not working on ecore model

I'm trying to use pyecoregen on an ecore model (which can be found here: https://github.com/EnergyTransition/ESDL/blob/master/esdl/model/esdl.ecore). The model is developed in Eclipse Photon.

I'm getting the following error when trying to generate the classes:

$ pyecoregen -vv -e C:\\Data\\git\\esdl.next\\model\\esdl.ecore -o gen
2018-10-03 15:20:56,926 INFO [multigen.generator] Generating code to 'gen'.
2018-10-03 15:20:56,926 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x6fffe7aa6d8> --> 'gen/esdl/__init__.py'
2018-10-03 15:20:57,566 DEBUG [multigen.generator] <pyecore.ecore.EPackage object at 0x6fffe7aa6d8> --> 'gen/esdl/esdl.py'
Traceback (most recent call last):
  File "/usr/bin/pyecoregen", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python3.6/site-packages/pyecoregen/cli.py", line 15, in main
    generate_from_cli(sys.argv[1:])  # nocover
  File "/usr/lib/python3.6/site-packages/pyecoregen/cli.py", line 62, in generate_from_cli
    ).generate(model, parsed_args.out_folder)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 330, in generate
    super().generate(model, outfolder)
  File "/usr/lib/python3.6/site-packages/multigen/generator.py", line 39, in generate
    task.run(element, outfolder)
  File "/usr/lib/python3.6/site-packages/multigen/generator.py", line 65, in run
    self.generate_file(element, filepath)
  File "/usr/lib/python3.6/site-packages/multigen/jinja.py", line 43, in generate_file
    context = self.create_template_context(element=element)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 118, in create_template_context
    imported_classifiers=self.imported_classifiers(element)
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 96, in imported_classifiers
    imported |= {t for t in attributes_types if t.ePackage not in {p, ecore.eClass, None}}
  File "/usr/lib/python3.6/site-packages/pyecoregen/ecore.py", line 96, in <setcomp>
    imported |= {t for t in attributes_types if t.ePackage not in {p, ecore.eClass, None}}
  File "/usr/lib/python3.6/site-packages/pyecore/ecore.py", line 895, in __getattribute__
    self._wrapped = decoded.eClass
AttributeError: 'NoneType' object has no attribute 'eClass'

Any idea what goes wrong here?

Wrong code generated for enums

See pyecore/pyecore#126 (comment)

Quoting from there, for enums,

pyecoregen generates the wrong code:
for classif in otherClassifiers:
    eClassifiers[classif.name] = classif
    classif.ePackage = eClass

for classif in eClassifiers.values():
    eClass.eClassifiers.append(classif.eClass)
What happens is that, initially, eClassifiers is empty. Then, first, enums are added to eClassifiers, and then the eClass of each object in eClassifiers is added to the EPackage (that for some reason is called eClass). So we have the EEnum class added to the package we're defining! And, of course, it gets removed from the Ecore package, because eClassifiers is a containment relationship. I don't know the purpose of the second for loop, maybe it was to handle custom metaclasses or stuff like that, but it should filter the classes that it puts into eClass, or be removed altogether if it's not necessary.

Derived collection generation

I'm currently adding the support for derived collections in PyEcore, which means that using pyecoregen (with some adjustments), it will be possible to have a dedicated Python UML implementation. The thing with the derived collection is that it will obviously requires to manually add code in order to implement their behaviors.

In the case of a simple generation (no mixins), there is no issue, the code could be generated in the module along with the EClass code and that's all.

However, I'm not sure where to insert the code for the derived collection in the case where the user code is generated in a mixin. For example, for UML superClass reference, which is a derived collection, each time a Class is added to the superClass collection, a Generalization must be created at the same time. A first naïve code would look like this:

class DerivedSuperClass(EDerivedCollection):
    # ... code for __len__ ...etc
    def insert(self, index, item):
        self.check(item)
        self.owner.generalization.append(Generalization(general=item))

    # ...

If the derived code is located with the mixins, it cannot make a import Generalization as Generalization is in uml and uml imports mixins. A way of 'breaking' the circular dependency could be to add the import directly in the method that needs it, but I'm not sure that would be the best solution. I find a little bit cumbersome to ask people of importing like this.

Including PRs and some branches

Hi @moltob ,

I contact you this way because I was really certain that we exchanged emails in the past, but I was totally mistaken. There is some PR pending and some fix that are on some branches. Would it be ok if I include them myself?

Thanks,
Vincent

how to generate the default value of an attribute to the python class

the default value of an attribute in ecore file is not generated to the python class. I can't find it in the document.
any way to do it?
thank you very much

Enum newline break at wrong position.

Hi!

In our model we have an EEnum with quite a lot of options and in the latest version of pyecoregen the generated python code cannot compile due to a wrong line break.
It generates the following output (scroll to the right):

PhysicalQuantityEnum = EEnum('PhysicalQuantityEnum', literals=['UNDEFINED', 'ENERGY', 'POWER', 'VOLTAGE', 'PRESSURE', 'TEMPERATURE', 'EMISSION', 'COST', 'TIME', 'LENGTH', 'DISTANCE', 'IRRADIANCE', 'SPEED', 'STATE_OF_CHARGE', 'VOLUME', 'AREA', 'POWER_REACTIVE', 'COMPOSITION', 'FLOW', 'STATE', 'HEAD', 'POSITION', 'COEFFICIENT', 'WEIGHT', 'FORCE
                                                               ', 'CURRENT'])

It breaks before the ' instead of after the ,.

A different EEnum in our model has even more options, but is OK:

ProfileTypeEnum = EEnum('ProfileTypeEnum', literals=['UNDEFINED', 'SOLARIRRADIANCE_IN_W_PER_M2', 'WINDSPEED_IN_M_PER_S', 'STATEOFCHARGE_IN_WS', 'ENERGY_IN_WH', 'ENERGY_IN_KWH', 'ENERGY_IN_MWH', 'ENERGY_IN_GWH', 'ENERGY_IN_J', 'ENERGY_IN_KJ', 'ENERGY_IN_MJ', 'ENERGY_IN_GJ', 'ENERGY_IN_TJ',
                                                     'ENERGY_IN_PJ', 'TEMPERATURE_IN_C', 'TEMPERATURE_IN_K', 'POWER_IN_W', 'POWER_IN_KW', 'POWER_IN_MW', 'POWER_IN_GW', 'POWER_IN_TW', 'MONEY_IN_EUR', 'MONEY_IN_KEUR', 'MONEY_IN_MEUR', 'PERCENTAGE', 'MONEY_IN_EUR_PER_KW', 'MONEY_IN_EUR_PER_KWH', 'VOLUME_IN_M3', 'VOLUME_IN_LITERS'])

Any idea what causes this?
Thanks!

Allow passing context data via generator

This is likely also a change in the base project pymultigen. Allow to pass in template context data from generators. Currently, contextual data is collected via tasks only. This does not work well with global data that e.g. is determined via user-given command line arguments. At that level, only the generator class is known, not the various tasks it utilizes.

from x import y vs. import x

Hi @aranega

I try to create static code from complex ecore models like capella. I always run into the problem of circular includes.
But I think this could be avoided if other modules are loaded by using import x instead of from x import y.
So if you have a module like this

module examlple
__init__.py
from .one import Foo
from .one import helper

one.py
from example import two 

class Foo(two.Bar):
    def __init__(self):
        super().__init__()
        print('Init Foo')

def helper():
    print('Help')

def main():
    foo = Foo()

if __name__ == '__main__':
    main()

two.py
from example import helper

class Bar:
    def __init__(self):
        print('Init Bar')
        helper()

As far as I understood, by using from x import y y is loaded into the local namespace. Therefore example above will not work.
but changing two.py to

two.py
import example

class Bar:
    def __init__(self):
        print('Init Bar')
        example.helper()

And it can be resolved. So instead loading single classifiers from different ecore files importing the whole package can maybe solve this issue.
Maybe I find some time to dive deeper and can prepare a patch and some example. Hope you have a rough idea of the problem.

Maybe you can give me a hint how I easily get the containing package from the classifier in the code gen template.

Best regards,
Andreas