lamenezes / simple-model Goto Github PK

In order to create models from a list currently we have to do the following:

my_list = [{'foo': 1, 'bar': 2}, {'foo': 0, 'bar': -10}]
models = [MyModel(**data) for data in my_list]

It would be nice to have a SimpleModel classmethod that did it for us in order to avoid repetition and minor mistakes:

my_list = [{'foo': 1, 'bar': 2}, {'foo': 0, 'bar': -10}]
models = MyModel.build_many(my_list)

Hi @lamenezes!

I'm doing some tests with this lib in a service that needs to load a large amount of data from the web, when I decided to see how the method build_many works, so I discovered that this method returns a list of models. I did a simple test to see the performance modifying this method to return a generator instead a list. The results are shown below:

Ran memory_usage to build_many (Generator) in 0 seconds
Initial RAM: 82.6 MiB
Final RAM: 82.6 MiB
Total built data: 19886

Ran memory_usage to build_many (list) in 5 seconds
Initial RAM: 82.6 MiB
Final RAM: 88.7 MiB
Total built data: 19886

I think that this method can return a generator to improve the memory usage and the total time to load the models.

In my tests, I not found any kind of problem with this conversion.

How should the model behave on the following case:

class Foo(Model):
    fields = ('foo',)
    def clean_foo(self, value):
        return value + 10

foo = Foo(foo=0)
foo.clean()  # foo.foo == 10
foo.clean()  # foo.foo == 20  this should be 20 or 10?

Sometimes it's necessary to link on model to another such as:

class Product(SimpleModel):
    fields = ('name', 'brand',  'attributes')
    attributes = fields.RelatedField(ProductAttribute)


class ProductAttribute(SimpleModel):
    fields = ('name', 'description', 'value')

It makes easier to create composed models as above by just doing:

>>> product_data = {
...    'name': 'foo prod',
...    'brand': 'bar brand',
...    'attributes': [{
...        'name': 'baz attr',
...        'description': "i'm a baz attribute",
...        'value': 69
...    },
...    {
...        'name': 'qux attr',
...        'description': "i'm a qux attribute",
...        'value': 6.9
...    }]
...}

>>> product = Product(**product_data)
>>> product.attributes
[ProductAttribute(name='baz attr', ...), ProductAttribute(name='qux attr', ...)]

For example, I can't create the following subclass to use "internally" for my models:

from simple_model import to_dict, Model

class AsDictModel(Model):
    def as_dict(self):
        self.validate()
        return to_dict(self)

When running the following code the following exception happens:

AssertionError: MyModel model must define class attributes

currently models doesn't have to support of having fields as properties such as:

class Foo(Model):
    bar: str

    @property
    def bar(self):
        return "i'm a bar"

Hi @lamenezes. Given the follow code snippet...

from simple_model import Model, model_many_builder


class A(Model):
    attr = ''
    
    def echo(self, text):
        print(text)


class B(A):
    pass

When I try to build many B models, like this:

data = [{'attr': 1}, {'attr': 2}]
models = model_many_builder(data, cls=B)

I got AttributeError trying to access parent methods from children models instances:

model_0 = next(models)
model_0.echo('test')

AttributeError: 'MyModel' object has no attribute 'echo'

Sometimes using dict(model) isn't enough to suit serialization needs. It would be nice to have a simple serializer to do such:

>>> data = {'foo': 123, 'bar': 456, 'baz': 789}
>>> model = DynamicModel(**data)
>>> class SimpleSerializer:
...    fields = ('foo', 'baz')  # do not serialize Model.bar
>>> serializer = SimpleSerializer(model)
>>> serializer.serialize()
{'foo': 123, 'baz': 789}

In the future this class can be used to perform actual serialization (not only dict conversion) for types like json, xml etc.

Currently the README file is being used as docs for this lib. This won't be changed in the near feature. So it would be great to have a TOC on it to improve readability.

ModelField logic [1] is kinda complicated and using typing.cast [2] might make it simpler.

[1] https://github.com/lamenezes/simple-model/blob/master/simple_model/fields.py#L33
[2] https://docs.python.org/3/library/typing.html#typing.cast

I have model like this:

from decimal import Decimal
from simple_model import Model
from .utils import decimal as sanitize_decimal

class MyModel(Model):
    name: str
    value: Decimal
    user: str

    def clean_value(self, value):
        return sanitize_decimal(value)

And I need a sanitized value because my value contains a comma. So I need to create an object like this:

my_object = MyModel(name='Rafael Henrique', amount='23232,43', user='rafaelhenrique')

But when I call clean this problem happens below:

>>> my_object = MyModel(name='Rafael Henrique', amount='23232,43', user='rafaelhenrique')
>>> my_object.clean()
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/rafael/.virtualenvs/my-project/lib/python3.6/site-packages/simple_model/models.py", line 122, in clean
   clean_value = descriptor.clean(self, value)
 File "/home/rafael/.virtualenvs/my-project/lib/python3.6/site-packages/simple_model/fields.py", line 59, in clean
   value = self.convert_to_type(instance, value)
 File "/home/rafael/.virtualenvs/my-project/lib/python3.6/site-packages/simple_model/fields.py", line 56, in convert_to_type
   return field_type(value)
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

To "solve" this problem i need use this method (below) to override clean:

    def clean(self):
        self.value = self.clean_value(self.value)
        super().clean()

My suggestion: pysimplemodel read the clean_<field> first and then call clean from the base class after.

That's it.

There are some simple model features undocumented such as:

• model_builder
• DynamicModel
• Errors

A few days ago, I tried to convert a model instance to a dict, using the syntax dict(model), and I got TypeError. The scenario is described below:

import typing
from dateutil import parser

from simple_model import Model


class MyModel(Model):
    name: str = ''
    timestamp: typing.Any = None

    def __post_init__(self):
        self.clean()
        
    def clean_timestamp(self, value):
        return parser.parse(value)


data = {'name': 'Name', 'timestamp': '2018-02-05T10:00:01.000001Z'}
model = MyModel(**data)

dict(model)  # here the error occurs 

So, the TypeError exception is raised, because simple model calls the method self.clean() during the conversion for a dict, and it tries to convert the timestamp attribute to a datetime again, but in this case, it already a datetime.

I would like to know if this behavior is what you expected when you created this resource, or if it is a thing that needs to be fixed. I can verify the value type on clean_timestamp method, but I don't know if it's the better approach.

When using enum attribute and use model.as_dict(), the enum attribute value doesn't is converted appropriately:

I guess .as_dict() should return this:

{'bar': 'bar', 'foo':'foo'}

Use re.compile instead of direct re.sub on the utils module [1] [2]

[1] https://github.com/lamenezes/simple-model/blob/master/simple_model/utils.py#L20
[2] https://github.com/lamenezes/simple-model/blob/master/simple_model/utils.py#L24