jaimegildesagredo / booby Goto Github PK

Data modeling and validation Python library

Home Page: https://booby.readthedocs.org

License: Other

Python 100.00%

booby's Introduction

Jaime Gil de Sagredo Luna

Passionate about building high-quality software products that deliver value to users. With a focus on fast, iterative and user-driven development, I specialize in developing both MVPs and long-term products using cost and time sustainable processes.

My core practices and experience encompass a wide range of methodologies, including eXtreme Programming, Simple Design, Lean and DevOps. I am well-versed in Test-Driven Development (TDD), Domain-Driven Development (DDD), and Hexagonal architecture, which enable me to build robust and scalable software solutions.

I can help your company, startup, or software development team build high-quality software products that satisfy your users needs. Whether it's through crafting new products, iterating your existing products, or guiding and mentoring your development team to improve their skills and capabilities.

Contact me to schedule a meeting or find more about me at: LinkedIn, Twitter or my personal page.

booby's People

Contributors

Stargazers

Watchers

Forkers

shakaran yunmanger1 rustem myaser ninumedia waytai jvrsantacruz svisser dannygoodall stevegzr adamcharnock dokraus dkellner makkus cr0hn jtszalay chrmorais metabolize-forks

booby's Issues

Validation for required fields doesn't work (Seems like no Validation is working)

The validation for 'required=True' on fields doesn't work for me, I tried to use it on a String, but after I applied it, I can still create a Model with empty fields and even set the fields to None.

Add an `is_valid` property to the Model class

Example:

if not jack.is_valid:
    print dict(jack.validation_errors)

Date / DateTime field

I think DateField and DateTimeField should be a part of this library.

This is what stopped me from potentially using booby in my project.

choices, defaults and None values

The API I am accessing returns JSON with a field that will return null for almost all fields. If I POST/PUT this model back into the API, it will interpret the null values as if they are unset.

There is an exception to this. One specific String field (however, there is no reason why in future this could not apply to other value types) in which a POST/PUT operation must be one of x values.

As a result, I apply the choices options and also a default The field is current defined as:
option = fields.String(choices=['FOO', 'BAR', 'ANY', 'NONE'], default='NONE')

Imagine the following though.

import json
from booby import Model, fields

class TestModel(Model):
    fake = fields.String()
    option = fields.String(choices=['FOO', 'BAR', 'ANY', 'NONE'], default='NONE')

test_json_b = "{'fake': 'AMZ',}"
test_json_c = "{'fake': 'AMZ', 'option': null}"

tb = json.loads(test_json_b)
tc = json.loads(test_json_c)

mod_a = TestModel()
mod_b = TestModel(**ta)
mod_c = TestModel(**tc)

assert mod_a.option == 'NONE'
assert mod_a.is_valid == True
assert mod_b.option == 'NONE'
assert mod_b.is_valid == True
assert mod_c.option == None  # Undesired
assert mod_c.is_valid == False  # Undesired

Is there a method to remove the undesired behaviour above, in which a null breaks the option field?

I have added None to the list of choices for the field, but then this also makes the default redundant, given that the API returns null everytime it is unset.

A keyed collection, to deserialize dictionaries of model objects with arbitrary keys

Hi there. I'm working on modeling a new format which has a field that contains a dictionary of model objects. I think what I'm looking for would be something like a KeyedCollection or a Mapping.

The keys of the dictionary depend on the parent object's type, and aren't known in advance, so I can't just use a model.

Am I right that this isn't really possible now?

Add data serialization/deserialization

I think serialization/deserialization should be a major feature for the next release, so I opened this issue to start designing and implementing it.

We're going to use the @paulmelnikow's work from #15 pull request as a starting point.

Based on it, some desirable features are:

Create a new model instance from a serialized dict: from_dict classmethod.
Create a serialized dict from a model instance: to_dict method rather than perform serialization in the __iter__ method.
Update a model instance from a serialized dict: update_from_dict method (suggestions?)

As data serialization/deserialization I mean:

Model field name - dict item name mapping (as implemented in the pull request)
Data formatters. Ex: convert from str to int and vice versa.
Something more?

I think it would be appropriate to start with the first item in the list above and later implement the second item.

Remove u-prefixed unicode strings in doc examples

model.ModelMeta should probably use slots when setting initial attributes

Using slots can potentially save memory as well as prevent arbitrary attribute packing into a dictlike object which has both attribute and hash key access methods.

Really I think that model.Model should choose one or the other access method instead of having "multiple personality disorder" but that is probably something for another issue.

Add pickle support

It would be nice if booby supported the pickle protocol.

repr() causes maximum recursion depth error

The repr() function of self-referencing Models and Models referencing to other Models which reference to the first one, cause an "maximum recursion depth" error.
Because the repr() function is called for every referenced model over and over again.
For example if you have something like this:

class Address(Model):
    street = String()
    persons = Collection(Person)

 class Person(Model):
     name = String()
     addresses = Collection(Address)

And repr() is called on an instance of one of the models, it causes the "maximum recursion depth" error.

Model attributes are being set on the class, not the instance.

The docs suggest using an empty list for the default value of a field with a list of models as its validator.

Unfortunately, this will lead to undesirable behavior if the user tries to append models to an instance of the model containing the list field.

Background: http://effbot.org/zone/default-values.htm

from booby import models, fields, validators

class A(models.Model):
    a = fields.StringField(choices=['a1', 'a2',])


class B(models.Model):
    boolean = fields.BooleanField(default=False)
    a_list = fields.Field(validators.List(validators.Model(A)), default=[])
    a_not_list = fields.EmbeddedField(A)

From IPython:

In [7]: b = B()

In [8]: b.a_list.append(A(a='a1'))

In [9]: b.to
b.to_dict  b.to_json  

In [9]: b.to_dict()
Out[9]: {'a_list': [{'a': 'a1'}], 'a_not_list': None, 'boolean': False}

In [10]: b = B()

In [11]: b.to_dict()
Out[11]: {'a_list': [{'a': 'a1'}], 'a_not_list': None, 'boolean': False}

In [12]: b.a_list.append(A(a='a2'))

In [13]: b.to_dict()
Out[13]: {'a_list': [{'a': 'a1'}, {'a': 'a2'}], 'a_not_list': None, 'boolean': False}

Notice how the output from In[11] contains [{'a': 'a1'}] instead of the anticipated [].

Ideally and ListField would be added to the library as a work around. But at the very least the docs should be changed so that this isn't promoted.

Python 3 support

Add an Entity model

It would be nice if the repr for fields was a bit more detailed

When using inspection or even just manually inspecting _fields it would be nice if it was more obvious what validators and decorators might be in play. Particularly it would be nice to be able to see what fields are required and which are nullable. I think manipulating the repr property of Field objects as they are modified would be a relatively backwards compatible way of achieving this.

Validation errors should include field name.

Validation errors should include the fields name. It might make sense to add get_validation_errors method to the model class that loops through all the fields and stores the validation error in a dict with the key being the field name and the value being the error.

Make FieldError work as python's KeyError

Now the FieldError exception is raised with a (probably) more human readable message, but I think it makes much more sense to raise FieldError only with the field name like the python KeyError does for dict items.

Please use "collections.MutableSequence" instead of "list"

There are several locations where the code currently checks specifically if something is a list. If I want to pass in something which implements the list interface (via subclassing collections.MutableSequence), it currently will cause errors in validation since my class isn't a subclass of list. I've worked around the issue for now by also subclassing from list, but this is a hack that shouldn't be necessary.

If, when using isinstance, you could use MutableSequence instead of list, and while we're at it, MutableMapping in place of dict, that would make things much more flexible and help a lot in my specific use-case, and it will work just the same as it currently does for instances of list and dict.

Email Validation is way too restrictive

The regex '^\w+\@\w+\.[a-z]{2,3}$' is far too restrictive, as it doesn't allow for some very common things in emails, such as periods, dashes, plusses, etc.

[email protected] fails validation, as does [email protected], [email protected]. [email protected] passes, at least. All of these should pass, however.

I learned a long time ago to not even try to fully validate an email address just via regex. It's not really possible since you don't know if the domain really exists, or whether there is a mail server configured for that domain, and then of course, even if all that checks out, you don't know if that specific user exists on that server without trying to send an email there.

So, in conclusion, the email validator should really just be a basic sanity check along the lines of '^[^@]+\@[^@]+$' since users of the library are going to have to do further validation anyway, perhaps using something like Flanker's addresslib or just by sending a verification email.

TypeError: <main.A object at 0x105e0aa90> is not JSON serializable

Booby throws an exception if you use to_json on a model with a field containing a list of embedded models.

Example:

from booby import models, fields, validators

class A(models.Model):
    a = fields.StringField(choices=['a1', 'a2',])


class B(models.Model):
    boolean = fields.BooleanField(default=False)
    a_list = fields.Field(validators.List(validators.Model(A)), default=[])
    a_not_list = fields.EmbeddedField(A)


class C(models.Model):
    b = fields.EmbeddedField(B)

c = C(b=B(a_list=[A(a='a2')], a_not_list=A(a='a1')))
c.validate()
c.to_dict() # returns {'b': {'a_list': [<__main__.A at 0x105e0b790>], 'a_not_list': {'a': 'a1'}, 'boolean': False}}
c.to_json()

Traceback:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-37-006aec74bb24> in <module>()
     18 c.validate()
     19 c.to_dict() # returns {'b': {'a_list': [<__main__.A at 0x105df68d0>], 'boolean': False}}
---> 20 c.to_json()

/Users/erik/.virtualenvs/project/lib/python2.7/site-packages/booby/models.pyc in to_json(self)
    165         """
    166 
--> 167         return json.dumps(self.to_dict())

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.pyc in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, encoding, default, sort_keys, **kw)
    241         cls is None and indent is None and separators is None and
    242         encoding == 'utf-8' and default is None and not sort_keys and not kw):
--> 243         return _default_encoder.encode(obj)
    244     if cls is None:
    245         cls = JSONEncoder

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in encode(self, o)
    205         # exceptions aren't as detailed.  The list call should be roughly
    206         # equivalent to the PySequence_Fast that ''.join() would do.
--> 207         chunks = self.iterencode(o, _one_shot=True)
    208         if not isinstance(chunks, (list, tuple)):
    209             chunks = list(chunks)

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in iterencode(self, o, _one_shot)
    268                 self.key_separator, self.item_separator, self.sort_keys,
    269                 self.skipkeys, _one_shot)
--> 270         return _iterencode(o, 0)
    271 
    272 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.pyc in default(self, o)
    182 
    183         """
--> 184         raise TypeError(repr(o) + " is not JSON serializable")
    185 
    186     def encode(self, o):

TypeError: <__main__.A object at 0x105e0aa90> is not JSON serializable

Allow to use any callable as field validator

Release 0.3

Add a "computed" field

See #28

Creating a decoder mixin

What would be the best way to create my own decoder mixin? I would like to change the name of the field during decoding to match an API's custom field value instead of the real name.

Is this possible?

Model objects should apply validation on attribute set

It seems like it would be trivial to extend models.Model.setitem to use fields.Field subclass's validate method to throw a TypeError when a value is applied which doesn't pass validation.

Try to improve pylint warnings

This is a bit pedantic and maybe just bike shedding. But probably you are interested in improve some of this warnings:

pylint -f parseable booby/
tee pylint.out
No config file found, using default configuration
booby/__init__.py:1: [C] Missing docstring
booby/models.py:38: [C] Line too long (96/80)
booby/models.py:63: [W] Anomalous backslash in string: '\*'. String constant might be missing an r prefix.
booby/models.py:63: [W] Anomalous backslash in string: '\*'. String constant might be missing an r prefix.
booby/models.py:117: [W] Anomalous backslash in string: '\*'. String constant might be missing an r prefix.
booby/models.py:117: [W] Anomalous backslash in string: '\*'. String constant might be missing an r prefix.
booby/models.py:46: [C, ModelMeta] Missing docstring
booby/models.py:47: [C, ModelMeta.__new__] Metaclass class method __new__ should have 'mcs' as first argument
booby/models.py:51: [C, ModelMeta.__new__] Invalid name "v" for type variable (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:55: [C, ModelMeta.__new__] Invalid name "v" for type variable (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:89: [W, Model.__new__] Access to a protected member _data of a client class
booby/models.py:87: [W, Model.__new__] Unused argument 'args'
booby/models.py:87: [W, Model.__new__] Unused argument 'kwargs'
booby/models.py:94: [C, Model.__init__] Invalid name "v" for type variable (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:95: [E, Model.__init__] Instance of 'Model' has no '_fields' member
booby/models.py:100: [C, Model.__raise_field_error] Missing docstring
booby/models.py:105: [E, Model.__getitem__] Instance of 'Model' has no '_fields' member
booby/models.py:110: [C, Model.__setitem__] Invalid name "v" for type argument (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:110: [C, Model.__setitem__] Invalid name "v" for type variable (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:111: [E, Model.__setitem__] Instance of 'Model' has no '_fields' member
booby/models.py:130: [C, Model._update] Missing docstring
booby/models.py:131: [C, Model._update] Invalid name "v" for type variable (should match [a-z_][a-z0-9_]{2,30}$)
booby/models.py:143: [E, Model.validate] Instance of 'Model' has no '_fields' member
booby/models.py:150: [E, Model.to_dict] Instance of 'Model' has no '_fields' member
booby/models.py:62: [R, Model] Badly implemented Container, implements __getitem__, __setitem__ but not __delitem__, __len__
booby/validators.py:63: [C] Line too long (82/80)
booby/validators.py:146: [W] Anomalous backslash in string: '\w'. String constant might be missing an r prefix.
booby/validators.py:146: [W] Anomalous backslash in string: '\@'. String constant might be missing an r prefix.
booby/validators.py:146: [W] Anomalous backslash in string: '\w'. String constant might be missing an r prefix.
booby/validators.py:146: [W] Anomalous backslash in string: '\.'. String constant might be missing an r prefix.
booby/validators.py:55: [C, nullable.wrapper] Missing docstring
booby/validators.py:65: [C, Required.validate] Missing docstring
booby/validators.py:65: [R, Required.validate] Method could be a function
booby/validators.py:62: [R, Required] Too few public methods (1/2)
booby/validators.py:80: [C, In.validate] Missing docstring
booby/validators.py:70: [R, In] Too few public methods (1/2)
booby/validators.py:89: [C, String.validate] Missing docstring
booby/validators.py:89: [R, String.validate] Method could be a function
booby/validators.py:85: [R, String] Too few public methods (1/2)
booby/validators.py:98: [C, Integer.validate] Missing docstring
booby/validators.py:98: [R, Integer.validate] Method could be a function
booby/validators.py:94: [R, Integer] Too few public methods (1/2)
booby/validators.py:107: [C, Float.validate] Missing docstring
booby/validators.py:107: [R, Float.validate] Method could be a function
booby/validators.py:103: [R, Float] Too few public methods (1/2)
booby/validators.py:116: [C, Boolean.validate] Missing docstring
booby/validators.py:116: [R, Boolean.validate] Method could be a function
booby/validators.py:112: [R, Boolean] Too few public methods (1/2)
booby/validators.py:134: [C, Embedded.validate] Missing docstring
booby/validators.py:121: [R, Embedded] Too few public methods (1/2)
booby/validators.py:142: [C, Email] Missing docstring
booby/validators.py:142: [R, Email] Too few public methods (1/2)
booby/fields.py:84: [C] Line too long (87/80)
booby/fields.py:91: [C] Line too long (89/80)
booby/fields.py:98: [C] Line too long (85/80)
booby/fields.py:105: [C] Line too long (89/80)
booby/fields.py:131: [C] Line too long (85/80)
booby/fields.py:39: [W] Anomalous backslash in string: '\*'. String constant might be missing an r prefix.
booby/fields.py:69: [W, Field.__get__] Access to a protected member _data of a client class
booby/fields.py:73: [W, Field.__set__] Access to a protected member _data of a client class
booby/fields.py:75: [C, Field.validate] Missing docstring
booby/fields.py:38: [R, Field] Too few public methods (1/2)
booby/fields.py:80: [R, StringField] Too few public methods (1/2)
booby/fields.py:87: [R, IntegerField] Too few public methods (1/2)
booby/fields.py:94: [R, FloatField] Too few public methods (1/2)
booby/fields.py:101: [R, BooleanField] Too few public methods (1/2)
booby/fields.py:122: [W, EmbeddedField.__set__] Used * or ** magic
booby/fields.py:108: [R, EmbeddedField] Too few public methods (1/2)
booby/fields.py:127: [R, EmailField] Too few public methods (1/2)

Personally, I will skip the warnings about length var naming and probably others too. Also could be interesting run pep8 for test compilance.

Booby model does not validate properly with ListField attribute

Example:
class User(models.Model):
letters = fields.ListField(validators.String, validators.In(choices=['a', 'b']))

, or

class User(models.Model):
letters = fields.ListField(validators.List(validators.String, validators.In(choices=['a', 'b'])))

So if I create instance:
u = User(phones=[12345])
u.validate()

Nothing raised, because LIstField ignores all validators that are not subclass of Model.
It would be good to add that feature.

class TestListWithInners(object):

def test_when_list_field_raises_then_model_raises(self):
    with assert_raises(errors.ValidationError):
        c = Cell(m='a')
        c.validate()
    c = Cell(m=self.test_b)
    with assert_raises(errors.ValidationError):
        self.validator.validate(Cell(m=[1, 2, 3, 4]))
    c.validate()

def setup(self):
    self.test_a = ['a', 'b', 'c']
    self.test_b = [1, 2]
    self.test_c = ['a', 'ww']
    self.model = Cell(m=['a', 'b', 'c'])
    self.validator = validators.Model(Cell)