Change build_many to return a generator instead list about simple-model HOT 4 CLOSED

I took the opportunity and already created the PR #16

from simple-model.

@fvlima I've been thinking about this issue and I've come to the conclusion that using generators only avoids the converstion to iterable that's inevitable on most cases. That's because models were designed to be used/accessed many times, and generators aren't the best choice for this case.

import typing

from simple_model import Model


class Product(Model):
    title: str


class ShoppingCart(Model):
    items = typing.List[Products]


products_data = [{'title': 'Cool Gadget #' + i} for i in range(1, 11)]
products = Product.build_many(products_data)  # returns generator. the slow conversion is avoided

cart = ShoppingCart(items=products)
cart.items  # generator
cart.clean()  # conversion happens here
cart.validate()  # or here
dict(cart)  # here too
cart.items = list(cart.items)  # maybe here (more complicated to use)

Generators really shine when used on for loops (or are consumed step by step once). For example, the range and reversed built-in functions are used, on most cases, on for loops and aren't used again. The same doesn't apply to models and its fields.

In fact, the Model.build_many classmethod was implemented to solve problems such as the example above where at some point the conversion will happen.

But I believe your suggestion is valid and there might be cases (IMHO not most cases though) where using generators is the solution with best performance. So for these specific circumstances we may use the model_many_builder [1] over Model.build_many. The model_many_builder was implemented to return a list, but I believe it should be changed to return generators. Its usage would bee something like:

from simple_model.builder import model_many_builder

# using previous example objects
products = model_many_builder(products_data, cls=Product)
for product in products:  # generator
    heavy_operation(product)  # something slow or "unreliable" as making requests to external APIs

What do you think @fvlima?

[1] https://github.com/lamenezes/simple-model/blob/master/simple_model/builder.py#L58

from simple-model.

About the Model.build_many I thought about it, and what I thought was that who needed to validate the objects on the fly, and have a small portion of data to build, just can do a cast like this list(models), just like you suggested in your previus comment. So, the validation would occur and it's allows to iterate many times as necessary in this list.

But I understand what you say and this proposal to use the model_many_builder is valid too and can solve my particular situation.

from simple-model.

This feature is going to be released on simple model version 2.0

from simple-model.