Git Product home page Git Product logo

Comments (14)

amirotin avatar amirotin commented on May 18, 2024 1

Thanks again 👍
Now i got correct result in both directions!

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024 1

@Fatal1ty cool I like the decorator approach and it works. But can we get rid of the redundant pointType ("1") in either the decorator arg or default variable?

You can use the pointType value:

def register_point(class_):
    MapPoint.__points__[class_.__dict__["pointType"]] = class_
    return class_


@register_point
class Point1:
    pointType: int = 1

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

Hi @amirotin

There would be no problem if Point1 and Point4 had differences in the names or types of their fields. They have different p and r fields but since they both have a default value, these two classes could be deserialized from the same input data.

I'm going to implement discriminated unions, which will allow choosing the right class based on the value of a field and solve this problem completely. But for now there is a couple of workarounds:

  1. Change type of p and r fields to make them required
  2. Write a deserialization method for Union[Point1,Point4] by yourself:
def deserialize_point(value: Dict) -> Union[Point1, Point4]:
    pointType = value.get("pointType")
    if pointType == 1:
        return Point1.from_dict(value)
    elif pointType == 4:
        return Point4.from_dict(value)
    else:
        raise ValueError(f"Unknown pointType {pointType}")

@dataclass(slots=True)
class MapData(BaseRequest):
    pointList: List[Union[Point1, Point4]]

    class Config(BaseConfig):
        serialization_strategy = {
            Union[Point1, Point4]: {
                "deserialize": deserialize_point
            }
        }

from mashumaro.

amirotin avatar amirotin commented on May 18, 2024

@Fatal1ty thanks for idea!
I've checked my input data - i've 55 (later will be more) different point types and unfortunately many of them have same letter p, r, m, b and etc. So option 1 won't work for me
Actualy i was thinking about making custom SerializableType class for that. But serialization_strategy will be easier. As far as i understand - i can replace Union[Point1, Poin2, ....] with Any. And it will work same way?

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

As far as i understand - i can replace Union[Point1, Poin2, ....] with Any. And it will work same way?

Yes, you can override deserialization of whatever you want. So, you can use List[Any] and override deserialization for Any.

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

Another solution is to use SerializableType, as you've noticed:

@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
    x: int
    y: int
    pointType: int

    def _serialize(self):
        return self.to_dict()

    @classmethod
    def _deserialize(cls, value: Dict):
        if value.get("pointType") == 1:
            return Point1.from_dict(value)
        elif value.get("pointType") == 4:
            return Point4.from_dict(value)
        ...

@dataclass(slots=True)
class MapData(BaseRequest):
    pointList: List[MapPoint]

You can have a registry like Dict[int, MapPoint] mapping to store all point types. For example, each point type can be registered with a decorator.

from mashumaro.

amirotin avatar amirotin commented on May 18, 2024

@Fatal1ty thanks! worked like a charm!

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

With a bit of black dirty magic in a simple case a universal deserialization method for all point types would look like this:

current_module = sys.modules[__name__]

...

@classmethod
def _deserialize(cls, value: Dict):
    pointType = value.get("pointType")
    try:
        point_cls = getattr(current_module, f'Point{pointType}')
    except AttributeError:
        raise ValueError(f"Unknown pointType: {pointType}")

    return point_cls.from_dict(value)

from mashumaro.

amirotin avatar amirotin commented on May 18, 2024

If we talk about json, should it be from_json instead of from_dict? And probably value is str, not Dict?

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

It should be from_dict. from_json is just a simple wrapper.

from mashumaro.

th0ger avatar th0ger commented on May 18, 2024

You can have a registry like Dict[int, MapPoint] mapping to store all point types. For example, each point type can be registered with a decorator.

Sounds interesting with a decorator, could you provide an example?

from mashumaro.

Fatal1ty avatar Fatal1ty commented on May 18, 2024

You can have a registry like Dict[int, MapPoint] mapping to store all point types. For example, each point type can be registered with a decorator.

Sounds interesting with a decorator, could you provide an example?

Yes, sure. It might look something like this:

def register_point(point_number: int):
    def decorator(class_):
        MapPoint.__points__[point_number] = class_
        return class_
    return decorator


@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
    __points__ = {}
    
    ...

    @classmethod
    def _deserialize(cls, value: Dict):
        return cls.__points__[value["pointType"]].from_dict(value)

@register_point(1)
@dataclass(slots=True)
class Point1(MapPoint):
    pointType: int = 1
    ...

@dataclass(slots=True)
class MapData(BaseRequest):
    pointList: List[MapPoint]

Another option is to use __init_subclass__ method since every PointX is meant to be a subclass of MapPoint:

@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
    __points__ = {}

    def __init_subclass__(cls, **kwargs):
        MapPoint.__points__[cls.__dict__["pointType"]] = cls

    ...

    @classmethod
    def _deserialize(cls, value: Dict):
        return cls.__points__[value["pointType"]].from_dict(value)

@dataclass(slots=True)
class Point1(MapPoint):
    pointType: int = 1
    ...

@dataclass(slots=True)
class MapData(BaseRequest):
    pointList: List[MapPoint]

from mashumaro.

th0ger avatar th0ger commented on May 18, 2024

@Fatal1ty cool I like the decorator approach and it works. But can we get rid of the redundant pointType ("1") in either the decorator arg or default variable?

from mashumaro.

mishamsk avatar mishamsk commented on May 18, 2024

I may open source a library I wrote on top of Mashumaro, that also includes serialization extension, that among other things sovles this issue. Here is a snippet from the body of my serialization mixin:

def __post_serialize__(self, d: dict[str, Any]) -> dict[str, Any]:
        out = {"class": self.__class__.__name__}
        for k, v in d.items():
            if k.startswith("_"):
                continue
            out[k] = v
        return out

    def _serialize(self) -> dict[str, Any]:
        if DataClassSerializeMixin.__mashumaro_dialect is not None:
            return self.to_dict(dialect=DataClassSerializeMixin.__mashumaro_dialect)
        else:
            return self.to_dict()

    @classmethod
    def _deserialize(cls: Type[T], value: dict[str, Any]) -> T:
        class_name = value.pop("class", None)

        if class_name is None:
            raise ValueError("Missing class name ('class' field) in source")

        clazz = TYPES.get(class_name, None)
        if clazz is None:
            raise ValueError(f"Unknown class name: {class_name}")

        if DataClassSerializeMixin.__mashumaro_dialect is not None:
            return cast(
                T, clazz.from_dict(value, dialect=DataClassSerializeMixin.__mashumaro_dialect)
            )
        else:
            return cast(T, clazz.from_dict(value))

    def __init_subclass__(cls, **kwargs: Any):
        if cls.__name__ in TYPES:
            package = "Unknown"
            module = inspect.getmodule(TYPES[cls.__name__])
            if module is not None:
                package = str(module.__package__)

            raise ValueError(
                f"DataClassSerializeMixin subclass <{cls.__name__}> is already defined in package <{package}>. Please use a different name."
            )

        TYPES[cls.__name__] = cls
        return super().__init_subclass__(**kwargs)

It also has some custom to/from_dict/json/msgpack wrappers with more complicated dialects supports, which is not in that snippet. Works like a charm. no decorators necessary

from mashumaro.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.