Comments (14)
Thanks again 👍
Now i got correct result in both directions!
from mashumaro.
@Fatal1ty cool I like the decorator approach and it works. But can we get rid of the redundant pointType ("1") in either the decorator arg or default variable?
You can use the pointType
value:
def register_point(class_):
MapPoint.__points__[class_.__dict__["pointType"]] = class_
return class_
@register_point
class Point1:
pointType: int = 1
from mashumaro.
Hi @amirotin
There would be no problem if Point1
and Point4
had differences in the names or types of their fields. They have different p
and r
fields but since they both have a default value, these two classes could be deserialized from the same input data.
I'm going to implement discriminated unions, which will allow choosing the right class based on the value of a field and solve this problem completely. But for now there is a couple of workarounds:
- Change type of
p
andr
fields to make them required - Write a deserialization method for
Union[Point1,Point4]
by yourself:
def deserialize_point(value: Dict) -> Union[Point1, Point4]:
pointType = value.get("pointType")
if pointType == 1:
return Point1.from_dict(value)
elif pointType == 4:
return Point4.from_dict(value)
else:
raise ValueError(f"Unknown pointType {pointType}")
@dataclass(slots=True)
class MapData(BaseRequest):
pointList: List[Union[Point1, Point4]]
class Config(BaseConfig):
serialization_strategy = {
Union[Point1, Point4]: {
"deserialize": deserialize_point
}
}
from mashumaro.
@Fatal1ty thanks for idea!
I've checked my input data - i've 55 (later will be more) different point types and unfortunately many of them have same letter p
, r
, m
, b
and etc. So option 1 won't work for me
Actualy i was thinking about making custom SerializableType
class for that. But serialization_strategy
will be easier. As far as i understand - i can replace Union[Point1, Poin2, ....] with Any. And it will work same way?
from mashumaro.
As far as i understand - i can replace Union[Point1, Poin2, ....] with Any. And it will work same way?
Yes, you can override deserialization of whatever you want. So, you can use List[Any]
and override deserialization for Any
.
from mashumaro.
Another solution is to use SerializableType
, as you've noticed:
@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
x: int
y: int
pointType: int
def _serialize(self):
return self.to_dict()
@classmethod
def _deserialize(cls, value: Dict):
if value.get("pointType") == 1:
return Point1.from_dict(value)
elif value.get("pointType") == 4:
return Point4.from_dict(value)
...
@dataclass(slots=True)
class MapData(BaseRequest):
pointList: List[MapPoint]
You can have a registry like Dict[int, MapPoint]
mapping to store all point types. For example, each point type can be registered with a decorator.
from mashumaro.
@Fatal1ty thanks! worked like a charm!
from mashumaro.
With a bit of black dirty magic in a simple case a universal deserialization method for all point types would look like this:
current_module = sys.modules[__name__]
...
@classmethod
def _deserialize(cls, value: Dict):
pointType = value.get("pointType")
try:
point_cls = getattr(current_module, f'Point{pointType}')
except AttributeError:
raise ValueError(f"Unknown pointType: {pointType}")
return point_cls.from_dict(value)
from mashumaro.
If we talk about json, should it be from_json
instead of from_dict
? And probably value
is str
, not Dict
?
from mashumaro.
It should be from_dict
. from_json
is just a simple wrapper.
from mashumaro.
You can have a registry like
Dict[int, MapPoint]
mapping to store all point types. For example, each point type can be registered with a decorator.
Sounds interesting with a decorator, could you provide an example?
from mashumaro.
You can have a registry like
Dict[int, MapPoint]
mapping to store all point types. For example, each point type can be registered with a decorator.Sounds interesting with a decorator, could you provide an example?
Yes, sure. It might look something like this:
def register_point(point_number: int):
def decorator(class_):
MapPoint.__points__[point_number] = class_
return class_
return decorator
@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
__points__ = {}
...
@classmethod
def _deserialize(cls, value: Dict):
return cls.__points__[value["pointType"]].from_dict(value)
@register_point(1)
@dataclass(slots=True)
class Point1(MapPoint):
pointType: int = 1
...
@dataclass(slots=True)
class MapData(BaseRequest):
pointList: List[MapPoint]
Another option is to use __init_subclass__
method since every PointX
is meant to be a subclass of MapPoint
:
@dataclass(slots=True)
class MapPoint(BaseRequest, SerializableType):
__points__ = {}
def __init_subclass__(cls, **kwargs):
MapPoint.__points__[cls.__dict__["pointType"]] = cls
...
@classmethod
def _deserialize(cls, value: Dict):
return cls.__points__[value["pointType"]].from_dict(value)
@dataclass(slots=True)
class Point1(MapPoint):
pointType: int = 1
...
@dataclass(slots=True)
class MapData(BaseRequest):
pointList: List[MapPoint]
from mashumaro.
@Fatal1ty cool I like the decorator approach and it works. But can we get rid of the redundant pointType ("1") in either the decorator arg or default variable?
from mashumaro.
I may open source a library I wrote on top of Mashumaro, that also includes serialization extension, that among other things sovles this issue. Here is a snippet from the body of my serialization mixin:
def __post_serialize__(self, d: dict[str, Any]) -> dict[str, Any]:
out = {"class": self.__class__.__name__}
for k, v in d.items():
if k.startswith("_"):
continue
out[k] = v
return out
def _serialize(self) -> dict[str, Any]:
if DataClassSerializeMixin.__mashumaro_dialect is not None:
return self.to_dict(dialect=DataClassSerializeMixin.__mashumaro_dialect)
else:
return self.to_dict()
@classmethod
def _deserialize(cls: Type[T], value: dict[str, Any]) -> T:
class_name = value.pop("class", None)
if class_name is None:
raise ValueError("Missing class name ('class' field) in source")
clazz = TYPES.get(class_name, None)
if clazz is None:
raise ValueError(f"Unknown class name: {class_name}")
if DataClassSerializeMixin.__mashumaro_dialect is not None:
return cast(
T, clazz.from_dict(value, dialect=DataClassSerializeMixin.__mashumaro_dialect)
)
else:
return cast(T, clazz.from_dict(value))
def __init_subclass__(cls, **kwargs: Any):
if cls.__name__ in TYPES:
package = "Unknown"
module = inspect.getmodule(TYPES[cls.__name__])
if module is not None:
package = str(module.__package__)
raise ValueError(
f"DataClassSerializeMixin subclass <{cls.__name__}> is already defined in package <{package}>. Please use a different name."
)
TYPES[cls.__name__] = cls
return super().__init_subclass__(**kwargs)
It also has some custom to/from_dict/json/msgpack wrappers with more complicated dialects supports, which is not in that snippet. Works like a charm. no decorators necessary
from mashumaro.
Related Issues (20)
- Switch to ruff from flake8
- Comprehension inlining on Python 3.12.0rc1 breaks some tests HOT 1
- Add support for Python 3.12
- Add support for PEP 696 HOT 1
- Can immutable dataclasses be hashed as dictionary keys? HOT 4
- Add an option to omit values equal to defaults on serialization
- Add an option to avoid collection data types copying
- Add support for LiteralString
- Generic dataclasses with Hashable typevar bound do not work HOT 3
- Query string support HOT 5
- Benchmark is not working in master branch HOT 2
- omit_default=True evaluates default_factory value at declaration time which can cause a circular reference HOT 3
- Multiple aliases for field HOT 6
- "`dict[str, T]` as a field type is not supported" HOT 2
- InitVar with no default value HOT 6
- The types defined inside the function result in syntactically invalid generated code
- Allow for more complex logic for subclass Discrimnator field matching HOT 3
- TypedDicts not working with `from __future__ import annotations` HOT 2
- omit_default breaks IntFlag serialization HOT 1
- Using Union with int/float casts to whichever appears first HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mashumaro.