The ItemAdapter class is a wrapper for data container objects, providing a
common interface to handle objects of different types in an uniform manner,
regardless of their underlying implementation.
Currently supported types are:
dictscrapy.item.Itemdataclass-based classesattrs-based classes
- Python 3.5+
scrapy: optional, needed to interact withscrapyitemsdataclasses(stdlib in Python 3.7+, or its backport in Python 3.6): optional, needed to interact withdataclass-based itemsattrs: optional, needed to interact withattrs-based items
itemadapter is available on PyPI, it can be installed with pip:
pip install itemadapter
itemadapter is distributed under a BSD-3 license.
The following is a simple example using a dataclass object.
Consider the following type definition:
from dataclasses import dataclass
from itemadapter import ItemAdapter, is_item
@dataclass
class InventoryItem:
name: str
price: float
stock: intThe ItemAdapter object can be treated much like a dictionary:
>>> obj = InventoryItem(name='foo', price=20.5, stock=10)
>>> is_item(obj)
True
>>> adapter = ItemAdapter(obj)
>>> len(adapter)
3
>>> adapter["name"]
'foo'
>>> adapter.get("price")
20.5The wrapped object is modified in-place:
>>> adapter["name"] = "bar"
>>> adapter.update({"price": 12.7, "stock": 9})
>>> adapter.item
InventoryItem(name='bar', price=12.7, stock=9)
>>> adapter.item is obj
TrueThe ItemAdapter class provides the asdict method, which converts
nested items recursively. Consider the following example:
from dataclasses import dataclass
from itemadapter import ItemAdapter
@dataclass
class Price:
value: int
currency: str
@dataclass
class Product:
name: str
price: Price>>> item = Product("Stuff", Price(42, "UYU"))
>>> adapter = ItemAdapter(item)
>>> adapter.asdict()
{'name': 'Stuff', 'price': {'currency': 'UYU', 'value': 42}}Note that just passing an adapter object to the dict built-in also works,
but it doesn't traverse the object recursively converting nested items:
>>> dict(adapter)
{'name': 'Stuff', 'price': Price(value=42, currency='UYU')}class itemadapter.adapter.ItemAdapter(item: Any)
ItemAdapter implements the
MutableMapping interface,
providing a dict-like API to manipulate data for the object it wraps
(which is modified in-place).
Some additional methods are available:
get_field_meta(field_name: str) -> MappingProxyType
Return a MappingProxyType
object, which is a read-only mapping with metadata about the given field. If the item class does not
support field metadata, or there is no metadata for the given field, an empty object is returned.
The returned value is taken from the following sources, depending on the item type:
scrapy.item.Fieldforscrapy.item.Itemsdataclasses.field.metadatafordataclass-based itemsattr.Attribute.metadataforattrs-based items
field_names() -> KeysView
Return a keys view with the names of all the defined fields for the item.
asdict() -> dict
Return a dict object with the contents of the adapter. This works slightly different than
calling dict(adapter), because it's applied recursively to nested items (if there are any).
itemadapter.utils.is_item(obj: Any) -> bool
Return True if the given object belongs to one of the supported types,
False otherwise.
scrapy.item.Item, dataclass and attrs objects allow the inclusion of
arbitrary field metadata, which can be retrieved with the
ItemAdapter.get_field_meta method. The definition procedure depends on the
underlying type.
>>> from scrapy.item import Item, Field
>>> from itemadapter import ItemAdapter
>>> class InventoryItem(Item):
... name = Field(serializer=str)
... value = Field(serializer=int, limit=100)
...
>>> adapter = ItemAdapter(InventoryItem(name="foo", value=10))
>>> adapter.get_field_meta("name")
mappingproxy({'serializer': <class 'str'>})
>>> adapter.get_field_meta("value")
mappingproxy({'serializer': <class 'int'>, 'limit': 100})>>> from dataclasses import dataclass, field
>>> @dataclass
... class InventoryItem:
... name: str = field(metadata={"serializer": str})
... value: int = field(metadata={"serializer": int, "limit": 100})
...
>>> adapter = ItemAdapter(InventoryItem(name="foo", value=10))
>>> adapter.get_field_meta("name")
mappingproxy({'serializer': <class 'str'>})
>>> adapter.get_field_meta("value")
mappingproxy({'serializer': <class 'int'>, 'limit': 100})>>> import attr
>>> @attr.s
... class InventoryItem:
... name = attr.ib(metadata={"serializer": str})
... value = attr.ib(metadata={"serializer": int, "limit": 100})
...
>>> adapter = ItemAdapter(InventoryItem(name="foo", value=10))
>>> adapter.get_field_meta("name")
mappingproxy({'serializer': <class 'str'>})
>>> adapter.get_field_meta("value")
mappingproxy({'serializer': <class 'int'>})>>> from scrapy.item import Item, Field
>>> from itemadapter import ItemAdapter
>>> class InventoryItem(Item):
... name = Field()
... price = Field()
...
>>> item = InventoryItem(name="foo", price=10)
>>> adapter = ItemAdapter(item)
>>> adapter.item is item
True
>>> adapter["name"]
'foo'
>>> adapter["name"] = "bar"
>>> adapter["price"] = 5
>>> item
{'name': 'bar', 'price': 5}>>> from itemadapter import ItemAdapter
>>> item = dict(name="foo", price=10)
>>> adapter = ItemAdapter(item)
>>> adapter.item is item
True
>>> adapter["name"]
'foo'
>>> adapter["name"] = "bar"
>>> adapter["price"] = 5
>>> item
{'name': 'bar', 'price': 5}>>> from dataclasses import dataclass
>>> from itemadapter import ItemAdapter
>>> @dataclass
... class InventoryItem:
... name: str
... price: int
...
>>> item = InventoryItem(name="foo", price=10)
>>> adapter = ItemAdapter(item)
>>> adapter.item is item
True
>>> adapter["name"]
'foo'
>>> adapter["name"] = "bar"
>>> adapter["price"] = 5
>>> item
InventoryItem(name='bar', price=5)>>> import attr
>>> from itemadapter import ItemAdapter
>>> @attr.s
... class InventoryItem:
... name = attr.ib()
... price = attr.ib()
...
>>> item = InventoryItem(name="foo", price=10)
>>> adapter = ItemAdapter(item)
>>> adapter.item is item
True
>>> adapter["name"]
'foo'
>>> adapter["name"] = "bar"
>>> adapter["price"] = 5
>>> item
InventoryItem(name='bar', price=5)
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent