MapAttribute serialisation issues about falcano HOT 2 CLOSED

From further digging, it seems like the data is being serilized twice before upload.

I checked this by re-running the code above, then also running a print
of

n = Item().data.serialize(Item().data.serialize(dict(hello="message", nested={"hello": "message"}, enabled=False, number=0)))
print(n)

>>

{'hello': {'M': {'S': {'S': 'message'}}}, 'nested': {'M': {'M': {'M': {'hello': {'M': {'S': {'S': 'message'}}}}}}}, 'enabled': {'M': {'BOOL': {'BOOL': False}}}, 'number': {'M': {'N': {'N': 0}}}}

Looking at the tests for map attribute, it's all based on mock data but has is there any tests of data that is stored in Dynamo then retrieved again?

will keep digging

from falcano.

And a bit more digging -

Looking at the api call to put_item, even though serialize is defaulted to false, it looks like Map Attributes are still serialized

If I print out kwargs that are passed to put_item under the Model class, i see this:

new = Item(hash_key=f'testItem#{id}', range_key="testRange#3749189f-3d4c-4937-b31a-924bc84b366c",
        data=dict(hello="message", nested={"hello": "message"}, enabled=False, number=0)
    )
print(new.save())

>>>

{'TableName': 'my-table', 'Item': {'Type': 'testItem', 'data': {'hello': {'S': 'message'}, 'nested': {'M': {'hello': {'S': 'message'}}}, 'enabled': {'BOOL': False}, 'number': {'N': 0}}, 'pk': 'testItem#<built-in function id>', 'sk': 'testRange#3749189f-3d4c-4937-b31a-924bc84b366c'}, 'ReturnValues': 'NONE'}

The issue seems to be that the put_item API for the Table Resource doesn't require any serialization and does it automatically.
I tested this by running the API calls directly with my raw input and it stores as expected in Dynamo

 import boto3

table = boto3.resource('dynamodb')
table = table.Table('my-table')

table.put_item(Item={'Type': 'testItem', 'data': dict(hello="message", nested={"hello": "message"}, enabled=False, number=0), 'pk': 'testItem#cca06093-e0c6-49a6-93d5-266bed23cad1', 'sk': 'testRange#3749189f-3d4c-4937-b31a-924bc84b366c'})

The result is a successful API call and the Dynamo console shows the expected outcome:

{
  "pk": {
    "S": "testItem#cca06093-e0c6-49a6-93d5-266bed23cad1"
  },
  "sk": {
    "S": "testRange#3749189f-3d4c-4937-b31a-924bc84b366c"
  },
  "Type": {
    "S": "testItem"
  },
  "data": {
    "M": {
      "number": {
        "N": "0"
      },
      "hello": {
        "S": "message"
      },
      "nested": {
        "M": {
          "hello": {
            "S": "message"
          }
        }
      },
      "enabled": {
        "BOOL": false
      }
    }
  }
}

Fix appears to stop whatever serilizes the MapAttribute before put_item is called as this is an unnecessary step for the current API choice - Either that or change the API from the Resource Table.put_item to the DynamoDB Client put_item, which does need serialization - however this would mean all values need to be serialized, not just Map.

I'll see if I can fix and submit a PR

from falcano.