Decoding an encoded record wraps the produced map in a list about avrora HOT 5 CLOSED

Hey @kevinbader sorry for a long response, but I've updated master with updated typespec for Avrora.Encoder.decode/1 and Avrora.Encoder.decode/2

Update: I plan to fix issue #27 and then make a release, so I will hand a while in a master branch

from avrora.

Hi @kevinbader, this is not a bug. When you don't have a schema registry, by default Avrora is using Avro Object Container FIles format, which assumes that you have always a list.

The reason to use OCF is the assumption that your data should be always decodable and since OCF includes schema in a header of a message, you can always decode it.

You can fine that under format section in the README.

Here is how you can make it plain:

iex(3)> Avrora.encode(%{"baz" => "test"}, schema_name: "foo.Bar")  
09:18:14.875 [debug] reading schema `foo.Bar` from the file ./tmp/schemas/foo/Bar.avsc
{:ok,
 <<79, 98, 106, 1, 3, 144, 2, 20, 97, 118, 114, 111, 46, 99, 111, 100, 101, 99,
   8, 110, 117, 108, 108, 22, 97, 118, 114, 111, 46, 115, 99, 104, 101, 109, 97,
   212, 1, 123, 34, 110, 97, 109, 101, 115, 112, 97, 99, 101, ...>>}
iex(4)> Avrora.encode(%{"baz" => "test"}, schema_name: "foo.Bar", format: :plain)
{:ok, "\btest"}
iex(5)> Avrora.decode("\btest", schema_name: "foo.Bar")
{:ok, %{"baz" => "test"}}

🤔 not sure why in :plain format you see it as a string and not as a charlist, maybe it's an iex thing but just in case will check

UPD: yes, it's a correct plain display

iex(6)> String.to_charlist("\btest")
'\btest'

from avrora.

Hi @Strech,

thanks for your quick response! From the docs I couldn't really tell that using OCF would always result in a list, even in case there is only a single map in the container. Given what you wrote, however, I propose to change the spec of encode to reflect this; currently Dialyzer thinks it always returns a binary and never a list:

@spec encode(map(), keyword(String.t())) :: {:ok, binary()} | {:error, term()}

from avrora.

Evening @kevinbader,

From the docs I couldn't really tell that using OCF would always result in a list

I re-read the docs and what I can quote from the docs:

Avro includes a simple object container file format. A file has a schema, and all objects stored in the file must be written according to that schema, using binary encoding. Objects are stored in blocks that may be compressed

A file data block consists of:

• A long indicating the count of objects in this block.
• A long indicating the size in bytes of the serialized objects in the current block, after any codec is applied
• The serialized objects. If a codec is specified, this is compressed by that codec.

As you can see, it always plural, as I understand it a block of objects can include just one for sure, but it's considered to be a container of objects and in Elixir/Erlang world it's a List

I propose to change the spec of encode to reflect this; currently Dialyzer thinks it always returns a binary and never a list:

Completely agree, but not encode spec, decode you mean. Because encode always return binary and no List is involved. Will do it for decode 👍

from avrora.

You're completely right - thanks!

from avrora.