An Avro encoding/decoding library written in pure Elixir.
The docs can be found on hex.pm
def deps do
[{:avro_ex, "~> 2.0"}]
endAvro uses schemas to define the shape and contract for data. The schemas that your application uses may be defined locally, or may come from a Schema Registry.
In either case, the first step is to decode a schema defined as JSON or Elixir terms into a t:AvroEx.Schema.t/0
iex> AvroEx.decode_schema!(["int", "string"])
%AvroEx.Schema{
context: %AvroEx.Schema.Context{names: %{}},
schema: %AvroEx.Schema.Union{
possibilities: [
%AvroEx.Schema.Primitive{metadata: %{}, type: :int},
%AvroEx.Schema.Primitive{metadata: %{}, type: :string}
]
}
}AvroEx will automatically detect Elixir terms or JSON, so you can decode JSON schemas directly
iex> AvroEx.decode_schema!("[\"int\",\"string\"]")
%AvroEx.Schema{
context: %AvroEx.Schema.Context{names: %{}},
schema: %AvroEx.Schema.Union{
possibilities: [
%AvroEx.Schema.Primitive{metadata: %{}, type: :int},
%AvroEx.Schema.Primitive{metadata: %{}, type: :string}
]
}
}When writing an Avro schema, it is helpful to get feedback on unrecognized fields. For this purpose,
it is recommended to use the :strict option to provide additional checks. Note that it is not
recommended to use this option in production when pulling externally defined schemas, as they may
have published a schema with looser validations.
iex> AvroEx.decode_schema!(%{"type" => "map", "values" => "int", "bogus" => "value"}, strict: true)
** (AvroEx.Schema.DecodeError) Unrecognized schema key `bogus` for AvroEx.Schema.Map in %{"bogus" => "value", "type" => "map", "values" => "int"}
(avro_ex 1.2.0) lib/avro_ex/schema/parser.ex:43: AvroEx.Schema.Parser.parse!/2When publishing Avro data, it first must be encoded using the schema.
iex> schema = AvroEx.decode_schema!(%{
"type" => "record",
"name" => "MyRecord",
"fields" => [
%{"name" => "a", "type" => "int"},
%{"name" => "b", "type" => "string"},
]
})
iex> AvroEx.encode!(schema, %{a: 1, b: "two"})
<<2, 6, 116, 119, 111>When receiving Avro data, decode it using the schema
iex> AvroEx.decode!(schema, <<2, 6, 116, 119, 111>>)
%{"a" => 1, "b" => "two"}AvroEx also supports encoding schemas back to JSON. This may be needed when registering schemas or
serializing them to disk.
iex> AvroEx.encode_schema(schema)
"{\"fields\":[{\"name\":\"a\",\"type\":{\"type\":\"int\"}},{\"name\":\"b\",\"type\":{\"type\":\"string\"}}],\"name\":\"MyRecord\",\"type\":\"record\"}"Additionally, schemas can be encoded to Parsing Canonical Form using
the :canonical option.
iex> AvroEx.encode_schema(schema, canonical: true)
"{\"name\":\"MyRecord\",\"type\":\"record\",\"fields\":[{\"name\":\"a\",\"type\":\"int\"},{\"name\":\"b\",\"type\":\"string\"}]}"For testing convenience, AvroEx.encodable?/2 is exported to check if data can be
encoded against the given schema. Note that in production scenarios, it is not
recommended to use this function.
defmodule MyModule.Test do
use ExUnit.Case
setup do
data = ...
schema = ...
{:ok, %{data: data, schema: schema}}
end
describe "my_function/1" do
test "builds a structure that can be encoded with our avro schema", context do
result = MyModule.my_function(context.data)
assert AvroEx.encodable?(context.schema, result)
end
end
end
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent