Binary Avro representation fails about avro-scala-macro-annotations HOT 3 CLOSED

Hi Sutram,

Thanks for trying out this library.

Use the schema as an argument to the datumwriter:

    val sw = new SpecificDatumWriter[AtomicSlice](AtomicSlice.SCHEMA$)

The above is how I see SpecificDatumWriters used in this Avro test and in an hbase example (according to a quick web search). Without specifying a schema, I get the same error in Java too(see below).

I consider this an Avro gotcha because SpecificDatumWriters need to be constructed differently for writing files than for writing straight binary, but we get no help from the compiler. All avro writing requires a schema at some point. DataFileWriters require a schema be specified upon file creation, so it doesn't matter if their datumwriters were constructed with schemas or not. Meanwhile, Encoders don't allow a schema to be specified: if its datumwriter was constructed without one, its schema never gets set and hence the null pointer exception when a write is actually perfomed.

At least this is how things appear to me; although I don't exactly see where things fall through to generic... perhaps the schema didn't match a class name because the schema was null, but why didn't that show up in the stack trace?

Anyways, hope this helps,
-Julian

The error in java:

mvn exec:java -Dexec.mainClass="com.example.Main"  
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building avro-encoder-example 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ avro-encoder-example ---
[WARNING] 
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:87)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at com.example.Main.main(Main.java:38)

from avro-scala-macro-annotations.

Thanks, that fixed it!

I really appreciate this project.

I have a couple more issues I wanted to run by you so I'll create those as separate issues.

from avro-scala-macro-annotations.

great!

from avro-scala-macro-annotations.