tickaroo / tikxml Goto Github PK
View Code? Open in Web Editor NEWModern XML Parser for Android
License: Apache License 2.0
Modern XML Parser for Android
License: Apache License 2.0
We should bundle the core
and annotation
module into a android library .aar
and add the proguard rules.
It would make sense to use gradle instead of maven as build system for the whole project ...
For example, this can be replaced with
private static final ByteString CDATA_CLOSE = ByteString.decodeUtf8("]]>");
// ...
long index = source.indexOf(CDATA_CLOSE);
if (index == -1) throw ...
return index;
Our requirements to the parser:
<element a="true" b="2015-12-15" />
. We need "type adapters" for attributes as well. However, I'm not sure if the same type adapters can be used for both, xml elements and xml element attributes. Maybe we need some kind of "type converter" as addition to "type adapters" for xml element attributes.Ok, so I have written down a proposal for the annotation API.
The proposal can be read here: https://github.com/Tickaroo/tikxml/blob/master/docs/AnnotatingModelClasses.md
Some parts of the API are inconsistent, but I don't know how to do it better way yet. XML is such a ugly format.
Inconsistency (maybe you find some more inconsistencies not listed here):
@ChildElement
vs. @ChildElementList
(list has an inline option while @ChildElement
has none), but do we really need two annotations for the similar thing? Can we use @ChildElement
on Lists, if yes, what about inlining?@PropertyElement
vs. @TextContent
(plus @Path
)What do you think about the name of the annotations?
This is with 0.8.9-SNAPSHOT
If you have the xml
<foo
></foo>
it will fail to parse with
reader.beginElement();
reader.nextElementName();
reader.endElement(); // fails here
with
java.io.IOException: Expected a closing element tag </foo
> but found </foo> at path /foo
/text()
at com.tickaroo.tikxml.XmlReader.syntaxError(XmlReader.java:850)
at com.tickaroo.tikxml.XmlReader.doPeek(XmlReader.java:276)
at com.tickaroo.tikxml.XmlReader.endElement(XmlReader.java:344)
It appears that it's incorrectly including a newline ('\n') char in the element tag name, causing it to fail to match the ending tag.
While the placement of the newline above may look odd, it's pretty common when attributes are included. Ex:
<foo
bar1="baz1"
bar2="baz2">
</foo>
At the moment we simply read the string and use java's build in read methods like Integer.parseInt(aString)
. We could optimize this by parsing the value directly while consuming the characters
TODO:
AutoValue 1.2 supports to hook in custom annotation processors so that TikXml's annotation processor can be integrated.
I get such error:
error: no suitable method found for add(Replica)
value.parts.add(v);
^
method Collection.add(CAP#1) is not applicable
(argument mismatch; Replica cannot be converted to CAP#1)
method List.add(CAP#1) is not applicable
(argument mismatch; Replica cannot be converted to CAP#1)
where CAP#1 is a fresh type-variable:
CAP#1 extends Part from capture of ? extends Part
for such var:
var parts: List<Part>
Hi,
first of all: thank you for the great library ;-)
Could you please add a possibility to skip those fields that are not defined within the pojo but are provideed by xml?
For now tikxml throws an io-exception.
thanks,
moritz
Caused by: java.lang.ClassCastException: com.sun.tools.javac.code.Symbol$VarSymbol cannot be cast to javax.lang.model.element.ExecutableElement
21:21:46.218 [ERROR] [org.gradle.BuildExceptionReporter] at com.tickaroo.tikxml.processor.scanning.FieldScanner.doScan(FieldScanner.kt:87)
21:21:46.218 [ERROR] [org.gradle.BuildExceptionReporter] at com.tickaroo.tikxml.processor.scanning.FieldScanner.scan(FieldScanner.kt:53)
21:21:46.218 [ERROR] [org.gradle.BuildExceptionReporter] at com.tickaroo.tikxml.processor.XmlProcessor.process(XmlProcessor.java:147)
21:21:46.218 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:794)
21:21:46.218 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:705)
21:21:46.220 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
21:21:46.222 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
21:21:46.222 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
21:21:46.222 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
21:21:46.222 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
21:21:46.222 [ERROR] [org.gradle.BuildExceptionReporter] at com.sun.tools.javac.main.Main.compile(Main.java:523)
It's not possible right now, because RealmList<MyObject>.class
doesn't work. The addTypeAdapter should support adding a typeadapter for a Java.lang.refl.Type.
As reported in #44 parsing an xml like
<week>
<day>Monday</day>
<day>Tuesday</day>
<day>Wednesday</day>
<day>Thursday</day>
<day>Friday</day>
<day>Saturday</day>
<day>Sunday</day>
</week>
is not possible with @PropertyElement
nor with @Element List<String>
directly.
The current workaround is to use @TextContent
but this requires an extra wrapper:
@Xml
public class Week {
@Element
List<Day> days;
}
@Xml
public class Day {
@TextContent String name;
}
Because in contrast to xml they are meaning something differently in xpath. #40
Determine the type (resolve polymorphism / inheritance) by xml element's attribute value. Might not be as efficient as current implementation polymorphism resolving strategies (xml element name) because we have to read all xml attributes (and store / buffer them temporarily) before we can continue with parsing.
If there is just one single attribute we don't have to allocate an extra HashMap
. In this case we can just use a simple if statement
Ignore static fields or throw an error if static field is annotated
Hi,
Is there any documentation where I can see how to use it?
I found only documentation about xml -> java mapping which is quite impressive, but I get some errors when I try to use it.
Do I need to write TypeAdapter for every class?
If I try to create object in following way
TikXml parser = new TikXml.Builder().build(); DataTable result = parser.read(Okio.buffer(Okio.source(is)), DataTable.class);
I get the exception:
com.tickaroo.tikxml.TypeAdapterNotFoundException: No TypeAdapter for class foo.bar.DataTable
Not really needed by us. However, some people might find it useful.
Ok, so after having thought about namespace support, this is what I cam up with:
We are going to have limited namespace support by allowing the user to write :
in the names.
This means we only support STABLE namespaces (not dynamically changeable) as we simply treat namespaces + element names / attribute names as one single name (string). Furthermore, namespace can't be implemented dynamically (read dynamically namespace and prefix while parsing xml) as annotation processor has already to know which
@Path
. Already implemented with #38@Attribute(name = "myNamespace:foo")
, @PropertyElement(name = "myNamespace:foo")
, @Element(name = "myNamespace:foo")
xmlns:m="http://www.w3.org/1998/Math/MathML"
so that .exceptionOnUnreadXml(true)
will not throw an exception when reading namespace definitions.@NamespaceDefinition(prefix="m", namespaceName="http://www.w3.org/1998/Math/MathML")
annotation that will be used when writing xml to add a namespace definition. Finde a better name for the annotation as the annotation will only be used when writing xml (has no impact to reading/parsing xml). The annotation name should state this out (only take into account while writing xml)In a second step (maybe TikXml 2.0), we could also try to add full dynamic namespace support. i.e. we could read namespace prefixes while reading / parsing xml and update internal mapping of TypeAdapter's
internal HashMaps for xml attribute names and xml elements as we read the namespaces ... but that is out of scope in version 1.0 as I'm pretty sure "stable" namespace support is enough for most users. Also, namespaces can be overridden in any child node at any time, which will effect all involved TypeAdapters. Maybe some kind of observer pattern will be needed to inform other TypeAdpters about overriden namespaces. To sum it up: not an easy task (doable though) and I'm not sure if it is worth the effort.
The original plan was to use annotations to represent namespaces, but it turns out that this is not possible with @Path()
annotations like @Path("x:foo/y:bar")
where we have 2 namespaces.
So this idea has been dropped!
The original idea was:
Would be nice to have namespaces annotation based, something like
@Namespace("foo")
public @interface FooNamespace{}
@Xml
class Something {
@FooNamespace
@PropertyElement String name;
}
@TextContent
can currently only be used on class fields of type String
. We could use TypeConverter
for @TextContent
.
Currently we are not handling @TextContent
and @Path
correctly since we ignore path and only allow one a @TextContent
on AnnotatedClass
.
Scan constructor for annotations.
Enables kotlin data classes :)
The parser simply ignores xml declaration <?xml version="1.0" encoding="UTF-8" ?>
and other Processing Instructions. Adding support for that is not simple and currently we don't need support for it.
However, we should consider that for feature development.
Okio 1.8 added a new APIs to match byte strings: indexOf(), startsWith(), and endsWith(). This should improve the performance for reading XML tags and attributes in your case.
Add support for alternate names like @PropertyElement(name="name1", alternate={"name2", "name3"})
Annotation Processor argument causes crash
Im doing the following to create the TikXml object
TikXml tikXml = new TikXml.Builder()
.exceptionOnUnreadXml(false)
.build();
adding the converter factory
...
.addConverterFactory(TikXmlConverterFactory.create(tikXml));
...
My object
@Xml(name = "myroottag", scanMode = ScanMode.ANNOTATIONS_ONLY)
public class ResultData {
@com.tickaroo.tikxml.annotation.Path("profile")
@com.tickaroo.tikxml.annotation.Element(name = "user")
private User mUser;
}
@Xml(scanMode = ScanMode.ANNOTATIONS_ONLY)
public class User extends RealmObject implements IIdentifyable {
@PrimaryKey
@PropertyElement(name = "id")
private int mIdentifier;
}
Add an option to the annotation processor, that primitive types like String
can be "converted" by a TypeConverter
The idea is to can replace HTML encoding characters directly when reading the xml element. So we could write a TypeConverter that executes Html.fromHtml(string)
to get a string that has replaced htlm encoding chars like "
directly with "
(otherwise we would do this afterwards anyway. Probably on the main UI thread in RecyclerViews Adapter.
https://developer.android.com/reference/android/text/Html.html
or other libraries like
http://commons.apache.org/proper/commons-lang/javadocs/api-3.1/org/apache/commons/lang3/StringEscapeUtils.html
When generating error messages XmlReader.getPath()
doesn't display the position of the element that has caused the error is part of a list with sibling elements.
Example:
<foo>
<bar attr="123" />
<bar attr="CausesErrorBecauseNotInteger" />
</foo>
then the error message should print /foo/bar[1]
(1 is the index/position of the element in the list, starting at 0) as path and not just /foo/bar
as it is right now. Makes debugging easier, because we know which element (in the list of elements) has caused this issue.
In the generated TypeAdapter we create a new StringBuilder
to read text content annotated with @TextContent
. We could optimize this by using a Pool of StringBuilders.
TypeConverter class com.tickaroo.tikxml.TypeConverter.NoneTypeConverter must provide an empty (parameter-less) public constructor
I was trying to parse this http://www.seminoles.com/rss.dbml?db_oem_id=32900&RSS_SPORT_ID=157113&media=news file. Somehow, I get the output RSS{newDBs=null}.
@Xml
public class RSS {
@Path("channel/item")
@Element
List<NewsDB> newDBs;
@Override
public String toString() {
return "RSS{newDBs=" + newDBs + "}";
}
}
@Xml
public class NewsDB {
@PropertyElement
String title;
@Override
public String toString() {
return "NewsDB{title = " + title + "}";
}
}
My proposal for the annotation API:
Marking a class to be recognized by TikXml: Simple XML uses @Root
annotation. In my oppinion not the best name, but we could use that one. Alternatively I purpose @Xmlable
.
@Root // alternatively @Xmlable
class Foo { }
produces and reads the following xml:
<foo></foo>
Simple xml's @Root
specification looks like this: @Root( name = "foo", strict = true)
and I think we can use the same one. Howerver, I think we should set strict
to false as default value (simple xml has true as default). This is compareable to jacksons @IgnoreUnknownJsonProperties
and will throw an exception if there are unknown or unparsed data. What do you think?
Attributes
@Root // alternatively @Xmlable
class Foo {
@Attribute
String a;
@Attribute
int b;
}
produces and reads the following xml:
<foo a="something" b="123"></foo>
Simple xml's @Attribute(name="something", required = true, empty="default value if not parsed in feed)
specification From my point of view @Attribute(required = true)
is in conflict with @Root(strict = true)
. I would say, regadring kicker feeds, that nothing should be required per default and I would suggest to keep @Attributes(required = true / false)
and @Element(required = true / false)
and discharge @Root(strict = false)
. What do you think?
Inner elements
@Root // alternatively @Xmlable
class Foo {
@Attribute
String a;
@Element Person person;
}
@Root
class Person {
@Element String name;
@Element int age;
}
produces and reads the following xml:
<foo a="something">
<person>
<name>Hannes</name>
<age>26</age>
</person>
</foo>
@Element(name="something", required = true/false)
Simple XML specification also provides CDATA
support and Class type(). While I would add CDATA support I'm not sure about the specific type. Reading (parsing) xml is determined by the xml element name (i.e.
maps to class
Foo`). However, when writing xml we might have explicitly specify which class to use to resolve polymorphism:
@Root
class Foo {}
@Root
class Other extends Foo {}
@Root
class Something {
@Element
Foo foo;
}
The problem is that now something.foo = new Foo()
or something.foo = new Other()
. However the xml to write looks entirely different for Foo
and Other
. Hence we have to specify somehow an explicit type, or do you see a better solution?
ElementList
The Simple XML specification looks good to me, especially @ElementList(inline = true / false)
. However, we still have the same problem with polymorphism when writing xml as already discussed for a single element
@Root
class Something {
@ElementList List<Foo> foos;
}
something.foo.add(new Foo() );
something.foo.add(new Other() );
Any good idea how to solve that issue? I think it can be estimated with annotation processing and instanceof
while writing xml to an outputstream, but I'm not to sure about it. Any hint?
XPath
As you already now, there are still a lot of wrappers etc. in kicker feeds. Those can be "skipped" or virtually emulated with @Path
annotation. I would use the same as Simple XML specification
I can't control remote xml files.
When XML Tag is like
<empty />
I get error message which is Expected xml element text content but was ELEMENT_END.
Why not to leave a blank text on it?
I got an issue when I tried to use annotated static inner classes with production build (Proguard obfuscated). With that config ClassLoader cannot find any of the generated classes that implement TypeAdapter. Unfortunately I couldn't modify Proguard config so that is works with that.
Solved it by not using static inner classes but I still had to exclude the classes from Proguard.
I think it's worth mentioning in the guide.
Originally reported in #44
Arrays are not supported because Arrays are inefficient to parse because we have to know the size of the array, which means we have to read the whole xml document and store the values somewhere temporary (i.e. in a list) to know the final size of the array and then we have to copy the values from temporary structure into array.
Therefore, we don't have plans to support arrays.
However, we should throw a compiler error message if people try to use arrays, but this would disallow using a global TypeConverter for arrays, but I think that this is acceptable.
If we specify a xml element name in an annotation like @Element(name="foo")
then we have to verify that the name doesn't contain any xml reserved characters:
<
or >
"
=
Right now HashMap<String, ChildElementBinder<T>>
or HashMap<String, AttributeBinder<T>>
is used in generated Code. This could be optimized for Android to use ArrayMap
We should add an Annotation Processor Option to turn this on / off in case that TikXml is in a java project
The class red.kometa.quest.common.entities.Paragraph.Part.Action.Way.NormalBehaviour is not annotated with @Xml, but is used in 'behaviour' in class @red.kometa.quest.common.entities.Paragraph.Part.Action.Way to resolve polymorphism. Please annotate @red.kometa.quest.common.entities.Paragraph.Part.Action.Way with @Xml
There is must be something like:
Please annotate red.kometa.quest.common.entities.Paragraph.Part.Action.Way.NormalBehaviour with @Xml
I see such compile-time error:
Error:Gradle: The constructor parameter 'arg0' in constructor Narration(java.lang.String) in class red.kometa.quest.common.entities.Paragraph.Part.Narration is annotated with a TikXml annotation. Therefore a getter method with minimum package visibility with the name getArg0() or isArg0() in case of a boolean must be provided. Unfortunately, there is no such getter method. Please provide one!
there is my data classes:
@Xml(name = "paragraph")
data class Paragraph(
@Attribute(name = "key") val key: String,
@Element(
typesByElement = arrayOf(
ElementNameMatcher(type = Part.Replica::class),
ElementNameMatcher(type = Part.Narration::class),
ElementNameMatcher(type = Part.Actions::class),
ElementNameMatcher(type = Part.GameOver::class),
ElementNameMatcher(type = Part.GameWin::class)
)
) val parts: List<Part>
) {
interface Part {
@Xml(name = "replica")
data class Replica(
@Attribute(name = "text_key") val textKey: String,
@Attribute(name = "character_key") val characterKey: String
) : Part
@Xml(name = "narration")
data class Narration(
@Attribute(name = "text_key") val textKey: String
) : Part
@Xml(name = "actions")
data class Actions(
@Element(
typesByElement = arrayOf(
ElementNameMatcher(type = Action.Way::class)
)
)
val actions: List<Action>
) : Part
interface Action {
val force: Boolean
val textKey: String?
@Xml(name = "way")
data class Way(
@Attribute(name = "force") override val force: Boolean,
@Attribute(name = "text_key") override val textKey: String?,
@Attribute(name = "to") val to: String,
@Element(name = "behaviour", typesByElement = arrayOf(
ElementNameMatcher(type = NormalBehaviour::class),
ElementNameMatcher(type = WaitBehaviour::class)
)) val behaviour: Behaviour
) : Action {
/**
* Do not use annotations as using
* @see red.kometa.quest.common.utils.BehaviourTypeAdapter
*/
interface Behaviour
class NormalBehaviour : Behaviour
data class WaitBehaviour(
val duration: Long
) : Behaviour
}
}
@Xml(name = "gameover")
data class GameOver(
@Attribute(name = "text_key") val textKey: String,
@Attribute(name = "restart") val restart: Boolean,
@Attribute(name = "checkpoint_paragraph_key") val checkpointParagraphKey: String?
) : Part
@Xml(name = "gamewin")
data class GameWin(
@get:Attribute(name = "text_key") @param:Attribute(name = "text_key") val textKey: String
) : Part
}
}
Kotlin must generate such getters&setters.
kotlin_version = '1.0.6'
tikxml_verions = '0.8.9-SNAPSHOT'
Add a check if TypeConverter matches the type of the annotated field.
Right now, the annotation processor would not complain about something like this
@Attribute(converter = DateConverter.class) // converts to Date.class
String notADate;
We should remove support for different scan modes and only support annotations to scan fields / constructors. Especially constructors are confusing with ScanMode.COMMON_CASE
The version I'm using is
compile 'com.tickaroo.tikxml:annotation:0.6.3-SNAPSHOT'
compile 'com.tickaroo.tikxml:core:0.6.3-SNAPSHOT'
apt 'com.tickaroo.tikxml:processor:0.6.3-SNAPSHOT'
Let's say I have a
<weatherForecast>
<day>Wednesday</day>
<temperature high="33" low="24" unit="Degrees Celsius"/>
<day>Thursday</day>
<temperature high="33" low="24" unit="Degrees Celsius"/>
<day>Friday</day>
<temperature high="34" low="25" unit="Degrees Celsius"/>
<day>Saturday</day>
<temperature high="33" low="24" unit="Degrees Celsius"/>
</weatherForecast>
I'll define a WeatherForecast class
@Xml(name="weatherForecast")
public static class WeatherForecast
{
@PropertyElement
public String[] day;
@Element
public List<Temperature> temperature;
}
@Xml(name="temperature")
public static class Temperature
{
@Attribute
public String unit;
@Attribute
public String high;
@Attribute
public String low;
}
For the WeatherForecast class, If I declare day as
String[]
, can compile and run, get crashNo TypeConverter found for type class [Ljava.lang.String;. You have to add one via TikXml.Builder().addTypeAdapter()
@PropertyElement List<String>
, can't compile
for WeatherForecast$$TypeAdapter, compile error value.day = config.getTypeConverter(java.util.List<java.lang.String>.class).read(reader.nextTextContent());
, Error:(43, 80) error: <identifier> expected
@Element List<String>
, can't compile
Error:(186, 29) error
: The type java.lang.String used for field 'day' in weather.fangzhzh.com.weather.data.model.FourDays.WeatherForecast can't be used, because is not annotated with @xml. Annotate java.lang.String with @xml!
DelegatingTypeAdapter
has already replaced with a purely generated (annotation processing) TypeAdapter
. There are some Unit tests using DelegatingTypeAdapter
, but really just for convenience reasoning (and / or to test DelegatingTypeAdapter
himself).
From the newly implemented TypeAdapter
code generator (introduced in version 0.6.0-SNAPSHOT) almost everything can be reused to generated highly optimized ChildElementBinders
so that NestedChildElementBinder
is not needed anymore
For example, this can be replaced with skip(3)
. Not a functional change, but more semantically correct.
These constants should have a type of byte
so they're inlined into the usage sites and avoid boxing when invoking indexOf(byte)
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.