pdvrieze / xmlutil Goto Github PK

Kotlin: 1.3.72 JVM
KotlinX serialization: 0.20.0
xmlutil: 0.20.0.1

When defining types than can recursively contain themselves (I'd like to avoid the discussion whether that is good design) the de-serialization process hangs and finally throws a StackOverflowError with the stacktrace jumping back and forth between Canary.kt:208 and Canary.kt:213 in Canary.serialDescriptor. Example:

@Serializable
data class XmlRec(val nested: List<XmlRec> = emptyList())

val recursionWonderland = XmlRec(listOf(XmlRec(listOf(XmlRec(), XmlRec())), XmlRec()))

val intoTheRabbitHole = XML.stringify(XmlRec.serializer(), recursionWonderland) // <--- this produces the expected output as defined below
val notComingBack = XML.parse(XmlRec.serializer(), intoTheRabbitHole) // <--- this call triggers the StackOverflowError

Albeit the arguably poor design decision, I believe that the following output representing the recursionWonderland variable is perfectly valid XML:

<XmlRec>
	<XmlRec>
		<XmlRec/>
		<XmlRec/>
	</XmlRec>
	<XmlRec/>
</XmlRec>

Just for the sake of mentioning it: I'm working with an external dictionary dump in XML format that has these kinds of design decisions and it is neither feasible (due to the immense size and historical artifacts) nor within my power to change the input format. This library is my only ray of hope for not having to work with JAXB! 😄

I have having some dfficulty trying to understand why problems are being seen. Though the core issue looks to be in StAXReader and above.

Below is a very quick sample after dragging down my diagnostics to the most basic level I can find.

class SampleParser {
    val baseXmlFormat = XML

    fun getSampleEntity(): TestBaseModel {
        val serializer = TestBaseModel.serializer()
        val input = getSampelXmlImplA()
        println(input)
        return baseXmlFormat.parse(serializer, input)
    }

    fun getSampelXmlImplA(): String {
        return """
            <replicatedDirectory>
            <path>/a/b/c</path>
            <name>repl1</name>
            <gsn>1024</gsn>
        </replicatedDirectory>
        """.trimIndent()
    }

    fun getSampelXmlImplB(): String {
        return """
        <replicatedDirectory>
            <path>/a/b/c</path>
            <name>repl1</name>
            <gsn>1024</gsn>
        </replicatedDirectory>
        """.trimIndent()
    }
}

When using the XML string returned by getSampelXmlImplA, parsing of my data is perfectly fine.

However, if parsing the XML string returned by getSampelXmlImplB, parsing dives into an indefinite loop.

A snapshot of the parsing thread.

"main" #1 prio=5 os_prio=0 tid=0x00007f4ef800d000 nid=0x6299 runnable [0x00007f4efebeb000]
   java.lang.Thread.State: RUNNABLE
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.arrangeCapacity(XMLEntityScanner.java:1764)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipString(XMLEntityScanner.java:1799)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1748)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
	at com.sun.org.apache.xerces.internal.impl.XMLStreamReaderImpl.next(XMLStreamReaderImpl.java:553)
	at nl.adaptivity.xmlutil.StAXReader.next(StAXReader.kt:172)
	at nl.adaptivity.xmlutil.XmlReaderUtil__XmlReaderKt.readSimpleElement(XmlReader.kt:278)
	at nl.adaptivity.xmlutil.XmlReaderUtil.readSimpleElement(Unknown Source)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.decodeStringElement(XMLDecoder.kt:777)
	at com.github.ppslim.models.samples.TestBaseModel$$serializer.deserialize(TestBaseModel.kt)
	at com.github.ppslim.models.samples.TestBaseModel$$serializer.deserialize(TestBaseModel.kt:8)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$XmlDecoder.decodeSerializableValue(XMLDecoder.kt:140)
	at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:333)
	at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:351)
	at nl.adaptivity.xmlutil.serialization.XML$Companion.parse(XML.kt:475)
	at com.github.ppslim.models.samples.SampleModel.getSampleEntity(SampleModel.kt:22)
	at com.github.ppslim.models.samples.SampleModelKt.main(SampleModel.kt:10)
	at com.github.ppslim.models.samples.SampleModelKt.main(SampleModel.kt)

Adding some breakpoints to XMLEntityScanner.arrangeCapacity, I can see that it never breaks out of the while loop.

I doubt the problem is in this library, but it has been confusing me all day

Currently I can't use the serialization plugin in jlink due to split package problem. It could be fixed by package renaming.

We've recently updated xmlutil to version 0.81.0, and seem to have come across a bug in how deserialization for nullable fields is handled.

Our application accepts XML from a third party system, which we don't control. At times, rather than omit an element for missing data, it will instead provide XML with an element which is present but not real data, e.g.
<Doc><Date>00.00.0000</Date></Doc> instead of <Doc></Doc>. We handle this by having a Serializer that recognises these values and outputs null instead, which is the semantically correct value.

On the previous xmlutil version we were using (0.13.0.2, so quite old), this worked fine. However, with 0.81.0, deserialization fails with the following error:

java.lang.NullPointerException: null cannot be cast to non-null type T of nl.adaptivity.xmlutil.serialization.XmlDecoderBase.TagDecoder.decodeNullableSerializableElement
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.decodeNullableSerializableElement(XMLDecoder.kt:393)

From looking at the XMLDecoder code, it seems to be casting the result of the operation to T rather than T?, which may be the cause, though I'm not certain.

A small test which reproduces the issue is here: https://gist.github.com/conorfarrell-coats/f6a6970dd74196b3ab47a289a2dc99c9

If we're holding it wrong, very happy to be told this! Also happy to work with you to make a fix if you can give us some guidance.

Kotlin: 1.3.72 JVM
KotlinX serialization runtime: 0.20.0
xmlutil: 0.20.0.1

When de-serializing XML into a class that has nullable attributes, the parsing fails under certain circumstances. Example:

@Serializable
// uncomment this annotation as quick-fix @XmlSerialName("XmlRec", "", "")
data class XmlRec(val item: Int, val maybeRec: XmlRec? = null)

val thisExplodes = XmlRec(3, XmlRec(2, XmlRec(1))) // inner-most entity implicitly has maybeRec = null

val namespaceLazySerial = XML { repairNamespaces = false }

val explosiveXml = namespaceLazySerial.stringify(XmlRec.serializer(), thisExplodes)
val explosion = namespaceLazySerial.parse(XmlRec.serializer(), explosiveXml)

With the XmlSerialName annotation commented out, de-serializing fails with the following error message: nl.adaptivity.xmlutil.serialization.UnknownXmlFieldException: Could not find a field for name XmlRec candidates: item, XmlRec? at position Line number = 2

Serializing works as expected. By the looks of the error message it seems that the internal code does a string comparison between XmlRec and XmlRec? somewhere, which of course turns out false. I would reckon however that with de-serializing it seems reasonable to use the XmlRec? deserialization strategy because the XML parser has already encountered an <XmlRec> tag (that he is trying to unserialize unsuccessfully, therefor prompting the error message) that will arguably not be null anyways.

Interestingly, with the XmlSerialName annotation in place everything works because we then force the parser to not infer the nullability ? in the deserialization strategy name.

Please do not print messages directly to standard output.

nl/adaptivity/xmlutil/serialization/XMLDecoder.kt:

println("Looking for a match for attribute $name, empty ns prefix is: $emptyNsPrefix")

`@Test fun "can serialize with xml-declaration"() {

    val expectedXml = """
        <?xml version="1.0" encoding="UTF-8"?>
        <prefix:model xmlns:prefix="namespace" version="0.0.1" anAttribute="attrValue">
            <prefix:anElement>elementValue</prefix:anElement>
            <prefix:aBlankElement></prefix:aBlankElement>
        </prefix:model>"""

    val model = MyExampleSerializableModel()

    val format = XML {
            omitXmlDecl = false
            indent = 4
    }

    val serializedModel = format.stringify(MyExampleSerializableModel.serializer(), model)

    assertEquals(expectedXml.trimIndent(), serializedModel)
}`

Hey Paul. First thank you for your great work!

Could you give me advice how to solve the issue?

Xml node with mixed content throws exception when content in particular order:
suppose we have classes:

@Serializable
data class Tree(val node1: Node1)

@Serializable
@SerialName("node1")
data class Node1(
    @XmlValue(true) val text: String = "",
    val subnode1: Subnode1
)

@Serializable
@SerialName("subnode1")
data class Subnode1(@XmlValue(true) val text: String)

this xml can be deserialized:

<tree>
   <node1>
       <subnode1> text1 </subnode1>
           some text here
   </node1>
</tree>

but if we swap subnode1 and some text:

<tree>
   <node1>
        some text here
       <subnode1> text1 </subnode1>
   </node1>
</tree>

exception is thrown:
Exception in thread "main" nl.adaptivity.xmlutil.XmlException: Found unexpected child tag

gist: here

 <?xml version="1.0" ?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
    <S:Body>
        <ns2:Ge xmlns:ns2="http://www.gxtlink.com/webservice/">
            <result>
                <code>0</code>
                <data>
                    <project>get</project>
                    <unit>p</unit>
                </data>
            </result>
        </ns2:Ge >
    </S:Body>
</S:Envelope>

how can I parse this xml ?
I tried to parse it but failed

Consider this code:

@Serializable
class Config {
    val jobList = mutableListOf<Job>()
}

@Serializable
@SerialName("job")
class Job {
}

The serialization works fine for single Job, but when deserialization fails, because it tries to find an element with name jobList, not job. When property name is the same as class serial name, it works fine.

This feature requires some additional work and external tool, but I am putting it here just to track it. It is possible to generate Kotlin class file compatible with kotlinx.serialization from xml schema. We should consider doing it in future. It will also help to cover different use-cases.

I think a library like TikXml has supported CDATA very well. Why XmlUtil isn't ?

I want to parse xml content from https://phatgiao.org.vn/loi-phat-day.rss however, I got an error.

I used inline class to wrap content, use annotation @XmlValue(true) but nothing works.

Hope you might support CDATA better.

Thank you.

Hi,
I'm currently investigating if we can abandon Jackson in favor of Kotlin serialization. We use XML, JSON, and YAML. For XML I was looking at this adapter. However, the generated XML is different from Jackson (w/o "wrapper") configuration:

jackson: <Team><members><name>Joe</name><age>15</age></members></Team>
kotlin: <Team><Person name="Joe" age="15"/></Team>

This was generated using

@Serializable data class Person(val name: String, val age: Int)
@Serializable data class Team(val members: List<Person>, val colors: List<String> = listOf())

fun main() {
    val t = Team(listOf(Person("Joe", 15)))

    val mapper = XmlMapper.builder().defaultUseWrapper(false).build().registerKotlinModule()
    println("jackson: ${mapper.writeValueAsString(t)}")

    val xml = XML()
    println("kotlin: ${xml.stringify(Team.serializer(), t)}")
}

If I add annotations

@Serializable data class Person(@XmlElement(true) val name: String, @XmlElement(true) val age: Int)
@Serializable data class Team(@XmlChildrenName("members", "", "") val members: List<Person>, val colors: List<String> = listOf())

I get

kotlin: <Team><members><members><name>Joe</name><age>15</age></members></members></Team>

which is closer (besides the double "members"). But I really would like to avoid doing this to all the data classes. Could we have a val xml = XML(jackson = true, wrapper = false) which does this all automatically?

Hello I'm triyng to add your serialization library to my android project but I always get
ERROR: Failed to resolve: net.devrieze:xmlutil-serialization-android:0.20.0.0

I am currently trying to consume a XML API with a ktor-client but struggle a bit with how I can embed the framework as part of a Feature that will auto serialize my input body to XML and automatic de-serialize the returned XML data to an object (similar to how I can use JSON at the moment: https://ktor.io/clients/http-client/features/json-feature.html)

Kotlin: 1.3.72 JVM
KotlinX serialization runtime: 0.20.0
xmlutil: 0.20.0.1

When annotating any serializable class with a namespaced XmlSerialName and then also annotating any of its properties with another, different namespaces, the individual namespaces are not prefixed correctly in the serialized XML.

@Serializable
@XmlSerialName("namespaced", "http://example.org", "ex")
data class PrefixBreaks(
    @XmlSerialName("property", "http://myownnamespace.com", "foo") val property: String
)

val thisExplodes = PrefixBreaks("bug")

val defaultSerializer = XML { repairNamespaces = false }
val repairingSerializer = XML { repairNamespaces = true }

// uses incorrect prefix "zdef-38325484" for property
println(repairingSerializer.stringify(PrefixBreaks.serializer(), thisExplodes))

// NPE, see below
println(defaultSerializer.stringify(PrefixBreaks.serializer(), thisExplodes))

The first println statement yields the following output:

<ex:namespaced xmlns:ex="http://example.org" xmlns:zdef-38325484="http://myownnamespace.com" zdef-38325484:property="bug"/>

The sourcecode explicitly specifies prefix foo, so the expected output would be:

<ex:namespaced xmlns:ex="http://example.org" xmlns:foo="http://myownnamespace.com" foo:property="bug"/>

The second println throws a nl.adaptivity.xmlutil.XmlException with the following stack trace:

Exception in thread "main" nl.adaptivity.xmlutil.XmlException: javax.xml.stream.XMLStreamException: Prefix cannot be null
	at nl.adaptivity.xmlutil.StAXWriter.doAttribute(StAXWriter.kt:245)
	at nl.adaptivity.xmlutil.StAXWriter.doFlushPending(StAXWriter.kt:147)
	at nl.adaptivity.xmlutil.StAXWriter.endTag(StAXWriter.kt:160)
	at nl.adaptivity.xmlutil.XmlWriterUtilCore.endTag(XmlWriter.kt:393)
	at nl.adaptivity.xmlutil.serialization.XmlEncoderBase$TagEncoder.endStructure(XMLEncoder.kt:270)
	at Playground$main$PrefixBreaks$$serializer.serialize(Playground.kt)
	at Playground$main$PrefixBreaks$$serializer.serialize(Playground.kt:63)
	at nl.adaptivity.xmlutil.serialization.XML.toXml(XML.kt:280)
	at nl.adaptivity.xmlutil.serialization.XML.stringify(XML.kt:189)
	at nl.adaptivity.xmlutil.serialization.XML.stringify(XML.kt:174)
	at Playground.main(Playground.kt:73)
Caused by: javax.xml.stream.XMLStreamException: Prefix cannot be null
	at java.xml/com.sun.xml.internal.stream.writers.XMLStreamWriterImpl.writeAttribute(XMLStreamWriterImpl.java:582)
	at nl.adaptivity.xmlutil.StAXWriter.doAttribute(StAXWriter.kt:242)
	... 10 more

where Playground.kt:63 is the location of my @Serializable annotation on the PrefixBreaks class and Playground.kt:73 is the call to defaultSerializer.stringify.

Three ways of configuring the XML() {}:

val xmlConfig = XmlConfig(
                indentString = "    ",
                indent = 4
        )

      val xmlConfigBuilder = XmlConfig.Builder()
       xmlConfigBuilder.indentString = "    "
       xmlConfigBuilder.indent = 4

val format = XML {
            indentString = "    "
            indent = 4
        }

indentString has always no resolution.

I don't get it, even after studying the sources:

class XmlConfig(
    val repairNamespaces: Boolean = true,
    val omitXmlDecl: Boolean = true,
    val indentString: String = "",
    val autoPolymorphic: Boolean = false,
    val unknownChildHandler: UnknownChildHandler = DEFAULT_UNKNOWN_CHILD_HANDLER
               ) {

    constructor(
        repairNamespaces: Boolean = true,
        omitXmlDecl: Boolean = true,
        indent: Int,
        autoPolymorphic: Boolean = false,
        unknownChildHandler: UnknownChildHandler = DEFAULT_UNKNOWN_CHILD_HANDLER
               ) : this(repairNamespaces, omitXmlDecl, " ".repeat(indent), autoPolymorphic, unknownChildHandler)

    constructor(builder: Builder) : this(
        builder.repairNamespaces,
        builder.omitXmlDecl,
        builder.indentString,
        builder.autoPolymorphic,
        builder.unknownChildHandler
                                        )

    @Deprecated("Use indentString for better accuracy")
    val indent: Int
        get() = indentString.countLength()

    /**
     * Configuration for the xml parser.
     *
     * @property repairNamespaces Should namespaces automatically be repaired. This option will be passed on to the [XmlWriter]
     * @property omitXmlDecl Should the generated XML contain an XML declaration or not. This is passed to the [XmlWriter]
     * @property indentString The indentation to use. This is passed to the [XmlWriter]. Note that at this point no validation
     *           of the indentation is done, if it is not valid whitespace it will produce unexpected XML.
     * @property indent The indentation level (in spaces) to use. This is derived from [indentString]. Tabs are counted as 8
     *                  characters, everything else as 1. When setting it it will update [indentString] with `indent` space characters
     * @property autoPolymorphic Should polymorphic information be retrieved using [SerializersModule] configuration. This replaces
     *                     [XmlPolyChildren], but changes serialization where that annotation is not applied. This option will
     *                     become the default in the future although XmlPolyChildren will retain precedence (when present)
     * @property unknownChildHandler A function that is called when an unknown child is found. By default an exception is thrown
     *                     but the function can silently ignore it as well.
     */
    class Builder(
        var repairNamespaces: Boolean = true,
        var omitXmlDecl: Boolean = true,
        var indentString: String = "",
        var autoPolymorphic: Boolean = false,
        var unknownChildHandler: UnknownChildHandler = DEFAULT_UNKNOWN_CHILD_HANDLER
                 ) {
        constructor(
            repairNamespaces: Boolean = true,
            omitXmlDecl: Boolean = true,
            indent: Int,
            autoPolymorphic: Boolean = false,
            unknownChildHandler: UnknownChildHandler = DEFAULT_UNKNOWN_CHILD_HANDLER
                   ) : this(repairNamespaces, omitXmlDecl, " ".repeat(indent), autoPolymorphic, unknownChildHandler)

        var indent: Int
            @Deprecated("Use indentString for better accuracy")
            get() = indentString.countLength()
            set(value) { indentString = " ".repeat(value) }
    }

I highly appreciate any help and comments!

xmlutil version: v0.80.0
platform: Android

Hi, I was decoding some XML from a third party and I noticed that decodeFromString was failing when the XML response had "&" or other HTML entities (like "'" for single quote) as text content inside an element/tag.

After finding that, I tried to create a simple example and I would like some help to understand how to deal with "&".

Example:

If I use encodeToString to encode the following content:

@Serializable
data class Results(
    val itemList: List<Item>
)

@Serializable
data class Item(
    @XmlValue(true)
    val text: String
)

[...]

val content =
    Results(
        listOf(
            Item("Item ' 1"),
            Item("Item & 2")
        )
    )

I get this XML as a result:

<Results>
  <Item>Item ' 1</Item>
  <Item>Item &amp; 2</Item>
</Results>

and now, if I use that same result output with decodeFromString I get an error

nl.adaptivity.xmlutil.XmlException: Found unexpected child tag: ENTITY_REF

I also noticed that, if I remove @XmlValue(true) and use "&" inside an attribute

<Results>
  <Item text="Item ' 1" />
  <Item text="Item &amp; 2" />
</Results>

the decoding goes perfectly.

What's the correct way of decoding XML that has "&" as text inside an element?

Kotlin 1.4 and serialization 1.0.

@pdvrieze I know that you work on it, but I put it here for tracking. Please do not forget to enable IR mode for JS.

Suppose we have a sealed class hierarchy like this:

sealed class S

@SerialName("a")
class A: Parent()

@SerialName("b")
class b: Parent()

class Holder{
  val children = ArrayList<@ContextualSerialization S>()
}

Then we create an array of S and pass it to serializer via `ContextSerializer, obtaining something like

<Holder>
  <a>...</a>
  <b>...</b>
</Holder>

Currently deserializer tries to interpret those tags as fields. It is important to be able to somehow tell it that those are parts of the list.

For some reason it compares the deserialized string with class name of Enum instead of its value.

I did not found tests for enums. I use @XmlElement(false) on my enum property to encode it as attribute instead of element, but it does not seem to cause the problem.

I have a usecase where I need to generate one of the tags dynamically. Lets assume I have the following data class:

data class Test(val id: String, val entity: MyEntity)

I want to have as an potential output an xml that could look like this

<Test>
   <Test_123>
       <MyEntityFields... />
   </Test_123>
</Test>

Where Test_123 is the generated tag based on a fixed string and the dynamic value 123 from the id field. I also would need to have MyEntity NOT have there SerialName printed and instead directly print all there attribute elements.

I know that we now have instructions to create dynamic tags based on variables but overall this requires a lot of boilerplate code. I was wondering if there could be an easy feature enhancement. I had two ideas in mind:

1. Add a @DynamicSerialName annotation

This could be applied to ether a variable (of type String) or to a method returning a string. If the serializer encounters a class with an annotation like this (only one annotation per class allowed) it would use the value instead of the static SerialName. The only downside could be deserialization that might be a bit more complex potentially (even though it shouldn't be to hard).

2. Add @XmlAnyGetter and @XmlAnySetter annotations

Similar to Jackson this annotations could allow annotating a method that returns a Map<String, T?> to create multiple dynamic elements. This is probably a lot harder to implement.

Curious to hear your thoughts on this.

I'm parsing XML which uses tabs for indent and I would like serialization of my models to XML to produce the same tab-indented output. Using a string for indent is a simple way to support both tabs and spaces without having to create individual options for both.

Creating a spring boot server in kotlin and my gradle looks like this

dependencies {
    ...
    implementation "net.devrieze:xmlutil-jvm:0.81.1"
    implementation "net.devrieze:xmlutil-serialization-jvm:0.81.1"
    ...
}

And when I run ./gradlew clean build I get:

 Could not resolve all files for configuration ':productionRuntimeClasspath'.
   > Could not resolve net.devrieze:xmlutil:0.81.1.
     Required by:
         project :
         project : > net.devrieze:xmlutil-serialization-jvm:0.81.1
      > Cannot choose between the following variants of net.devrieze:xmlutil:0.81.1:
          - androidRuntimeElements-published
          - jvmRuntimeElements-published
        All of them match the consumer attributes:
          - Variant 'androidRuntimeElements-published' capability net.devrieze:xmlutil:0.81.1:
              - Unmatched attributes:
                  - Found net.devrieze.android 'true' but wasn't required.
                  - Required org.gradle.dependency.bundling 'external' but no value provided.
                  - Found org.gradle.jvm.version '6' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Found org.jetbrains.kotlin.platform.type 'androidJvm' but wasn't required.
              - Compatible attributes:
                  - Required org.gradle.libraryelements 'jar' and found compatible value 'jar'.
                  - Required org.gradle.usage 'java-runtime' and found compatible value 'java-runtime'.
          - Variant 'jvmRuntimeElements-published' capability net.devrieze:xmlutil:0.81.1:
              - Unmatched attributes:
                  - Found net.devrieze.android 'false' but wasn't required.
                  - Required org.gradle.dependency.bundling 'external' but no value provided.
                  - Found org.gradle.jvm.version '8' but wasn't required.
                  - Found org.gradle.status 'release' but wasn't required.
                  - Found org.jetbrains.kotlin.platform.type 'jvm' but wasn't required.
              - Compatible attributes:
                  - Required org.gradle.libraryelements 'jar' and found compatible value 'jar'.
                  - Required org.gradle.usage 'java-runtime' and found compatible value 'java-runtime'.

How can I solve this? I've tried to add org.gradle.usage attribute to dependency attributesSchema and other options but nothing seems to fix it. This only happens when I try to build it from the console, when I run/compile it from IntelliJ everything works fine.
I'm kind of stuck, anyone have any idea on how to solve this?

Parsing the following line :

<reclamation code="1"><![CDATA[30/06/2012 - M75 - Normal - NON LIVRE]]></reclamation>

Using the following class:

@Serializable
@XmlSerialName(value = "reclamation", namespace = "", prefix = "")
data class Reclamation(@XmlValue(true) val content: String, val code: String)

works when using the jvm artifact (on a jvm machine), but on android it fails with the following error:

E/TestRunner: java.lang.AssertionError: Unexpected event in stream
        at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.decodeElementIndex(XMLDecoder.kt:705)
        at com.nitrog42.xmlserializationandroidbug.Reclamation$$serializer.deserialize(Unknown Source:19)
        at com.nitrog42.xmlserializationandroidbug.Reclamation$$serializer.deserialize(Reclamation.kt:9)
        at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$XmlDecoder.decodeSerializableValue(XMLDecoder.kt:140)
        at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:333)
        at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:351)
        at com.nitrog42.xmlserializationandroidbug.TestReclamationAndroid.testParse(TestReclamationAndroid.kt:24)
        at java.lang.reflect.Method.invoke(Native Method)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:104)
        at org.junit.runners.Suite.runChild(Suite.java:128)
        at org.junit.runners.Suite.runChild(Suite.java:27)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
        at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392)
        at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2145)

This repository https://github.com/NitroG42/XmlSerializationAndroidBug reproduces the issue, you just need to run the androidTest or the test to get it.

It works if the CDATA part is replaced with a normal String.

Version: xmlutil-serialization-jvm 0.13.0.1

See tests - 'Foo' and 'Baz' are two basically identical XML files apart from the name of the top level element. Yet deserializing Baz results in error java.lang.AssertionError: This should not happen as decodeSerializableValue should be called first. Making the field non-nullable by defaulting to an empty list allows deserialization to occur. If this is intended behaviour, please can you update the docs? If not, hopefully this edge case helps you.

@Serializable
@SerialName("Foo")
data class Foo(
        @XmlElement(true)
        @SerialName("Str")
        val aString: String,

        @XmlElement(true)
        @SerialName("Bar")
        val aList: List<Bar>
)

@Serializable
@SerialName("Bar")
data class Bar(
        @XmlElement(true)
        @SerialName("AnotherStr")
        val anotherString: String
)

@Serializable
@SerialName("Baz")
data class Baz(
        @XmlElement(true)
        @SerialName("Str")
        val aString: String,

        @XmlElement(true)
        @SerialName("Bar")
        val aList: List<Bar>? = null
)

class XmlDeserializeBug: StringSpec() {

    init {
        "Repeated tags without enclosing tag should be deserialized" {
            val xml = """<?xml version="1.0"?>
                        <Foo>
                        <Str>A String</Str>
                        <Bar><AnotherStr>Another String1</AnotherStr></Bar>
                        <Bar><AnotherStr>Another String2</AnotherStr></Bar>
                        </Foo>""".trimMargin()
            val result = XML.parse(xml.xmlReader(), Foo.serializer())
            result.aList.size shouldBe 2
        }

        "Repeated tags without enclosing tag should be deserialized when list is nullable" {
            val xml = """<?xml version="1.0"?>
                        <Baz>
                        <Str>A String</Str>
                        <Bar><AnotherStr>Another String1</AnotherStr></Bar>
                        <Bar><AnotherStr>Another String2</AnotherStr></Bar>
                        </Baz>""".trimMargin()
            val result = XML.parse(xml.xmlReader(), Baz.serializer())
            result.aList?.size shouldBe 2
        }
    }

}

The second test fails.

In a code like this:

    @XmlSerialName("first", "", "")
    override var first: GDMLRef<GDMLSolid>,

regular @SerialName annotation does not work for some reason (it uses GDMLRef as tag name instead of first), so I have to use @XMLSerialName, but writing empty strings each time is annoying. I think that additional arguments could be made optional.

If I have a XML file like this:

<?xml version="1.0"?>
<gdml xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://service-spi.web.cern.ch/service-spi/app/releases/GDML/schema/gdml.xsd">

The parsing is failed with message:

nl.adaptivity.xmlutil.serialization.UnknownXmlFieldException: Could not find a field for name {http://www.w3.org/2001/XMLSchema-instance}noNamespaceSchemaLocation

Hello, @pdvrieze! First of all, than you very much for all your contributions to Kotlin community, you are making a big change and this is highly appreciated!

Could you please clarify whether LGPL v3 is selected intentionally or you might review picked license. To my knowledge, there were no precedents yet as to whether using LGPL dependency forces JVM-based application to become LGPL-licensed or not. Though FSF has an article sharing the intent that LGPL should work for Java in the same way as it does for languages compiling to shared objects, this still only describes intention, which, as you are probably aware of, can mean nearly nothing in court trials. This fact could hinder adoption of the library even for open source projects having under more permissive license, e.g. Apache 2.0 or MIT.

In any case, keep up the great work!

Regards,
Andrew.

I want to call SOAP web services on Android platform using Retrofit Http Client. However, I need a Converter Factory to serialize/deserialize request & response bodies to POJOs used in my project.

For JSON there are some really good serializers, like Jackson, Gson & Moshi. But the converter factory for XML parsing they give is SimpleXML and it's no as feature rich at all for our use cases. And the JAXB Converter does not work on Android, so that is out of the question.

So I guess I'm left with no choice but to write my own Converter Factory. Thing is I've never really created my own Converter Factory for serialization. So this is all unknown territory for me here. So any help will be gladly appreciated.

Maybe you guys can provide a retrofit converter package for android in the near future. That would be absolutely amazing. so yeah, NEED HELP.

I have an element which could contain either element with type A or with type B distinguished by its tag name. In previous versions I could write something like

    @XmlPolyChildren(arrayOf("physvol","divisionvol"))
    @Polymorphic
    var placement: GDMLPlacement? = null

and it worked. Now if I do that, then I get Stackoverflow exception caused by the fact that descriptor could not be located (it is fixed in 0.13.0, but latest version is not compatible with xmlutil). It is very hard to understand what exactly causes the error.

Current workaround is to replace single element by a list:

    var placement = ArrayList<@Polymorphic GDMLPlacement>()

There is a quite frequent case when we do not need to parse the whole XML, but only meaningful parts of it. In those cases it makes no sense to describe the whole structure in code, but only those things that are actually useful.

The proposal is following: Make a global opt-in flag which will replace all field not found errors with warnings and add additional per-class annotation flag which will override global flag. It will also allow to solve #15 by automatically toggling unsafe parse for top-level node.

I can try to contribute a solution if you approve.

stringify -> encodeToString
toXML -> encodeToXml
parse -> decodeFromXml

Right now you have both, but it is a bit confusing. I think it won't hurt to use canonical names.

I'm currently trying to deserialize XML-files that are formatted something like these two with version 0.80.1:

<?xml version="1.0" encoding="UTF-8"?>
<fruit type="apple" name="MyApple" numAppleSeeds="5">
  <!-- child elements omitted for simplicity -->
</fruit>

<?xml version="1.0" encoding="UTF-8"?>
<fruit type="tomato" name="MyTomato" color="red">
  <!-- child elements omitted for simplicity -->
</fruit>

I would like to deserialize them into something like this:

@Serializable
@SerialName("fruit")
sealed class Fruit {
  abstract val name: String
}

@Serializable
@SerialName("apple")
data class Apple(
  override val name: String,
  val numAppleSeeds: Int
): Fruit()

@Serializable
@SerialName("tomato")
data class Tomato(
  override val name: String,
  val color: String
): Fruit()

This is almost what I want, but it expects the XML to look like this:

<?xml version="1.0" encoding="UTF-8"?>
<fruit type="apple">
  <value name="MyApple" numAppleSeeds="5">
    <!-- child elements omitted for simplicity -->
  </value>
</fruit>

<?xml version="1.0" encoding="UTF-8"?>
<fruit type="tomato">
  <value name="MyTomato" color="red">
    <!-- child elements omitted for simplicity -->
  </value>
</fruit>

But I need it to look more like the JSON serialized from the same class hierarchy with all attributes on the same element:

{"type":"apple","name":"MyApple","numAppleSeeds":5}

{"type":"tomato","name":"MyTomato","color":"red"}

What's the easiest way to achieve this? Is there some annotation to put somewhere, that puts the attributes of the <value> element directly on the <fruit> element? Or do I have to write a custom serializer?

Unfortunately I don't have control over the XML-Format, because it is generated by a third-party program.

I played around with this, but couldn't come up with a satisfactory solution so far.

Thank you in advance!

Hi,

not sure what I'm missing but after updating to kotlinx-serialization-runtime 0.20.0 and xmlutil-serialization-jvm 0.20.0.0 (from 0.14.0 and 0.14.0.2).

Following code:

@Serializable
@XmlSerialName(value = "metadata", namespace = "", prefix = "")
data class Metadata(
    @XmlElement(true)
    val groupId: String = "NONE",
    @XmlElement(true)
    val artifactId: String = "NONE",
    @XmlElement(true)
    val version: String = "NONE"
)

fun main() {

    val response = """<?xml version="1.0" encoding="UTF-8"?>
<metadata>
    <groupId>com.test</groupId>
    <artifactId>test</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</metadata>"""

    val xmlMapperX = XML {
        indent = 4
    }

    println(xmlMapperX.parse(Metadata.serializer(), response))
}

Produces:

Exception in thread "main" nl.adaptivity.xmlutil.XmlException: Namespace null does not match expected ""
	at nl.adaptivity.xmlutil.StAXReader.require(StAXReader.kt:113)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.endStructure(XMLDecoder.kt:748)
	at com.test.Metadata$$serializer.deserialize(XmlTest.kt)
	at com.test.Metadata$$serializer.deserialize(XmlTest.kt:11)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$XmlDecoder.decodeSerializableValue(XMLDecoder.kt:140)
	at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:334)
	at nl.adaptivity.xmlutil.serialization.XML.parse(XML.kt:352)
	at com.test.XmlTestKt.main(XmlTest.kt:33)
	at com.test.XmlTestKt.main(XmlTest.kt)

The library has less focus on documentation than ideal. Better documentation is desirable.

While the library should support Javascript this has not been well-tested. A Javascript based test suite is needed.

What is about supporting of Kotlin Native?

Hi Paul,
is there a reason why it's still mentioned implementation("...:0.13.0.1") ?

I find it cool to have latest versions reflected in quick sample and instructions samples.
I must admit, it's just a cosmetic thing. Thus if you have another opinion or find it too silly, nevermind and set the "issue" to wontfix :-D

I believe, it is possible via custom XMLWriter, but I did not manage to find a simple way to do it.

What I want is to provide XML without closing tags for elements without children. But probably there are other options that could be moved to the top level.

Getting

java.lang.ArrayIndexOutOfBoundsException
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.decodeSerializableElement(XMLDecoder.kt:535)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$PolymorphicDecoder.decodeSerializableElement(XMLDecoder.kt:1054)
	at kotlinx.serialization.internal.AbstractPolymorphicSerializer.deserialize(AbstractPolymorphicSerializer.kt:59)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoder.decodeNullableSerializableElement(XMLDecoder.kt:568)
	at scientifik.gdml.GDMLVolume$$serializer.deserialize(GDMLNode.kt)

on read from complicated input. Specifically this test.

How can I get mapOf attribute names and values?
Sample xml would be:

<a b="c" d="e" f="g" .../>

Every attribute is unique so mapOf<String,String> is very good for me.

I'm using 0.20.0.0 version and thank you so much for your work.

Task :core:compileKotlinAndroid FAILED
e: D:\Work\misc\xmlutil\core\src\commonMain\kotlin\nl\adaptivity\xmlutil\NamespaceHolder.kt: (130, 57): Return type of 'getPrefixes' is not a subtype of the return type of the overridden member 'public abstract fun getPrefixes(p0: String!): (Mutable)Iterator<String!>! defined in javax.xml.namespace.NamespaceContext'
e: D:\Work\misc\xmlutil\core\src\javaShared\kotlin\nl\adaptivity\xmlutil\_actualAliasses.kt: (23, 18): Actual class 'NamespaceContext' has no corresponding members for expected class members:

    public abstract expect fun getPrefixes(namespaceURI: String): Iterator<Any?>

    The following declaration is incompatible because return type is different:
        public abstract fun getPrefixes(p0: String!): (Mutable)Iterator<String!>!

It is probably due to changes in XML API.

(I'm relatively new to Kotlin serialization, so apologies if any of this is missing something simple)

I'm trying to parse/write XML where the value of a tag is essentially a mixed subset of HTML. For example:

<tag>some text <b>some bold text<i>some bold italic text</i></b></tag>

I have this working similarly to the MixedValueContainer example from the unit tests with data classes for tag, b, and i all containing List<@Polymorphic Any> children and a SerializersModule to support serialization.

The main drawback here is the lack of compile-time type safety around what can go in these lists. What I'd like to do is ensure that the lists only contain b, i, or String elements. AFAIK, this can't be done with String directly because you can't declare something like an interface or sealed class that String implements.

One somewhat promising approach was to define a wrapper class for a String with a custom serializer that just serializes/deserializes the underlying string, and then define a sealed class with this wrapper class as well as the other tags. This introduces full type safety and works when serializing an object; however, deserialization seems to fail because it doesn't know what to do with the vanilla string in the XML.

Is it feasible to define a mixed value container with only some restricted subset of valid child types at compile time which works for both serializing and deserializing?

I recently run into an issue where the XML I am consuming has tags like this: DISPLAYCONTACT.FIRSTNAME

My entity defines them like this:

    @XmlElement(true)
    @SerialName("DISPLAYCONTACT.FIRSTNAME")
    val firstName: String? = null,

in the same entity class all XML elements that do not have a . work fine and getting deserialized accordingly but all the once with . just getting ignored/skipped.

Due to Kotlin/kotlinx.serialization#1331 libraries are not backward compatible on JS-IR. Is it possible to publish a 1.1.0-RC compatible version?

Here is the issue from Jupyter kernel: Kotlin/kotlin-jupyter#131
It works fine for Json, so I think the problem is with XmlUtil.

Here is the stack trace:

java.lang.NoSuchMethodError: 'boolean kotlinx.serialization.descriptors.SerialDescriptor.isInline()'
nl.adaptivity.xmlutil.serialization.structure.XmlDescriptor$Companion.from$xmlutil_serialization(XmlDescriptor.kt:153)
nl.adaptivity.xmlutil.serialization.structure.XmlDescriptor$Companion.from$xmlutil_serialization$default(XmlDescriptor.kt:132)
nl.adaptivity.xmlutil.serialization.structure.XmlRootDescriptor.getElementDescriptor(XmlDescriptor.kt:182)
nl.adaptivity.xmlutil.serialization.XML.encodeToWriter(XML.kt:186)
nl.adaptivity.xmlutil.serialization.XML.toXml(XML.kt:316)
nl.adaptivity.xmlutil.serialization.XML.encodeToString(XML.kt:130)
nl.adaptivity.xmlutil.serialization.XML.encodeToString(XML.kt:114)
space.kscience.gdml.SerializationKt.encodeToString(serialization.kt:102)

The reference is present in the module file, but the artifact itself is missing.

I'd be able to distinguish if the content of a XML tag uses CDATA or not, and potentially if it uses '<' or '&lt;'.

With my current setup, the text I get from these 2 exemples is the same result.

@Serializable
data class Foo(
    @XmlValue(value = true) val text: String = ""
)

<foo>1 &lt; 2</foo>
<foo><![CDATA[1 < 2]]</foo>