Embedded structs cause decoding issues. about go HOT 8 CLOSED

I wouldn't get to this one soon.

Just so you are not stuck, you can change the const recoverPanicToErr to true (line 46 of helper.go). This will propagate the panic and crash the program with a stack trace when an error happens, and can get some insight into where the error is originating from.

from go.

Can you work on getting this issue to a self-contained one, so I can fix.

Something short and straight to the point, like all your other issues.

Thanks.

from go.

Yes I can. I actually have no idea what's going wrong with it, so that's why I couldn't condense it further. I'll dig in more and try to get a smaller reproduce.

from go.

Okay, here's a new reproduce. The error should be easier to see since I've made the number of structs smaller, and the msgpack much simpler.

This snippet fails to decode if the Astruct inside Bstruct is embedded, if it is named (not embedded) then I receive no error and it decodes correctly. The error is as follows:

2013/11/16 09:07:18 Failed to decode to struct: codec.decoder:
Unhandled single-byte unsigned integer value: Unrecognized descriptor byte: c4

Now, the error message is different but I believe it's a symptom of the same underlying problem. If we can get this one fixed, I'll verify that it fixes the original snippet as well.

package main

import (
    "github.com/ugorji/go/codec"
    "bytes"
    "log"
    "github.com/davecgh/go-spew/spew"
)

type Astruct struct {
    Abyte1 []byte
}

type Bstruct struct {
    *Astruct
    Bint1     int32
}

var test1 = []byte{
    0x92, //Bstruct{
    0x91, //Astruct{
    0xc4, 0x5, 0x1, 0x2, 0x3, 0x4, 0x5, // []byte{1, 2, 3, 4, 5}
    0x6, // Bint1 = 6
}

func main() {
    var m codec.MsgpackHandle
    m.StructToArray = true
    m.WriteExt = true

    b := Bstruct{}

    decoder := codec.NewDecoder(bytes.NewBuffer(test1), &m)
    if err := decoder.Decode(&b); err != nil {
        log.Println("Failed to decode to struct:", err)
    } else {
        spew.Dump(b)
    }

    var i interface{}
    decoder = codec.NewDecoder(bytes.NewBuffer(test1), &m)
    if err := decoder.Decode(&i); err != nil {
        log.Println("Failed to decode to interface:", err)
    } else {
        spew.Dump(i)
    }
}

Output when Bstruct's Astruct is not named:

2013/11/16 09:02:59 Failed to decode to struct: codec.decoder:
Unhandled single-byte unsigned integer value: Unrecognized descriptor byte: c4
([]interface {}) {
 ([]interface {}) {
  ([]uint8) {
   00000000  01 02 03 04 05                                    |.....|
  }
 },
 (int64) 6
}

Output when Bstruct's Astruct is named:

(main.Bstruct) {
 A: (*main.Astruct)(0xc2000fa280)({
  Abyte1: ([]uint8) {
   00000000  01 02 03 04 05                                    |.....|
  }
 }),
 Bint1: (int32) 6
}
([]interface {}) {
 ([]interface {}) {
  ([]uint8) {
   00000000  01 02 03 04 05                                    |.....|
  }
 },
 (int64) 6
}

from go.

K, this is a user error.

https://github.com/ugorji/go/blob/master/codec/encode.go#L663

// Anonymous fields are encoded inline if no struct tag is present.
// Else they are encoded as regular fields.

So the encoding will be like this was encoded:
type Bstruct struct {
Abyte1 []byte
Bint1 int32
}

However, you are passing something that looks like
type Bstruct struct {
type Astruct struct {
Abyte1 []byte
}
Bint1 int32
}

It wouldn't work. Your struct is not aligned with the encoded data (the encoded data says that an array is inside an array).

If you want it to work, you have 2 options:

1. Add a struct tag to the embedded field, so the Astruct is not encoded/decoded as if it's members are inline i.e. type Bstruct { *Astruct codec:"A"; Bint1 int32 }
2. Do not have the Astruct be embedded

With that, things work, and error goes away.

To see how this works, you should set the const recoverPanicToErr to true (line 46 of helper.go). This way, when an error occurs, we don't recover the panic and you see where in the code it occurred, and you can follow it. It may help.

from go.

This encoding was actually done by the C library wrapped in Python not by me. The original big scary example has data directly from that encoder. I simply reduced the problem to be this small bit.

I haven't used reflect enough to know if there's a way we can find out if the struct was embedded to detect and handle this case. However, if you're satisfied with this resolution then I am. Being able to embed the struct and encode/decode properly with a struct field literal tag is an acceptable workaround. Thanks.

from go.

its actually not a work around per se. and the problem wasn't with the
encoding. the problem was the struct.
in this scenario, the lib usage is that fields of embedded struct members
are encoded/decoded inline, except a strict tag is used to force the
regular enc/Dec. this is one of those things I thought hard about ;)

(on go ... typing from phone ... enjoy)
On Nov 15, 2013 9:06 PM, "Aaron L" [email protected] wrote:

This encoding was actually done by the C library wrapped in Python not by
me. The original big scary example has data directly from that encoder. I
simply reduced the problem to be this small bit.

I haven't used reflect enough to know if there's a way we can find out if
the struct was embedded to detect and handle this case. However, if you're
satisfied with this resolution then I am. Being able to embed the struct
and encode/decode properly with a struct field literal tag is an acceptable
workaround. Thanks.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/25#issuecomment-28617396
.

from go.

Either way, let's consider this one buried. I think with all these issues out of the way, I can start coding haha. Thanks again!

from go.