polydawn / refmt Goto Github PK

View Code? Open in Web Editor NEW

46.0 5.0 14.0 640 KB

Object mapping for golang.

License: MIT License

Go 99.07% Shell 0.93%

stream tokenizer cbor json objectmapper polymorphic serialization

refmt's People

Contributors

Stargazers

Watchers

Forkers

whyrusleeping dignifiedquire kubuxu aboodman zenground0 stebalien wade-welles backwardn jonnycrunch holajiawei daotlresearch hunjixin masih rvagg

refmt's Issues

Inconsistency in spelling of (un)marshaler.

The README says things like:

pair cbor.Decoder with obj.Unmarshaller to get a cbor deserializer.

When I look at https://godoc.org/github.com/polydawn/refmt/obj, I don't find obj.Unmarshaller there.

That's because it's spelled with a single l rather than double l in code:

https://godoc.org/github.com/polydawn/refmt/obj#NewUnmarshaler

I've done a quick search, and it's not just README that's inconsistent, there's some internal code that uses different variations:

Note, the Go project uses the single l variation. See https://dmitri.shuralyov.com/idiomatic-go#use-consistent-spelling-of-certain-words for reference.

[cbor] Unable to use numbers as map keys

I am trying to marshal a type map[uint64]*somestruct, but cannot do so, as I get the error messags unsupported map key type "uint64". This is really unfortunate, because one of the great things about cbor is that I can use numbers as keys in maps.

Code that throws the error: https://github.com/polydawn/refmt/blob/master/obj/marshalMapWildcard.go#L24-L55

Support for arbitrary structs without explicitly registering them

To fully replace encoding/json, it would be great to be able to pass an arbitrary interface{} to refmt and have it auto-generate the necessary atlas entries.

What are the performance implications of this? I assume the tricky part here is caching these auto-generated atlases.

UTF-8 versus bytes

In go, string is just an immutable version of []byte. Unfortunately, CBOR (and possibly JSON?) expect strings to be UTF-8 encoded unicode.

So, the immediate issue here is that one can end up producing non-utf8 strings in CBOR objects. Would it be possible to check on encode and either fall back on a byte array (preferable) or error out?

Possible to create my own raw marshal/unmarshal?

We have a few objects where refmt serialization is the bottleneck in our app... taking up to 1ms to serialize an object.

I'm wondering if it's possible to specify a fast-path for these objects... here it seems to say I might be able to have my own machine:

 For the most esoteric needs, you can fall all the way back to providing a custom MarshalMachine
     (but avoid that; it's a lot of work, and one of these other transform methods should suffic

However, the MarshalMachine interface has unexported type expectations in the fields.

Any other suggestions for keeping refmt around but being able to register my own "fast-path" ?

cbor tags?

Is this a thing I can do?

Things fail with weird errors if your atlas doesnt know about all your types

I had a type that looked something like:

type A struct {
  Foo []B
}

type B struct {
  Cats uint64
  ParrotQuote string
  DungHeap Heap
}

type Heap struct {
  // heap related fields
}

And I only made an atlas entry for A. When marshaling an instance of A however, it failed with cryptic messages such as "value already consumed". I finally figured out that making atlas entries for B and Heap made things happier, but it took me a while.

Allow pointers when using UseTag

Currently, when decoding into a blank interface (interface{}), some formats (e.g., CBOR), support picking the atlas entry by tag. Unfortunately, atlas entries can't be built for types behind pointers. Normally, this isn't an issue as refmt will just "do the right thing" however, in this case, refmt doesn't have enough information to do that.

It would be nice if there were some way to tell refmt to deserialize to a pointer.

Possible solutions:

Add a method to the atlas builder.
Allow types behind pointers in atlas.BuildEntry. refmt would still "do the right thing" but it would now have enough information to pick the right "default form" (behind pointer or not).

Context: ipfs/go-ipld-cbor#30. We want to deserialize "cids" to *cid.Cid but can only deserialize them to cid.Cid.

No go.mod, so modern idioms for building and testing fail?

README.md seems to be lacking Instructions for how to build this repo.

The usual "go test ./..." fails with "directory prefix . does not contain main module"

Eventually, the determined user will look at .travis.yml and be enlightened, but a line or two in README.md wouldn't hurt.

panic("todo") when marshalling cbor tags

Hey,

I am trying to use the new tag functionality discussed in #7. But I have run into a beautiful panic todo.

You can see the trace below.

--- FAIL: TestBasicMarshal (0.00s)
panic: todo [recovered]
	panic: todo

goroutine 8 [running]:
testing.tRunner.func1(0xc4200b23c0)
	/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:711 +0x2d2
panic(0x11b56e0, 0x121abe0)
	/usr/local/Cellar/go/1.9.1/libexec/src/runtime/panic.go:491 +0x283
github.com/polydawn/refmt/obj._yieldMarshalMachinePtr(0xc4200c52d0, 0xc420076390, 0xc4200763c0, 0x12e8a60, 0x11d88e0, 0x10, 0x11c2a60)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshalSlab.go:141 +0x718
github.com/polydawn/refmt/obj.(*marshalSlab).requisitionMachine(0xc420082a50, 0x12e8a60, 0x11d88e0, 0x12e8a60, 0x11e74a0)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshalSlab.go:60 +0x179
github.com/polydawn/refmt/obj.(*marshalMachineWildcard).Reset(0xc4200c51a8, 0xc420082a50, 0x11c2a60, 0xc42004caf0, 0x94, 0x12e8a60, 0x11c2a60, 0x11b56e0, 0xc42004cae0)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshalWildcard.go:26 +0xe7
github.com/polydawn/refmt/obj.(*Marshaller).Recurse(0xc420082a50, 0xc4200b6d20, 0x11c2a60, 0xc42004caf0, 0x94, 0x12e8a60, 0x11c2a60, 0x12e45a0, 0xc4200c51a8, 0x1365000, ...)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshal.go:100 +0xe2
github.com/polydawn/refmt/obj.(*marshalMachineMapWildcard).Step(0xc4200c5050, 0xc420082a50, 0xc420082a50, 0xc4200b6d20, 0x0, 0x0, 0x0)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshalMapWildcard.go:78 +0x1e6
github.com/polydawn/refmt/obj.(*Marshaller).Step(0xc420082a50, 0xc4200b6d20, 0x0, 0x0, 0x0)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/obj/marshal.go:58 +0x50
github.com/polydawn/refmt/shared.TokenPump.Run(0x12e3720, 0xc420082a50, 0x12e36a0, 0xc420082aa0, 0x0, 0xc4200c5000)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/shared/pump.go:31 +0x5b
github.com/polydawn/refmt/cbor.(*Marshaller).Marshal(0xc420076a80, 0x11c6aa0, 0xc420076a50, 0xc4200763c0, 0xc420076a80)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/cbor/cborHelpers.go:54 +0x83
github.com/polydawn/refmt/cbor.MarshalAtlased(0x11c6aa0, 0xc420076a50, 0xc420076390, 0xc4200763c0, 0x10b6c81, 0xc4200a0000, 0xc420010700, 0x34, 0x40)
	/Users/dignifiedquire/.go/src/github.com/polydawn/refmt/cbor/cborHelpers.go:39 +0x81
github.com/ipfs/go-ipld-cbor.DumpObject(0x11c6aa0, 0xc420076a50, 0xc4200a0000, 0x0, 0x0, 0xc420048e08, 0x10b6d32)
	/Users/dignifiedquire/.go/src/github.com/ipfs/go-ipld-cbor/node.go:424 +0x51
github.com/ipfs/go-ipld-cbor.WrapObject(0x11c6aa0, 0xc420076a50, 0x12, 0xffffffffffffffff, 0x1, 0x34, 0x0)
	/Users/dignifiedquire/.go/src/github.com/ipfs/go-ipld-cbor/node.go:165 +0x4d
github.com/ipfs/go-ipld-cbor.TestBasicMarshal(0xc4200b23c0)
	/Users/dignifiedquire/.go/src/github.com/ipfs/go-ipld-cbor/node_test.go:84 +0x2a3
testing.tRunner(0xc4200b23c0, 0x12043f0)
	/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
	/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:789 +0x2de

The

test is here
atlas is here
marshal call is here
the branch is https://github.com/ipfs/go-ipld-cbor/blob/refmt

From what I gathered this seems to come from here: https://github.com/polydawn/refmt/blob/master/obj/marshalSlab.go#L141 but I am not sure if this is something I should fix in my code or this needs to be fixed in the library.

[cbor] Unable to roundtrip map[string]*CustomStruct

This might be a bug in the library or a user error, I am not a 100% certain.

When using atlas.BuildEntry(CustomStruct{}).StructMap().Autogenerate() with a struct like this

type CustomStruct{
  Hello int
}

And I try roundtrip the following structure

list := map[string]*CustomStruct{
  "hello": &CustomStruct{Hello: 1},
  "world" &CustomStruct{Hello: 9},
}

I get back

map[string]*CustomStruct{
  "hello": &CustomStruct{Hello: 9},
  "world" &CustomStruct{Hello: 9},
}

When I check the raw bytes, it gets serialized correctly, but on the output it seems the values are overwriting each other somehow.

Panics and OOMs in cbor decoding

Hi, I've started fuzzing go-ipld-cbor in which we use the refmt.

I've found following crashers: https://ipfs.io/ipfs/QmeGTCjuXnjVzQHZJ8krt7UtPqEKpCCDpK2T9RBJA7DtMm/ (tar here: https://ipfs.io/ipfs/QmV7nc5WhA4k5rpdtjQEC9G66boq13FtaNoec7g3jFiUZP)

You can test them with refmt cbor=pretty < crasherFile.

Some of them cause out of memory, some are tagged timeout but those probably are OOM as well.

cbor output is not canonical

Keys in maps need to be sorted first by length, then lexicographically within that, ref: https://tools.ietf.org/html/rfc7049#section-3.9

If I can get a pointer on where to look to add this, i'll take a hack at it.

Can't round trip nil slice

package main

import (
	"github.com/polydawn/refmt/cbor"
	"github.com/polydawn/refmt/obj/atlas"
)

type Thing struct {
	Array []byte
}

func main() {
	atlas := atlas.MustBuild(
		atlas.BuildEntry(Thing{}).StructMap().Autogenerate().Complete(),
	)

	b, err := cbor.MarshalAtlased(Thing{}, atlas)
	if err != nil {
		panic(err)
	}

	var t Thing
	err = cbor.UnmarshalAtlased(b, &t, atlas)
	if err != nil {
		panic(err) // fails here.
	}
}

Fails to decode a valid CBOR object

Specifically (base64 encoded):

pWNtYXgYGmNtaW4AZHByZXb3ZWJsb2Nr2CpYJQABcRIgnFmrwDoxE64kVL5FW13qCRd7b2Ik1wiT
53QXBnkSmNtlaW5kZXiic2VkZ2UuYXguc2YuVGVybWluYWyBa3RoZVRlcm1pbmFsdGVkZ2UuYXgu
c2YuVWlTZXNzaW9ugmg0RTgxNUU2M2lTb21lIEJvZHk=

This should decode to (python syntax):

{'max': 26, 'min': 0, 'prev': None, 'block': Tag(42, b'\x00\x01q\x12 \x9cY\xab\xc0:1\x13\xae$T\xbeE[]\xea\t\x17{ob$\xd7\x08\x93\xe7t\x17\x06y\x12\x98\xdb'), 'index': {'edge.ax.sf.Terminal': ['theTerminal'], 'edge.ax.sf.UiSession': ['4E815E63', 'Some Body']}}

Error:

Error: Invalid majorByte: 0xf7

Test case:

On go-ipfs 0.4.18, run: ipfs dag get zdpuArHM9moUGtzo47sHeQ5qS4jiuQApyMnFFLoBDdYeouwHy

Downstream bug ipfs/kubo#5776.

Is it possible to defer unmarshaling?

I basically want the equivalent of https://golang.org/pkg/encoding/json/#RawMessage

I can look at implementing this if its not already possible.

The handling of nil byte slices is different than the handling of other types of nil slices

type T struct{}
var t1, t2 []T
data, _ = cbor.Marshal(t1)
cbor.Unmarshal(data, &t2)
assert.Nil(t1)  // true
assert.Nil(t2)  // true

var b1, b2 []byte
data, _ := cbor.Marshal(b1)
cbor.Unmarshal(data, &b2)
assert.Nil(b1) // true
assert.Nil(b2) // false

Emptish objects fail to roundtrip cbor marshal & unmarshal

Trying some simple edge cases and got these issues

Emptyish objects that fail to round trip

(fmt.Printf("%s - %s\n", obj, reflect.TypeOf(obj)))

[] - interface[]{}: panic: cborEncoder stack overpopped (marshal)
[<nil>] - interface[]{}: Invalid majorByte: 0xff (unmarshal)
<nil> - %!s(<nil)): panic: cborEncoder stack overpopped (marshal)

(Setup is as described in #9)

Encoding zero values as null

Currently, refmt:

Will decode null to a zero value (i.e., MyStruct{})
Can be instructed to skip zero values when encoding by specifying "omitempty".

However, there isn't a way to say "encode zero values to null" (the inverse of 1). This can be useful when we always want to have a field present in an object but would like to make it clear that the field is empty.

Specifically, this would be really useful for CIDs in IPLD. Users can currently specify "omitempty" but, if they fail to include this, encoding a zero-value CID will return an error. With some way to say "encode the zero value to null", this would "just work".

Proposal: Add a Nullable(nullable bool) option to the atlas entry builder.

Check atlas.BuildEntry type hint arg is not a pointer.

obj/atlas.BuildEntry should check that the interface{} param it uses as a type hint is not a pointer (or, it should fully dereference it?). The rest of the atlas logic expects fully deferenced pointers and types, so providing a pointer to this method causes behavior that's somewhere between weird and useless.

Support skipping unknown fields

Otherwise, forwards compatibility is difficult.

refmt atlas generation from tags should error loudly if tags are repeated

Repeating the same field name in a struct's tags should raise an error.

Currently it seems to silently stop serializing either field. Which is very strange and clearly wrong.

blocking parallel execution?

Hi! Love the project. Really great work.

While working with the go IPLD library. I noticed real-world slow down in my app that looked like blocking behavior... up to 200-600ms for a simple action to complete.

I eventually narrowed it down to a line in the IPLD cbornode library and then continued down the rabbit hole into refmt. ipfs/go-ipld-cbor#37

It seems like individual encoding/decoding is fast, but that somehow decoding can block other decodes. I haven't yet delved into the internals of refmt to check this out.

Here's a failing test with 160ms decodes. It has 10k parallel processes, but in my real-world app I had hundreds but more complicated objects showing the same behavior (or longer times).

https://gist.github.com/tobowers/d7a0e05d68459da54808a65c145c50d5

Using a transform function to/from an interface cause a nil pointer dereference

At least when used with CBOR tags.

Repro: master...Stebalien:repro/nil-deref

Trace:

  * /home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go 
  Line 1540: - runtime error: invalid memory address or nil pointer dereference 
  goroutine 6 [running]:
  panic(0x5d2260, 0x78f230)
  	/usr/lib/go/src/runtime/panic.go:513 +0x1b9
  github.com/polydawn/refmt/obj.(*unmarshalMachineWildcard).Step(0xc00033c058, 0xc0000e9c80, 0xc0000e9c80, 0xc0002ce9a0, 0x5fdc60, 0xc00047ed01, 0xc0002ce9a0)
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/unmarshalWildcard.go:32 +0x39
  github.com/polydawn/refmt/obj.(*Unmarshaller).Step(0xc0000e9c80, 0xc0002ce9a0, 0x2, 0x2, 0x1)
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/unmarshal.go:60 +0x4c
  github.com/polydawn/refmt/obj.TestUnmarshaller.func1.1.1.2()
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go:1540 +0x1d1
  github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:97 +0x3f2
  github.com/jtolds/gls.EnsureGoroutineId(0xc000487500)
  	/home/steb/projects/go/src/github.com/jtolds/gls/gid.go:19 +0x103
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc00004a550, 0xc0004874a0, 0xc0002c2e20)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:63 +0x147
  github.com/polydawn/refmt/obj.TestUnmarshaller.func1.1.1()
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go:1532 +0x2b3
  github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:97 +0x3f2
  github.com/jtolds/gls.EnsureGoroutineId(0xc0004873e0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/gid.go:19 +0x103
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc00004a550, 0xc000487380, 0xc0002c2de0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:63 +0x147
  github.com/polydawn/refmt/obj.TestUnmarshaller.func1.1()
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go:1520 +0x3bd
  github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:97 +0x3f2
  github.com/jtolds/gls.EnsureGoroutineId(0xc0004872c0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/gid.go:19 +0x103
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc00004a550, 0xc000487260, 0xc0002c2d60)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:63 +0x147
  github.com/polydawn/refmt/obj.TestUnmarshaller.func1()
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go:1511 +0x20d
  github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:97 +0x3f2
  github.com/jtolds/gls.EnsureGoroutineId.func1()
  	/home/steb/projects/go/src/github.com/jtolds/gls/gid.go:24 +0x2e
  github.com/jtolds/gls._m(0x0, 0xc0003b85c0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/stack_tags.go:74 +0x31
  github.com/jtolds/gls.github_com_jtolds_gls_markS(0x0, 0xc0003b85c0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/stack_tags.go:54 +0x35
  github.com/jtolds/gls.addStackTag(0x0, 0xc0003b85c0)
  	/home/steb/projects/go/src/github.com/jtolds/gls/stack_tags.go:49 +0x3a
  github.com/jtolds/gls.EnsureGoroutineId(0xc00039f410)
  	/home/steb/projects/go/src/github.com/jtolds/gls/gid.go:24 +0xc3
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc00004a550, 0xc00039f3b0, 0xc0003b8580)
  	/home/steb/projects/go/src/github.com/jtolds/gls/context.go:63 +0x147
  github.com/polydawn/refmt/obj.TestUnmarshaller(0xc0000c6200)
  	/home/steb/projects/go/src/github.com/polydawn/refmt/obj/objFixtures_test.go:1509 +0x99
  testing.tRunner(0xc0000c6200, 0x616c50)
  	/usr/lib/go/src/testing/testing.go:827 +0xbf
  created by testing.(*T).Run
  	/usr/lib/go/src/testing/testing.go:878 +0x353

Nested Cbor Tags are not unmarshalled properly

Something strange is going on when trying to unmarshal cbor tags that are nested. Object in question can be found here: https://github.com/ipfs/go-ipld-cbor/blob/refmt/node_test.go#L124-L133

obj := map[string]interface{}{
    "foo":   "bar",
    "hello": c1,
    "baz": []interface{}{
      c1,
      c2,
    },
    "cats": map[string]interface{}{
      "qux": c3,
    },
  }

The byte code looks fine

a36362617a82d82a5823001220aed1fe98cda4ba5a1681a19aa768f73b9d707c5621c7effdf2938e242080505ed82a58230012205773487d221545d26fd0f57fdb3a7d986bc479a850d7b0d762e8c7f4772790a06463617473a163717578d82a5823001220434be0538b88f385bdb9d62de34e296a0908c857a876aec09e1f52673596d89763666f6f63626172

Diagnostic view from cbor.me

{"baz": [42(h'001220AED1FE98CDA4BA5A1681A19AA768F73B9D707C5621C7EFFDF2938E242080505E'), 42(h'0012205773487D221545D26FD0F57FDB3A7D986BC479A850D7B0D762E8C7F4772790A0')], "cats": {"qux": 42(h'001220434BE0538B88F385BDB9D62DE34E296A0908C857A876AEC09E1F52673596D897')}, "foo": "bar"}

But after unmarshalling this is what I get as json representation:

original

{"baz": [{"/":"Qma75NN8GaM99ioqsNUF9Ho816SonoGsVrvSnqz9uL4LPF"},{"/":"QmUE28rcN99es8ntD4T3sBScfyrADkF3q8qzM1gEW82oMh"}],"cats":{"qux":{"/":"QmSsM8Xm1g5WtfwkBvnttZafpDhaW6jkXeSFccECUnx7hg"}},"foo":"bar","hello":{"/":"Qma75NN8GaM99ioqsNUF9Ho816SonoGsVrvSnqz9uL4LPF"}}

after roundtrip

{"baz":[{},{}],"cats":{"qux":{}},"foo":"bar","hello":{}}

.gitmodules has extra trailing slashes that break "git submodule update --init"

Here's the symptom:

$ git clone https://github.com/polydawn/refmt
$ cd refmt
$ git submodule update --init
...
fatal: clone of 'https://github.com/go-yaml/yaml/' into submodule path '/Users/dkegel/src/refmt/.gopath/src/github.com/go-yaml/yaml' failed
fatal: clone of 'https://github.com/urfave/cli/' into submodule path '/Users/dkegel/src/refmt/.gopath/src/github.com/urfave/cli' failed

The fix is to remove the trailing slashes on those two lines in .gitmodules.

Json encoder float support is todo

Hi friend, present for you:

panic: TODO finish more jsonEncoder primitives support: unhandled token <f:1> [recovered]
        panic: TODO finish more jsonEncoder primitives support: unhandled token <f:1>

goroutine 18 [running]:
testing.tRunner.func1(0xc420098d20)
        /home/why/go/src/testing/testing.go:711 +0x2d2
panic(0x5c8320, 0xc42013a800)
        /home/why/go/src/runtime/panic.go:491 +0x283
github.com/polydawn/refmt/json.(*Encoder).flushValue(0xc42009b080, 0xc420142070)
        /home/why/gopkg/src/github.com/polydawn/refmt/json/jsonEncoder.go:189 +0x233
github.com/polydawn/refmt/json.(*Encoder).Step(0xc42009b080, 0xc420142070, 0x0, 0x0, 0x0)
        /home/why/gopkg/src/github.com/polydawn/refmt/json/jsonEncoder.go:134 +0xb59
github.com/polydawn/refmt/shared.TokenPump.Run(0x6e97a0, 0xc420105f40, 0x6e9760, 0xc42009b080, 0x0, 0xc42009b080)
        /home/why/gopkg/src/github.com/polydawn/refmt/shared/pump.go:35 +0x95
github.com/polydawn/refmt/json.(*Marshaller).Marshal(0xc4200b1de8, 0x5cb500, 0xc4200e1c50, 0xc4200636d0, 0x0)
        /home/why/gopkg/src/github.com/polydawn/refmt/json/jsonHelpers.go:54 +0x8b
github.com/polydawn/refmt/json.MarshalAtlased(0x5cb500, 0xc4200e1c50, 0xc42005c330, 0xc42005c360, 0x12, 0xffffffffffffffff, 0xc420140000, 0x0, 0x0)
        /home/why/gopkg/src/github.com/polydawn/refmt/json/jsonHelpers.go:39 +0x2b0
github.com/ipfs/go-ipld-cbor.(*Node).MarshalJSON(0xc420140000, 0x0, 0x40, 0x12, 0xffffffffffffffff, 0xc420140000)
        /home/why/gopkg/src/github.com/ipfs/go-ipld-cbor/node.go:438 +0x4f
github.com/ipfs/go-ipld-cbor.TestExamples.func1(0xc420098d20)
        /home/why/gopkg/src/github.com/ipfs/go-ipld-cbor/node_test.go:385 +0x296
testing.tRunner(0xc420098d20, 0xc42000bec0)
        /home/why/go/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
        /home/why/go/src/testing/testing.go:789 +0x2de
exit status 2

Remove random binaries from git repo

😠

Option to encode nil slice/map as []/{}

Given that:

Empty slices/maps are often represented as nil in go (i.e., go doesn't really care).
Most other languages distinguish between null and an empty list/object.

It would be nice to have an option to always encode empty slices/maps as []/{}.

Unfortunately, doing this with an atlas doesn't appear to be possible as maps/slices are generic.