Comments (5)
Taking a step back, it seems that you want support for sorted encoding of mappings. Is that a fair summary of your request?
from go.
Yep! That's the one thing I've hit in practice that was tricky for getting deterministic output.
I'd almost say that what I want is canonical cbor output, but with the interesting exception that I also need streams, which is technically a break from what the rfc suggests for canonical encoding.
I went ahead and made a prototype that does what I needed, so I'm going full steam ahead with using codec
on that project (and it's fantastic! Thank you for all this awesome work!), but I did end up using GenHelperEncoder
to do so despite the may-change-without-notice warnings. (We pin dependencies by commit hash in that project, so I'm not worried, and totally accept I might have to nontrivial refactors in the future. :) ) If you want to see the example of how that turned out, it's here and here.
It might also be nice to have a method for encoding maps that'll do keys in an order specified by a []string keys
(well, or just expose an interface like the GenHelperEncoder
officially where map appends can be called in the desired order directly) -- this would be useful in a situation where someone is implementating an encoding for something that has to match an existing spec that's deterministic order but not sorted.
from go.
K.
I have an upcoming change to support canonical encodings.
It works by configuring an option on the handle i.e. Canonical=true.
Canonical mode is ignored for code generation. It only works for the standard reflection-based runtime introspection. Also, there is a performance hit, as each key must be encoded out-of-band, then all key encodings compared, and then the key-values are encoded.
The good part is that it fits in with the rest of the library, so is supported.
Using "internal", "unsupported" APIs will prevent you from easy upgrade, so is highly discouraged.
The change should be uploaded today. Let me know if this will work for you before I make the update.
Thanks.
from go.
The performance hit caveat makes sense. Even with that, it'd definitely be great to have this in the supported APIs.
Do you think an explicit MapAppend(k, v interface{})
is also exposable? Maybe it's not the most general API, but I think it makes sense for most encoding formats I can think of, and it seems to make sense from the way the internals work right now (though I suppose what I wrote along those lines was cbor specific...). And raising some sort of "unsupported" error an encoding format doesn't support user-chosen map orders seems totally reasonable.
from go.
This is awesome! I look forward to using it in future protocols. :D
from go.
Related Issues (20)
- codec.MsgpackHandle.RawToString is confusingly named HOT 2
- Go 1.20 - invalid memory address or nil pointer dereference HOT 5
- Msgpack deserialization with codecgen produces wrong result HOT 3
- Unexpected encoding with interface{} key using Canonical and MapKeyAsString.
- Decoding a slice of structs encoded as arrays into a slice of structs without the same number of fields HOT 2
- Duplicate key inside of a map triggers a "cannot decode unsigned integer" msgpack decode error HOT 1
- Efficient direct conversion between supported formats? HOT 2
- Expose Handle.Name() in some form through Encoder and Decoder for use by codec.Selfer HOT 1
- `io.ErrUnexpectedEOF` should be returned instead of `io.EOF` when only partial data is present
- Add `DecodeOptions.IntType` and `DecodeOptions.FloatType` ? HOT 1
- Docs do not seem to mention case-sensitive field handling
- mem leak? HOT 1
- Tests use rand.Intn(1) for boolean rand, but always returns 0
- Decoder accepts indefinite-length CBOR strings containing chunks with mismatched major types HOT 1
- UTF-8 validation isn't applied to CBOR text strings nested inside an indefinite-length text string HOT 2
- Add support for github.com/go-json-experiment/json / UnmarshalerV2 ? HOT 1
- runtime error with golang master (furture go 1.22)
- Leading zeros is not allowed in rfc7159
- attr.go
- [Question] Big Endian architecture
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go.