Comments (8)
The proposed future "encoding/json/jsontext" package produces a better error message:
jsontext: invalid surrogate pair `\udca0'none'` within string
as it preserves the context that this is occurring within a surrogate pair.
from protobuf.
U+DCA0 is a Unicode surrogate, and should not appear in valid UTF-8 text. A conformant UTF-8 decoder is required to reject input containing an unpaired surrogate; see "How do I convert an unpaired UTF-16 surrogate to UTF-8?" here: https://unicode.org/faq/utf_bom.html
I'm not sure about the history that led to encoding/json
accepting this invalid input and protojson
rejecting it, but I don't think protojson
is doing the wrong thing here.
from protobuf.
I think this is still generating an improper error message. It seems to be complaining that the invalid escape code is 'none'
(which is not at all being used as an escape code), when it should probably be reporting the invalid escape code to be dca0
, right?
from protobuf.
Yes, you're right; I wasn't looking at the error message. The error is definitely wrong, or at least confusing.
Also, the protojson parser expects a surrogate to be followed by \uXXXX
. I'm not certain if we should be expecting the potential for input like \udca0
(the surrogate) followed by a single unescaped character with no \u
prefix.
from protobuf.
Yeah… not sure what the proper behavior should be in error conditions. But also, do we check at all that a Hi Surrogate is followed by a Lo Surrogate, and vice versa, or do we just check that surrogates occur in pairs?
I smell the familiar scent of a rabbit role into a pedantic reading of standards… 😩 At least the minimal solution here seems to be that if a surrogate isn’t followed by a surrogate, we report the escape itself as invalid? Or some sort of message that surrogates must be paired?
from protobuf.
The JSON mapping for protobuf follows RFC 7493, which requires strict checking of surrogate halves (see section 2.1 of RFC 7493). The "encoding/json" package only adheres to RFC 8259, which leaves split surrogate halves as undefined behavior (see section 8.2 of RFC 8259). As a side note, the "encoding/json/v2" draft proposal will target compliance with RFC 7293 by default.
I'm not certain if we should be expecting the potential for input like \udca0 (the surrogate) followed by a single unescaped character with no \u prefix.
A standalone surrogate half is invalid and should be rejected. The validity of a surrogate pair is determine using utf16.DecodeRune
.
I don't see anything wrong with what protojson
is doing. It correctly identified a surrogate half and was now expecting another escaped surrogate half. The error message says:
proto: syntax error (line 1:9): invalid escape code "'none'" in string
Strictly speaking, this error message is correct as 'none'
is literally an invalid escape code.
from protobuf.
Interesting, I see what you mean with the “'none'
is an invalid escape code” message. It also just coincidentally has the same length one would expect a \uXXXX
escape sequence to be.
I do like the error message from jsontext
which makes it clear exactly what is actually going wrong.
from protobuf.
Improving the error message seems reasonasble to use and we are happy to accept a contribution.
from protobuf.
Related Issues (20)
- go_features.proto has incorrect package and error-prone file path HOT 5
- Tracking issue: Fully Lazy Extension decoding HOT 7
- feature: Generate test client / `bufconn` constructors HOT 4
- v1.33.0 breaks bazel build HOT 14
- Reflection: protopath parsing and path+message access HOT 7
- Proposal: Set go.mod Go language version to oldest supported Go version (1.21 currently) HOT 8
- Proposal: Add `Override` mode to `proto.Merge` HOT 2
- editions: maps are incorrectly serialized if file has default message encoding of DELIMITED HOT 8
- protodesc: too strict about proto3 field names when creating descriptors from FileDescriptorProto
- undefined: descriptorpb.Default_FileOptions_PhpGenericServices HOT 2
- `ProtoReflect().GetUnknown()` Not Working HOT 4
- Editions language feature: (pb.go).struct_tag HOT 11
- encoding/protojson: handling google.protobuf.Empty HOT 15
- Make M option overwrite `option go_package` in the `.proto` file (as opposed to return an error) HOT 5
- related to #1054 and similar "plugin output is unparseable" errors HOT 4
- Random 0xfeff / ZERO WIDTH NO-BREAK SPACE being added to returned string values HOT 3
- custom options doesn't seem to work for protobuf map fields HOT 2
- Can protobuf provide tags for customized message generated fields? HOT 2
- encoding/protojson: unmarshalBytes assumes no padding if byte string length is not divisible by 4 HOT 27
- Service crash caused by encoding HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protobuf.