Comments (3)
You haven’t provided any error messages or examples of the text containing a ZWNBSP or where in the string.
However, if this is happening at the start of your string, then this is probably a result of Byte-Order Marking, where a string starts with 0xfeff, and since 0xfffe is defined as an invalid Unicode codepoint, you can then identify if you’re dealing with UTF-16LE, from UTF-16BE. Especially, if it’s pulling this data from lines from a Windows text file, like .BAT
as it is known to add these BOMs in files saved in Unicode.
from protobuf.
Sorry for that missing information. The data is come from a MySQL select query. It is essentially has a gRPC api server that node.GetWork is calling and returning this.
I have confirmed that inside of the server it is not being added. It is being added some where in the encoding and transfer across the wire and then the subsequent decode on the client side.
Yes I have been able to work around it by specifically stripping the 0xfeff character from the string but it seems odd it is there in the first place.
I also agree that this is likely the result of Byte-Order Marking because it is at the very start of the string.
What I find most odd is that it only happens sometimes, it is not every string. It almost feels as if it is being used as padding for the encode/decode for the wire transfer but is not properly being stripped off in all cases.
I am happy to provide more detail or code or debug out put, I just was not sure what all would be helpful. Or if this was some known expected behavior I was not aware of.
from protobuf.
Protobuf doesn’t typically use any padding let alone 0xfeff specifically.
Have you tried looking at the raw MySQL query values directly? Maybe someone is copy-pasting in from a Windows text file somewhere? It can be a notoriously difficult character to notice because it’s zero-width, and thus might not seem to show up normally.
Maybe a short copy of an encoded Work
message that triggers the issue? I maybe wouldn’t jump straight to copy-pasting here an excerpt of the MySQL data for that message, but also, it probably wouldn’t hurt.
from protobuf.
Related Issues (20)
- encoding/protojson: handling google.protobuf.Empty HOT 15
- Make M option overwrite `option go_package` in the `.proto` file (as opposed to return an error) HOT 5
- related to #1054 and similar "plugin output is unparseable" errors HOT 4
- custom options doesn't seem to work for protobuf map fields HOT 2
- Can protobuf provide tags for customized message generated fields? HOT 2
- encoding/protojson: unmarshalBytes assumes no padding if byte string length is not divisible by 4 HOT 27
- Service crash caused by encoding HOT 3
- structpb.Struct.MarshalJSON is not consistent HOT 4
- --gofast_out option in protoc discards inline comments from .proto files HOT 1
- How to redefine the json tag? HOT 2
- proto.Equal does not check for nested invalidity HOT 5
- A terrible work-around: HOT 2
- protoreflect x dynamicpb interaction: should descriptors match names or be pointer-identical?
- add proto.IsWellFormed helper function to validate that a message value is well-formed HOT 1
- google.protobuf.Empty as response serializes to not empty response depending on request headers HOT 2
- protocmp: Panic on embed message field comparison HOT 7
- Panic when use enum.String() before main() HOT 2
- Proposal: remove presumably unused purego support
- Why does deserialize []byte("xxxx") into a proto structure work? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protobuf.