Comments (19)
In this exact case:
<0a 0d 08 f9 27 12 02 4f 4b 18 8a 8c 06 20 4e>
0A hex = 1 | 010 bin which is the first tag constructed from two values (marked by "|") = wire type 2 (=010), id 1 (=1)
0a<0d 08 f9 27 12 02 4f 4b 18 8a 8c 06 20 4e>
wire type 2 is a length delimited value (see), so
0D hex = 13 dec is the length (which is single byte varint of 0001 0011 bin actually, which is easy to decode by a simple binary to decimal conversion)
which is the rest of the data.
Assuming that this is an inner message with a length of 13 bytes we get for its contents:
0a 0d<08 f9 27 12 02 4f 4b 18 8a 8c 06 20 4e>
08 hex = 1 | 000 bin = wire type 0, id 1
wire type 0 is a varint, which is a bit difficult to calculate by hand if it is built from multiple bytes. However, we are able to determine its length:
0a 0d 08<f9 27 12 02 4f 4b 18 8a 8c 06 20 4e>
F9 hex = 1111 1001 bin with first bit set, continue
27 hex = 0010 0111 bin with first bit not set, end
You have to determine whether it is 32 or 64 bit (assuming 64 bit will always work as it will work with 32 bit values, too) and if it is unsigned (uint_), signed (int_) or zig-zag encoded (sint*, see).
0a 0d 08 f9 27<12 02 4f 4b 18 8a 8c 06 20 4e>
12 hex = 10 | 010 bin = wire type 2, id 2
wire type 2 is, as we already know, a length delimited value, so
0a 0d 08 f9 27 12<02 4f 4b 18 8a 8c 06 20 4e>
02 hex = 2 dec is the length
0a 0d 08 f9 27 12 02<4f 4b 18 8a 8c 06 20 4e>
4F 4B hex = 0100 1111 0100 1011 bin - you have to determine what actual type this is
0a 0d 08 f9 27 12 02 4f 4b<18 8a 8c 06 20 4e>
18 hex = 11 | 000 bin = wire type 0, id 3
again a varint
0a 0d 08 f9 27 12 02 4f 4b 18<8a 8c 06 20 4e>
8A hex = 1000 1010 bin with first bit set, continue
8C hex = 1000 1100 bin with first bit set, continue
06 hex = 0000 0110 bin with first bit not set, end
0a 0d 08 f9 27 12 02 4f 4b 18 8a 8c 06<20 4e>
20 hex = 100 | 000 bin = wire type 0, id = 4
again a varint
0a 0d 08 f9 27 12 02 4f 4b 18 8a 8c 06 20<4e>
4E hex = 0100 1110 bin with first bit not set, end.
0a 0d 08 f9 27 12 02 4f 4b 18 8a 8c 06 20 4e|
The buffer looks ok so far.
The original .proto could, substituting your already provided data types, look somehow like this:
message Outer {
repeated Inner inner = 1;
message Inner {
optional int64 a = 1;
optional string b = 2;
optional int64 c = 3;
optional int32 d = 4;
}
}
from protobuf.js.
If you do not have the .proto file, all you can do is to reverse engineer it from the buffer, which is possible using the protobuf documentation: https://developers.google.com/protocol-buffers/docs/encoding
Note that a couple of data types map to the same wire type (especially wire type 2 and unsigned / signed integers) and you'll have to sort out which is the right one on your own. Because of this, there is no automated approach.
from protobuf.js.
Very nice... Tnx bout your patience. Its very clear now...
Let me ask,,, isnt this:
message BuyInResponse {
enum Code {
OK = 0;
ERROR = 1;
AUTH_ERROR = 2;
}
repeated PaymentResponseElement response = 1;
}
message PaymentResponseElement {
optional int64 pnPaymentId = 1;
optional string messageCode =2;
optional int64 balanceAfterTransaction = 3;
optional int32 version = 4;
}
exactly like
message Outer {
repeated Inner inner = 1;
message Inner {
optional int64 a = 1;
optional string b = 2;
optional int64 c = 3;
optional int32 d = 4;
}
}
Why the error persist?
from protobuf.js.
I'd say yes, it is - meaning if I don't miss something. If so, it could be an encoding issue, like that the buffer becomes converted to a string somewhere and is corrupted in that process or such.
How are you obtaining the data / reading it into a ByteBuffer?
from protobuf.js.
you know if it can happen by passing the buffer by various scripts? ... currently I pass the buffer (by parameter) for 3 different files.
from protobuf.js.
If you pass it just as a function argument, this depends. Just passing it does not modify its type but any of the functions involved could possibly convert the buffer back and forth to some other data type, like a string, which might corrupt the data (like when en/-decoding to/from UTF8, US-ASCII etc.).
If you obtain it through HTTP and binaryType="arraybuffer" like with WebSockets or similar isn't available, I'd suggest that you encode it to Base64 before transmitting it over any network connection, and decode it properly to bytes prior to putting it into a byte buffer.
from protobuf.js.
tnx bout you code passion :)
from protobuf.js.
It's a good example and I've linked it from the FAQ in the wiki :)
from protobuf.js.
Another example: #143 (comment)
from protobuf.js.
since wire type 2 (Length-delimited) represents string, bytes, embedded messages, packed repeated fields, is there way to differentiate between string vs message vs repeated field ?
from protobuf.js.
Not from the raw message alone, but with the correct .proto file loaded, you have everything at hand to evaluate the reflection structure for what to expect.
The other option is guessing.
from protobuf.js.
Thanks for getting back so quick. I am basically using java to decode the raw protobuf and trying to generate the same result as protoc --decode_raw. It looks to me that there must be a way to differentiate since protoc command is doing it just by reading the raw protobuf.
from protobuf.js.
Well, protoc then probably does some guessing for you. There is no other information on the type than "length delimited", as that's all a decoder needs (like with --decode_raw). Combined with the .proto definition, it becomes interpreted as the type it is.
from protobuf.js.
I have problem with extraction of string values from binary data.
Following is the structure to extract values and corresponding binary data also.
9 {
1: "106671829932240464836"
4: "Vijayakrishnasai Gudavalli"
}
4a 33 0a 15 31 30 36 36 37 31 38 32 39 39 33 32 32 34 30 34 36 34 38 33 36 22 1a 56 69 6a 61 79 61 6b 72 69 73 68 6e
61 73 61 69 20 47 75 64 61 76 61 6c 6c 69
from protobuf.js.
Can any one help me to how to decode the binary data as specified in above comment..
from protobuf.js.
How do we differentiate length delimited string, bytes, sub messages in binary data..Can any one help me how to parse the binary data
from protobuf.js.
@venkatpathapati There is no other way to differentiate string and submessage except it's content. But even content may be confusing. So if you haven't schema you only can try to predict what message is and try to decode it.
from protobuf.js.
I am getting an index out of range exception,
when I am subscribing a message from a server which is a console application,
this is the code snippet which is sending the message
and the below is the JS snippet where the data is recieced but on continous recieving the protobufjs is throwing an index out of range exception at protobufjs\src\reader.js :13:12
can any one help me in decoding the message getting from the server continously.
from protobuf.js.
When sending and receiving multiple messages, use length delimited messages, because otherwise the decoder doesn't know where one message ends and another one starts. protobuf.js provides Message#decodeDelimited
for this purpose (your server then has to send delimited messages as well, of course). Now, you basically want a buffer on the client side that collects all the so far received data. You'd then put this into a protobuf.Reader
and provide it, instead of a buffer, to Message#decodeDelimited
so you can find out how many bytes of the buffer have been read through inspecting Reader#pos
afterwards. decodeDelimited
might be callable multiple times depending on the number of messages within the so far received buffer. Cut that chunk (0 - reader.pos
) away, rinse and repeat.
from protobuf.js.
Related Issues (20)
- Why are repeated fields in proto, in .dts file not optional?
- No timestamps
- Load .proto file content from variable HOT 2
- custom class is output undefined
- How to decode grpc-web-text
- Enum value ignored HOT 1
- Fails to parse "{" in options
- int64 precesion problem
- Enum reserved keyword removed when compiling proto from json
- Add Auth to Xhr fetch
- The problem with load
- Update from version 6.x.x to version 7.x.x
- not support eval and new Function()/Function() HOT 2
- how do you type decode(...)
- how do I type a namespace?
- how to fix property does not exist on type "Message<MessageType>"
- Cannot read properties of undefined (reading 'add') HOT 3
- protobuff.encode() does not work as expected
- Backport prototype setProperty checks to 6.11.3 HOT 5
- protobufjs-cli still pull older version of escodegen HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protobuf.js.