Comments (3)
Hey, thanks for the suggestion! Once we've added some benchmarks we can try comparing the performance of different serialization libs. Though I suspect that currently, the main bottleneck is the actual proto-lens code that does the encoding/decoding and the intermediate data structures that it uses.
There's a couple other features we'd probably need in order to try this out; I couldn't immediately tell from the docs if they're supported by store:
- Detect the end of input (e.g.,
Data.Attoparsec.ByteString.endOfInput
), since the proto encoding doesn't specify its length. - Get a substring of a specific length, e.g.,
Data.Attoparsec.ByteString.take
; or, even more convenient, isolate a parse to a specific subsequence of the data, likeData.Binary.Get.isolate
. This helps to parse sub-messages.
from proto-lens.
Detect the end of input (e.g., Data.Attoparsec.ByteString.endOfInput), since the proto encoding doesn't specify its length.
The decode
family of functions require that the the Peek
action consume exactly the whole input, no more, no less.
Get a substring of a specific length, e.g., Data.Attoparsec.ByteString.take; or, even more convenient, isolate a parse to a specific subsequence of the data, like Data.Binary.Get.isolate. This helps to parse sub-messages.
Having known-size extraction of sequences of bytes is covered by mgsloan/store#40
We do have isolate, but it's mostly just a way to skip forward those bytes and ensure the inner peek didn't go beyond. It's still the responsibility of the inner Peek to not advance beyond the isolated region (but that gets checked).
from proto-lens.
I'm closing this ticket since we ended up using our own custom parser monad.
Its implementation is fairly similar to that of store
. Compare:
http://hackage.haskell.org/package/store-core-0.4.4/docs/Data-Store-Core.html#t:Peek
https://github.com/google/proto-lens/blob/master/proto-lens/src/Data/ProtoLens/Encoding/Parser/Internal.hs#L14
In addition to the issues mentioned above, the main blocker was the lack of support for endian-specific numeric types (mgsloan/store#31).
from proto-lens.
Related Issues (20)
- Generic runtime representation HOT 2
- QuickCheck and bytestring upper bounds are outdated
- Find a path forward for CI HOT 2
- Installing protoc doc
- Text format parser doesn't accept all valid boolean values
- Too strict bounds on profunctors on hackage HOT 2
- Release for Stackage LTS-18.0 HOT 5
- Support GHC 9
- `readMessage` from `TextFormat` chokes on unicode in strings
- Re-add blackgnezdo HOT 2
- Increment upper bounds for some dependencies HOT 2
- New Hackage release HOT 3
- Support lazy construction for byte fields HOT 2
- Not compatible with mtl-2.3 and transformers-0.6 HOT 3
- google/protobuf/struct.proto not included in protobuf types HOT 2
- Show instance not working for first/default enum values
- Show instance not working for first/default enum values
- Support GHC 9.4 HOT 10
- Compilation hangs after some `default-extensions` added. HOT 1
- Replace deprecated `set-output` command with environment file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from proto-lens.