Comments (8)
Hey, thanks for reporting. Just to let you know, I've seen this, and started looking into it, but I'm feeling a bit under the weather and fixes might take a few days.
Looks like allocating 3472328296227680304
bytes is a bit of a no-go :) I guess the first round of fixes here will be to double-check the sanity of length headers before continuing on to an alloc, and that should fix the basic crashers. A "good" solution probably should count the rough total of alloc'd space over time so we can aim for finite memory usage even on outright antagonistic input -- that will take a little more coding though.
from refmt.
I went ahead and added the basic guardrails around overly-huge allocs in the byte and string paths in the cbor decoder. It fixes all the crashers in the dataset you provided, turning them into error messages instead. Those are merged to master now.
The further solution of keeping a stateful running tally of estimated memory demanded by the deserialization so far is something I'd still like to do as well, but I figured no harm in plucking the lowest hanging fruit first :)
from refmt.
Cool, this will allow me to fuzz it further. Is there any reason for 33554432 as a limit?
from refmt.
Also would you be interested in hosting fuzzing corpus and test directly in this repo? Corpus currently is very small <10KB.
from refmt.
Here is second set of crashers, all connected with either makeslice: len out of range
or slice bounds out of range
crashers2.tar.gz
from refmt.
Yeah! I've never used this kind of tool before, so I don't know exactly how to most productively incorporate it. But I'm all ears!
Does this tool have a minimizer feature that goes further? I feel some of these should be integrated as regression test cases as well, but I'm a little adding test cases that have a bunch of coincidental fluff in them.
Arbitrarily 1024*1024*32
= 32MB. This should probably be configurable. I was basically going precisely for the "Cool, this will allow me to fuzz it further" reaction :D
from refmt.
Does this tool have a minimizer feature that goes further?
Yes it does.
What you can do is have the minimized corpus tested with CI on commit (read corpus input off the file and test it).
from refmt.
Yeah, could you go ahead and fire off a PR with your setup? I'm been thinking about this more and I'd love to take whatever you've got and run with it.
from refmt.
Related Issues (20)
- Things fail with weird errors if your atlas doesnt know about all your types
- cbor output is not canonical HOT 7
- Is it possible to defer unmarshaling? HOT 1
- [cbor] Unable to roundtrip map[string]*CustomStruct HOT 2
- Option to encode nil slice/map as []/{}
- The handling of nil byte slices is different than the handling of other types of nil slices HOT 2
- blocking parallel execution? HOT 8
- Allow pointers when using UseTag HOT 3
- [cbor] Unable to use numbers as map keys HOT 5
- Using a transform function to/from an interface cause a nil pointer dereference
- Can't round trip nil slice HOT 1
- Encoding zero values as null HOT 5
- Fails to decode a valid CBOR object HOT 2
- Remove random binaries from git repo HOT 1
- Support for arbitrary structs without explicitly registering them HOT 2
- Possible to create my own raw marshal/unmarshal? HOT 6
- Support skipping unknown fields
- No go.mod, so modern idioms for building and testing fail?
- .gitmodules has extra trailing slashes that break "git submodule update --init"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from refmt.