Git Product home page Git Product logo

gofuzz's Introduction

gofuzz

gofuzz is a library for populating go objects with random values.

GoDoc

This is useful for testing:

  • Do your project's objects really serialize/unserialize correctly in all cases?
  • Is there an incorrectly formatted object that will cause your project to panic?

Import with import "github.com/google/gofuzz"

You can use it on single variables:

f := fuzz.New()
var myInt int
f.Fuzz(&myInt) // myInt gets a random value.

You can use it on maps:

f := fuzz.New().NilChance(0).NumElements(1, 1)
var myMap map[ComplexKeyType]string
f.Fuzz(&myMap) // myMap will have exactly one element.

Customize the chance of getting a nil pointer:

f := fuzz.New().NilChance(.5)
var fancyStruct struct {
  A, B, C, D *string
}
f.Fuzz(&fancyStruct) // About half the pointers should be set.

You can even customize the randomization completely if needed:

type MyEnum string
const (
        A MyEnum = "A"
        B MyEnum = "B"
)
type MyInfo struct {
        Type MyEnum
        AInfo *string
        BInfo *string
}

f := fuzz.New().NilChance(0).Funcs(
        func(e *MyInfo, c fuzz.Continue) {
                switch c.Intn(2) {
                case 0:
                        e.Type = A
                        c.Fuzz(&e.AInfo)
                case 1:
                        e.Type = B
                        c.Fuzz(&e.BInfo)
                }
        },
)

var myObject MyInfo
f.Fuzz(&myObject) // Type will correspond to whether A or B info is set.

See more examples in example_test.go.

dvyukov/go-fuzz integration

You can use this library for easier go-fuzzing. go-fuzz provides the user a byte-slice, which should be converted to different inputs for the tested function. This library can help convert the byte slice. Consider for example a fuzz test for a the function mypackage.MyFunc that takes an int arguments:

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func Fuzz(data []byte) int {
        var i int
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(i)
        return 0
}

Happy testing!

gofuzz's People

Contributors

aleksi avatar alvaroaleman avatar ashikpaul avatar banks avatar bbigras avatar colinnewell avatar danwinship avatar disconnect3d avatar dnephin avatar eclipseo avatar ericcornelissen avatar jake-ciolek avatar kwongtailau avatar lavalamp avatar pmorie avatar posener avatar punkeel avatar samirkut avatar strrl avatar thockin avatar yanzhoupan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gofuzz's Issues

Idea: Fuzzing with only non-zero values?

Specifically in my usecase I want to make sure all the struct is fuzzed always with non-zero values.

Currently I have no way to check this except checking the fuzz output every time after it runs - seems like it could be put using reflect in the generator and would be somewhat similar to the Nil option?

Is / should gofuzz safe for concurrent use?

I've had several data races with stack traces like this:

  math/rand.(*rngSource).Uint64()
      /usr/local/go/src/math/rand/rng.go:249 +0x190
  math/rand.(*rngSource).Int63()
      /usr/local/go/src/math/rand/rng.go:234 +0x1f1
  math/rand.(*Rand).Int63()
      /usr/local/go/src/math/rand/rand.go:84 +0x70
  math/rand.(*Rand).Int31()
      /usr/local/go/src/math/rand/rand.go:98 +0x81
  math/rand.(*Rand).Int31n()
      /usr/local/go/src/math/rand/rand.go:133 +0x45
  math/rand.(*Rand).Intn()
      /usr/local/go/src/math/rand/rand.go:171 +0x48
  github.com/google/gofuzz.UnicodeRanges.randString()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:586 +0x48
  github.com/google/gofuzz.randString()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:598 +0x6f
  github.com/google/gofuzz.glob..func6()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:491 +0x70
  github.com/google/gofuzz.(*fuzzerContext).doFuzz()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:271 +0x76e
  github.com/google/gofuzz.(*fuzzerContext).doFuzz()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:302 +0xa29
  github.com/google/gofuzz.(*fuzzerContext).doFuzz()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:327 +0x946
  github.com/google/gofuzz.(*Fuzzer).fuzzWithContext()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:239 +0x224
  github.com/google/gofuzz.(*Fuzzer).Fuzz()
      /home/guillem/golang/pkg/mod/github.com/google/[email protected]/fuzz.go:213 +0x18e

so, I guess it is not. Should it be?

Add note to docs that private fields of structs cannot be fuzzed using this package

I tried to use this package (through go-fuzz) for the first time when I was fuzzing a struct that has only private fields. It took me a while to figure out that this package doesn't put any values into private fields...

Now, I'm not terribly familiar with reflection in Go but from what I can tell based on the reflection docs and some testing is that it would be impossible for this package to set the value of private fields. Am i correct?

If the above statement is correct, I think it would be useful if there was a note in the README (or somewhere else in the docs) stating that private fields cannot be populated by this package. Thoughts?

Tag a new release: 1.3.0

Would it be possible to tag a new version. The latest version is from 2020 and misses some nice things like better "Can't handle" messages and generating unexpected fields.

And thank you for the great library!

go-fuzz integration may give nil values despite NilChance(0)

I have not fully investigated this problem yet, but I figured I can share it already at this stage. I found that while using gofuzz (this project) with the new go-fuzz integration I can get nil values even if I use NilChance(0).

For a working example I will use the go-fuzz integration in a normal test. Following (roughly) one of the examples found in the README, I create a test where I fuzz fancyStruct using data that was generated for me by go-fuzz while setting NilChance to 0. Then I check if any of the struct fields is nil. You can see the code blow ⬇️ Now, if you run this test you will (should?1) find that it fails and outputs "not good A".

Based on my experience so far this happens when the byte string is null-terminated (i.e. it ends with \x00). I don't know if it can happen 1) if \x00 appears elsewhere in the string (but in the example below it doesn't work with e.g. []byte("H000000\x000")), 2) if there are any null-terminated strings for which it doesn't happen, or 3) if other "special" characters can cause it. I also haven't look at the source code of this project to see what might be causing the problem.

package testing_something

import (
	"testing"

	fuzz "github.com/google/gofuzz"
)

func TestGoFuzz(t *testing.T) {
	data := []byte("H0000000\x00")
	f := fuzz.NewFromGoFuzz(data).NilChance(0) // Instead of f := fuzz.New().NilChance(.5)
	var fancyStruct struct {
		A, B, C, D *string
	}
	f.Fuzz(&fancyStruct) // None of the pointers should be nil, as NilChance is 0

	if fancyStruct.A == nil {
		t.Error("not good A")
	}
	if fancyStruct.B == nil {
		t.Error("not good B")
	}
	if fancyStruct.C == nil {
		t.Error("not good C")
	}
	if fancyStruct.D == nil {
		t.Error("not good D")
	}
}

  1. I'm guessing this may possibly be different per OS or golang version. In case that is true, I was running this in a golang docker container for v1.14 (so docker run [ARGUMENTS] golang:1.14) and installed just go-fuzz using go get -u github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build

Test failure on 32 bits arch

Fedora Rawhide with Go 1.15.x, on i686 and armv7hl

Test fails with:

Testing    in: /builddir/build/BUILD/gofuzz-1.2.0/_build/src
         PATH: /builddir/build/BUILD/gofuzz-1.2.0/_build/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
       GOPATH: /builddir/build/BUILD/gofuzz-1.2.0/_build:/usr/share/gocode
  GO111MODULE: off
      command: go test -buildmode pie -compiler gc -ldflags " -X github.com/google/gofuzz/version=1.2.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld  '"
      testing: github.com/google/gofuzz
github.com/google/gofuzz
# github.com/google/gofuzz [github.com/google/gofuzz.test]
./fuzz_test.go:641:10: constant 5563767293437588600 overflows int
FAIL	github.com/google/gofuzz [build failed]

Generating blank strings even though nilChance is set to 0

I am trying to fuzz a string field like below, but it is returning empty string ("") even though I set the nilChance to zero. The field I am fuzzing does not support empty strings so I would like to exclude it from my fuzzing input. Is there a way to do this?

for i := 0; i < 100; i++ {
  var s string
  fuzz.New().NilChance(0).Fuzz(&s)
  _, err := MyMethod(s) // this function will return an error if s is an empty string
  if err != nil {
    // handle error
  }
}

Improve time.Time default fuzzer

A default fuzzer for time.Time is now in. @thockin changed it to generate times only in the next 1000 years or so; json marshalling has difficulties with negative times and times with more than four digit years.

Should the set of time values we produce include invalid times, or no? There are use cases where people could want either behavior. Users can override this default, but we should document what we decide.

New Release?

Hey, I'm trying to use gofuzz with go-fuzz and it is very difficult, because there is no release yet that has the fuzz.NewFromGoFuzz feature. I think it may be time to create a new release...

NewFromGoFuzz doesn't work well with go-fuzz

I've come across projects using go-fuzz with NewFromGoFuzz where fuzzing didn't catch obvious bugs. I've distilled these down into very simple, unstructured cases (which you'd probably not use gofuzz for) which demonstrate the problems. In the projects in questions, I've used JSON and protobuf unmarshalling of go-fuzz provided data into structs with better results. Not sure if there's anything obvious to do with this information except to document the downsides?

It's probably easiest to describe the problem with examples:

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func MyFunc(i int) {
    if i == 1337 {
        panic("found correct i")
    }
}

func Fuzz(data []byte) int {
    var i int
    fuzz.NewFromGoFuzz(data).Fuzz(&i)
    MyFunc(i)
    return 0
}

The above (an extension of the README example) would have to run for a very long time to find a crasher. Go-fuzz has its sonar to guide it to new coverage gated by comparisons e.g. the following finds crashers almost immediately:

// +build gofuzz

package mypackage

import "encoding/binary"

func MyFunc(i int) {
    if i == 1337 {
        panic("found correct i")
    }
}

func Fuzz(data []byte) int {
    i := binary.BigEndian.Uint64(data)
    MyFunc(int(i))
    return 0
}

Decoding the data from go-fuzz as little-endian, hex encoded strings or ascii will also result in the crasher being found quickly as the number "1337" is used in the mutator in all of these forms.

I found that applying this patch

diff --git a/bytesource/bytesource.go b/bytesource/bytesource.go
index 5bb3659..efd6a65 100644
--- a/bytesource/bytesource.go
+++ b/bytesource/bytesource.go
@@ -40,7 +40,9 @@ func New(input []byte) *ByteSource {
                fallback: rand.NewSource(0),
        }
        if len(input) > 0 {
-               s.fallback = rand.NewSource(int64(s.consumeUint64()))
+               seed := make([]byte, 8)
+               copy(seed, input)
+               s.fallback = rand.NewSource(int64(binary.BigEndian.Uint64(seed)))
        }
        return s
 }

allowed go-fuzz to work a little bit better for this scenario. The improvement comes because the first 8 bytes of the go-fuzz input are not lost to the random number generator seed.

The following demonstrates how inputs from go-fuzz with string literals are unrecognisable after passing through NewFromGoFuzz.Fuzz:

package main

import (
    "fmt"

    fuzz "github.com/google/gofuzz"
)

func main() {
    data := []byte("-----BEGIN RSA PRIVATE KEY-----")
    var str string
    fuzz.NewFromGoFuzz(data).Fuzz(&str)
    fmt.Printf("str = %s\n", str)
}

outputs

str = Ȱ#ǻŐ蒉

Here, randString will consume at least 8*(n+1) bytes of the go-fuzz input (n is the length chosen for the string) in such a way as to "slice and dice" the input leaving it with no recognisable fragments. This prevents the fuzzer finding crashers for the following:

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func MyFunc(str string) {
    if str == "1337" {
        panic("found correct")
    }
}

func Fuzz(data []byte) int {
    var str string
    fuzz.NewFromGoFuzz(data).Fuzz(&i)
    MyFunc(str)
    return 0
}

An equivalent fuzzer without gofuzz

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func MyFunc(str string) {
    if str == "1337" {
        panic("found correct string")
    }
}

func Fuzz(data []byte) int {
    MyFunc([]byte(str))
    return 0
}

immediately finds a crasher. It should also be noted that strings from gofuzz will be limited in length and well-formed utf-8 which may be at odds with the aims for fuzzing.

issue building with go 1.11.3 and previous

9532c33 is causing versions prior to go 1.11.4 to have issues requiring the module.

e.g. semaphore v1 has go 1.11.0 as latest version and builds fail with:

go build github.com/google/gofuzz: module requires Go 1.12

note: travis-ci hasn't tested the last 2 commits on master.

Improve quality of integer output

This code:

func TestFoo(t *testing.T) {
    var i int
    fuzzy := fuzz.New()
    forN(100, func() {
        fuzzy.Fuzz(&i)
        log.Println(i)
    })
}

produces:

2015/09/04 12:35:09 -7835246273949666689
2015/09/04 12:35:09 -3335834746906823390
2015/09/04 12:35:09 -797614582871569465
2015/09/04 12:35:09 -1880490989454751177
2015/09/04 12:35:09 8642978877542446388
2015/09/04 12:35:09 7210419396307716596
2015/09/04 12:35:09 -1013781862395183072
2015/09/04 12:35:09 -6870665591055426828
2015/09/04 12:35:09 7613138460266199146
2015/09/04 12:35:09 -5153746402492891673
2015/09/04 12:35:09 810562363728706214
2015/09/04 12:35:09 -6642518234775288016
2015/09/04 12:35:09 -2591875516334598574
2015/09/04 12:35:09 -8510913015654777828
2015/09/04 12:35:09 4315878390670269427
2015/09/04 12:35:09 4755147220257554544
2015/09/04 12:35:09 -595777994041846809
2015/09/04 12:35:09 -3921103541046646506
2015/09/04 12:35:09 -2687620294811223767
2015/09/04 12:35:09 59623211572459200
2015/09/04 12:35:09 580834844145044863
2015/09/04 12:35:09 -7587201369927064342
2015/09/04 12:35:09 -8485154379888275139
2015/09/04 12:35:09 4417374548573622764
2015/09/04 12:35:09 -720185865975215166

Looks like ints never come out with an absolute value lower than one septillion, which is undesirable. I'd like to see an option to guarantee at-least one instance of zero as well.

presence of an interface value always panics

Hello there, thank you for this project!

I am currently looking at using it to fuzz inputs to https://github.com/census-instrumentation/opencensus-service

However, if a struct has a field with a value whose type is an interface, regardless of if that field is set or not. For example

type A struct {
            Name string
            Debug bool
            Err error
}
a := new(A)
f.Fuzz(a)

gofuzz will always crash with

On no value set

panic: Can't handle <nil>: interface [recovered]
	panic: Can't handle <nil>: interface

On value set

panic: Can't handle &errors.errorString{s:"Foo"}: interface [recovered]
	panic: Can't handle &errors.errorString{s:"Foo"}: interface

Perhaps we could skip trying to mutate interface values that we don't know about and let tests proceed normally, otherwise it becomes unproductive to debug this cryptic error.

Thank you.

Improve quality of string output

So using the following inside a loop

kind := fuzzString()
log.Println(kind)

outputs the following to the console

2015/09/03 19:04:44 ª漠Ȯʜ摽OĹ/JǼ乇C
2015/09/03 19:04:44 ǟJ翾
2015/09/03 19:04:44 J?ǻ
2015/09/03 19:04:44 (v獔s赭İĹ
2015/09/03 19:04:44 鄗彸湦軈2Ħ攩V螱
2015/09/03 19:04:44 婦_僤Ⱥ^ȡ莯yŀ亶h鍡
2015/09/03 19:04:44 鼈ǃ綺z盚ɴ憤答ǘ騮r冔J<ȈWp靌_
2015/09/03 19:04:44 禽ÊÉ3鏡碞`ɭ筹2溏#Q:M
2015/09/03 19:04:44 mĿį窌橛鑼ȠQD貧Ũroz踮穰"
2015/09/03 19:04:44 o增aʇŊL腙ɣS[ŠŪæ羑/甴q漥嚬
2015/09/03 19:04:44 úưn鉋囔敘匿
2015/09/03 19:04:44 彧森ČÅ
2015/09/03 19:04:44 ǿƲ鯱ı&ʧŞ嵔6Ÿ
2015/09/03 19:04:44 ¾U儾蓝ɽÙŝ鏁
2015/09/03 19:04:44 "绖ʚ跋拘瀤Ȍ}FWȗǻkU
2015/09/03 19:04:44 |Y&
2015/09/03 19:04:44 
2015/09/03 19:04:44 2躡溹
2015/09/03 19:04:44 
2015/09/03 19:04:44 Ğŋ歺饽i1%ʇƺF池睺嶙ĉ
2015/09/03 19:04:44 馬ƫ椹vHıÕ莍垺噽q具
2015/09/03 19:04:44 6
2015/09/03 19:04:44 ƓH(ĴƆ4t4嬚宧1
2015/09/03 19:04:44 G-ª,
2015/09/03 19:04:44 Ʀ焬陱ʃȌ飠e俿ɟ躁彞Ȱd鉛 5聳炬血
2015/09/03 19:04:44 鏶牏ª.濔挹U忘Eʤe黍
2015/09/03 19:04:44 ǩ歹ȊǾz肌憌.ʪ谌O蟆WƤ
2015/09/03 19:04:44 嵤贂Ƶm1赉
2015/09/03 19:04:44 涉
2015/09/03 19:04:44 岲憏蓹
2015/09/03 19:04:44 ʧȜP珐芫汋鰢NJ詺^涍ȷ饤劮
2015/09/03 19:04:44 秤柃SŘɎh巽貗g扆¢W醦4ɖ

This does not appear to be a good random distribution of characters. Also empty strings are outputted more than once, so duplicates are not being addressed. Ideally this library has a consistent random distribution of characters, no duplicates, and interleaved white space (also absent).

Regarding newer versions

Hi team.

First of all, very important library for us to use in testing, so thanks for creating this!

Just wanted to ask, if there are any plans to release a new version? It has been 3 years since a release. And if yes, then any timelines?

In particular, this issue has been bugging us for a while, so would love a new release.

Thank you!

Integrate with fuzzit?

Hi

Is it possible to enable usage of fuzzit with gofuzz?
I think we can use the random input in order to generate the random objects, so it will make fuzzing easier. Maybe it is possible to use the random bytes as a seed for a random source?

Does it all make sense?

Thanks!

go mod checksum mismatch

verifying github.com/google/[email protected]/go.mod: checksum mismatch
	downloaded: h1:HP5RmnzzSNb993RKQDq4+1A4ia9nllfqcQFTQJedwGI=
	sum.golang.org: h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.