There doesn't seem to be a way to get CFLite to upload files crashing fuzz targets if they crash the latest build about clusterfuzzlite HOT 16 CLOSED

How about a switch to shut off crash novelty checking instead?

from clusterfuzzlite.

It would make it harder to tell really new bugs and bugs that just happen to collide apart, which would probably mean that I wouldn't even try to turn it off for projects that have just started fixing bugs found by fuzz targets.

from clusterfuzzlite.

Sorry, I think I missed something in your original request.
Do you want uploading of fuzz targets and crashes that are not novel?

from clusterfuzzlite.

Should we reopen this and close google/oss-fuzz#6984 instead?

from clusterfuzzlite.

Should we reopen this and close google/oss-fuzz#6984 instead?

I thought I was doing exactly that :-) Looks like I lost track of all the issues I've opened :-)

Another feature that I think would be useful if CFLite could show backtraces of the last builds as well. I've just discovered an issue that I haven't seen before but

2021-12-08 23:31:17,581 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2021-12-08 23:31:17,582 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:18,452 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2021-12-08 23:31:18,452 - root - INFO - Crash is reproducible.
2021-12-08 23:31:18,462 - root - INFO - Trying to downloading previous build 51106acb0d7f01845ebab85a3ce4e8b46ae95ab5.
2021-12-08 23:31:20,151 - root - INFO - Done downloading previous build.
2021-12-08 23:31:20,152 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:20,982 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/cifuzz-prev-build/fuzz-dwfl-core

so I have a patch and a new backtrace and to figure out what is going there it seems I have to build and run the fuzz target locally to get the file triggering it, then I have to revert that patch and build the fuzz target once again to pass the testcase to it to get the backtrace. It wouldn't be necessary if CFLite uploaded files crashing fuzz targets unconditionally and showed "old" backtraces.

from clusterfuzzlite.

Should we reopen this and close google/oss-fuzz#6984 instead?

I thought I was doing exactly that :-) Looks like I lost track of all the issues I've opened :-)

Another feature that I think would be useful if CFLite could show backtraces of the last builds as well. I've just discovered an issue that I haven't seen before but

2021-12-08 23:31:17,581 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2021-12-08 23:31:17,582 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:18,452 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2021-12-08 23:31:18,452 - root - INFO - Crash is reproducible.
2021-12-08 23:31:18,462 - root - INFO - Trying to downloading previous build 51106acb0d7f01845ebab85a3ce4e8b46ae95ab5.
2021-12-08 23:31:20,151 - root - INFO - Done downloading previous build.
2021-12-08 23:31:20,152 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:20,982 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/cifuzz-prev-build/fuzz-dwfl-core

so I have a patch and a new backtrace and to figure out what is going there it seems I have to build and run the fuzz target locally to get the file triggering it, then I have to revert that patch and build the fuzz target once again to pass the testcase to it to get the backtrace. It wouldn't be necessary if CFLite uploaded files crashing fuzz targets unconditionally and showed "old" backtraces.

Sorry, I'm having trouble understanding this. We already show the stacktrace of the crash on the newer binary, why do we need to show the crash from the older binary?

from clusterfuzzlite.

why do we need to show the crash from the older binary?

Because the backtrace can be different if PRs introduce new bugs on top of old bugs.

from clusterfuzzlite.

@jonathanmetzman judging by https://github.com/evverx/elfutils/actions/runs/1574802879, it seems to be working. Thanks!

from clusterfuzzlite.

I'm not sure why CFLite is red there though. Those issues are reproducible with the latest build as well so I think CFLite should be green.

from clusterfuzzlite.

I'm not sure if it helps but here's what I use in the CFLite workflow:

    env:
      UPLOAD_ALL_CRASHES: 'True'
      REPORT_OOMS: 'False'

My understanding it that UPLOAD_ALL_CRASHES=True combined with REPORT_OOMS=False should prevent PRs from turning red when those OOMs are reproducible with the latest builds.

from clusterfuzzlite.

I moved that to evverx/elfutils#53 to avoid interfering with evverx/elfutils#49 (which I'm using to test https://sourceware.org/pipermail/elfutils-devel/2021q4/004520.html)

from clusterfuzzlite.

Without UPLOAD_ALL_CRASHES: 'True' it's green there as expected

from clusterfuzzlite.

Turned it off for now because without google/oss-fuzz#6997 the fuzzers can't go any further

from clusterfuzzlite.

Landed google/oss-fuzz#6997

from clusterfuzzlite.

Looks like those OOMs are gone. Thanks! The fuzz target triggers a timeout though but once https://sourceware.org/pipermail/elfutils-devel/2021q4/004463.html is merged it should disappear as well.

from clusterfuzzlite.

With that patch backported the fuzz target can run for 10 minutes on CFLite so it's probably safe to say that it's prepared for the next round of patches. It would be great if PRs crashing the latest builds didn't turn red with UPLOAD_ALL_CRASHES=True and it was possible to show old backtraces though.

from clusterfuzzlite.