google / clusterfuzzlite Goto Github PK

View Code? Open in Web Editor NEW

437.0 18.0 41.0 215 KB

ClusterFuzzLite - Simple continuous fuzzing that runs in CI.

Home Page: https://google.github.io/clusterfuzzlite

License: Apache License 2.0

security fuzzing fuzz-testing vulnerabilities ci continuous-integration

clusterfuzzlite's Issues

Support Azure pipelines

Envoy has expressed interest in using ClusterFuzzLite with Azure pipelines.

There doesn't seem to be a way to exclude corpora from artifacts

From google/oss-fuzz#7186 (comment)

I think CFLite should pass UPLOAD_BUILD to build_fuzzers to make it possible to exclude those large corpora from artifacts. Without that environment variable I can't skip that step with something like

if [[ "$MERGE_WITH_OSS_FUZZ_CORPORA" == "yes" ]]; then

    # When the latest builds are uploaded by CFLite the large OSS-Fuzz corpora
    # should be excluded regardless of whether MERGE_WITH_OSS_FUZZ_CORPORA
    # is set to "yes" or not.
    [[ "$UPLOAD_BUILD" == "true" ]] && exit 0
    ....

CFlite shouldn't set rss_limit_mb when REPORT_OOMS is set to false

I tried to ignore OOMs by setting REPORT_OOMS to false but fuzz targets just kept failing as soon as they started with

==23== ERROR: libFuzzer: out-of-memory (malloc(4294971391))
   To change the out-of-memory limit use -rss_limit_mb=<N>
...

It would be great if REPORT_OOMS turned off rss_limit_mb and let allocator_may_return_null (which is set by default as far as I can tell) take care of those allocations.

Failing to build python project

This CFL setup is breaking in the build process on this line.

The build works fine on OSS-Fuzz but fails to complete in the CI.

Stacktrace from the log:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/atheris/__init__.py", line 17, in <module>
    from typing import List
  File "/src/pydantic/typing.py", line 4, in <module>
    from typing import (  # type: ignore
ImportError: cannot import name 'TYPE_CHECKING' from partially initialized module 'typing' (most likely due to a circular import) (/src/pydantic/typing.py)```

CFLite and OSV

It's more of a question than a feature request. I'm considering moving some projects from OSS-Fuzz to CFLite and I wonder whether it's safe to assume that CFLite won't be integrated with https://osv.dev/ by sharing its results with the database or anything like that?

There doesn't seem to be a way to point CFLite to public OSS-Fuzz corpora

I integrated CFLite into my fork of the elfutils project and it seems to be working in the sense that the "code-change" fuzzing covers PRs in combination with the continuous builds triggered on every push and preventing CFLite from reporting issues that aren't introduced in PRs. It would be great if I could also point CFLite to public OSS-Fuzz corpora (that CIFuzz downloads on every PR) instead of adding "batch fuzzing" and "corpus pruning". It would be even better if I could point "coverage reports" to those public corpora as well.

I understand that this kind of hybrid CFLite/OSS-Fuzz mode is probably too specific but I can imagine projects integrated into OSS-Fuzz using CI services (like buildbot, for example) that aren't supported and where they would probably have to reimplement every type of the CFLite modes manually and if two of them could be replaced with just downloading public OSS-Fuzz corpora it would probably help.

There doesn't seem to be a way to use the latest docker images (without the "v1" tag)

As mentioned in google/oss-fuzz#7206 (comment) I'm planning to unpin CFLite but looking at the action it appears for some reason it uses tags to download the docker images:

$ git grep v1 actions/
actions/build_fuzzers/action.yml:  image: 'docker://gcr.io/oss-fuzz-base/clusterfuzzlite-build-fuzzers:v1'
actions/run_fuzzers/action.yml:  image: 'docker://gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1'

Those tags are bogus in the sense that they keep rolling forward so I wonder if it's possible to remove them to make it clear that they always point to the latest images.

The idea is to always use the "main" branch and the latest images (by analogy with CIFuzz) and avoid getting bogus Dependabot updates when/if google/oss-fuzz#7212 is implemented

Allow to customize FUZZ_SECONDS option per target

First thank you so much for clusterfuzzlite. I have a great time using it in Oxigraph.

Oxigraph provides multiple targets, some fuzzing simple parsers and one fuzzing the query optimizer. The query optimizer target is way slower than the parsing ones, requirering much more time to yield interesting results or even to go through the already existing corpus. I would love a way to set different time limit per target or have an option in the google/clusterfuzzlite/actions/run_fuzzers GitHub to only run a specific list of targets instead of all of them.

Minimal gitlab configuration is failing to cp to default $OUT

Probably a duplicate of #100.

I'm stuck integrating with gitlab. Here is a snippet showing the failure mode where a copy to $OUT is failing.

Successfully built 1e6e6b3e9a9c
Successfully tagged external-cfl-project-664774e26cc14adcb17b756aa191a6ca:latest
2023-04-26 18:30:07,330 - root - INFO - repo_dir: /builds/software/<PROJECT>.
2023-04-26 18:30:07,341 - root - INFO - Docker container: .
2023-04-26 18:30:07,341 - root - INFO - Building with address sanitizer.
2023-04-26 18:30:07,341 - root - INFO - Running: docker run --rm --privileged --shm-size=2g --platform linux/amd64 -e FUZZING_ENGINE=libfuzzer -e CIFUZZ=True -e SANITIZER=address -e ARCHITECTURE=x86_64 -e FUZZING_LANGUAGE=c++ -e OUT=/builds/33190/build-out -v /builds/33190:/builds/33190 -v /builds/software/<PROJECT>:/builds/software/<PROJECT> external-cfl-project-664774e26cc14adcb17b756aa191a6ca /bin/bash -c compile.
---------------------------------------------------------------
Compiling libFuzzer to /usr/lib/libFuzzingEngine.a...  done.
cp: cannot create regular file '/builds/33190/build-out/': Not a directory

Following the documentation, I have confirmed that the fuzzers can be built and run on my local machine. I've also tried an explicit mkdir -p $OUT line in the Dockerfile or the build.sh with no luck.

My configuration file is pretty minimal.

variables:
  SANITIZER: address
  CFL_PLATFORM: gitlab
  # DOCKER_HOST: "tcp://docker:2375" # may be removed in self-managed Gitlab instances
  DOCKER_IN_DOCKER: "true" # may be removed in self-managed Gitlab instances
  GIT_SUBMODULE_STRATEGY: recursive
  GIT_SUBMODULE_FORCE_HTTPS: "true"
  MODE: "code-change"

clusterfuzzlite:
  image:
    name: gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers:v1
    entrypoint: [""]
  services:
    - docker:dind # may be removed in self-managed Gitlab instances
    
  stage: test
  parallel:
    matrix:
      - SANITIZER: [address, undefined]
  #rules:
  #  # Default code change.
  #  - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  #    variables:
  #      MODE: "code-change"
  before_script:
    # Get gitlab's container id.
    - export CFL_CONTAINER_ID=`cut -c9- < /proc/1/cpuset`
  script:
    # Will build and run the fuzzers.
    - python3 "/opt/oss-fuzz/infra/cifuzz/cifuzz_combined_entrypoint.py"
  artifacts:
    # Upload artifacts when a crash makes the job fail.
    when: always
    paths:
      - artifacts/

What should I try to keep troubleshooting this?

Support local deployments

This should be straightforward and will help with testing.
Just implement a filesystem based filestore and document it.

There doesn't seem to be an easy way to import existing corpora

I'm not sure it can be fixed easily in the sense that it seems to be hard to upload artifacts using curl or something like that but having spent about an hour trying to import a corpus I think the easiest way is to open a PR where the corpus is put in $OUT/fuzz-target_seed_corpus.zip and temporarily unblock cflite_batch to run it on PRs. This way, the corpus is uploaded and can be used by PRs before cflite_batch itself is merged.

Document who should use ClusterFuzzLite and compare to other options

This was suggested by @WorksButNotTested

There doesn't seem to be a way to pass --architecture=i386 to CFLite

Some elfutils bugs manifest themselves only on 32-bit platforms like i386 or armhfp. They usually can be reproduced by simply compiling elfutils along with its fuzz targets with -m32 on 64-bit platforms like x86_64 but some bugs are spread across all architectures and to verify patches the code should be compiled with and without -m32. It's easy to miss something in the process (for example, in https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 I didn't wait for the tests on 32 bit machines to finish and said the patches were OK). Luckily it was caught by that bespoke regression testsuite but I think testsuites like that shouldn't be neccessary when CFLite is used. It would be great if by analogy with sanitizer: ... it was possible to additionally pass something like architecture: i386 to CFLite.

assumptions about CFLite runtime environment

According to https://google.github.io/clusterfuzzlite/build-integration/#fuzzer-execution-environment

You should not make any assumptions on the availability of dependent packages in the execution environment and the built fuzzers should have dependencies statically linked.

I understand why fuzz targets integrated into OSS-Fuzz should be written that way but I'm not sure why CFLite should follow suit. If this restriction was lifted I think I could in theory use CFLite (in combination with OSS-Fuzz and CIFuzz) to build systemd fuzz targets linked dynamically against its dependencies and run them on GitHub. Those two modes are already kind of supported by the systemd build system but only one of them is used regularly on OSS-Fuzz.

Make sure build uploading is done for each sanitizer in use.

Requesting addition of AFL++ Fuzzer Engine.

Hello Clusterfuzz Team,

Good Evening.

I would like to kindly request for an addition of the AFL++ Fuzzer Engine to the Clusterfuzz/ClusterfuzzLite project.

Awaiting your response.

Thanks & Regards,
~ Kushal Arvind Shah.

Figure out how to determine who is using ClusterFuzzLite

We could go with the same approach as the cifuzz dashboard, but 1. It is brittle 2. It won't work for private repos. 3. Or non github users.
Since we aren't going to add any tracking, I think the best option is to have a link in the docs to a google form where users can let us know they are using cflite if they desire.
This will help us drive development, since working on a project like this can be better justified if it is having a big impact.

Possible to continue fuzzing despite errors?

Hi,

Is there a setting to continue fuzzing and ignore crashes? Possibly deduplicating the crashes?

Currently the fuzzer stops at the first found crash, but this is not very helpful if there are multiple possible crashes.

Gitlab instructions do not just work.

I had to make this config file which borrowed from https://gitlab.com/securitykernel/cflite-example/-/blob/main/.gitlab-ci.yml#L7 because the docs were incomplete.
CC @catenacyber

Timeout in pruning job

Getting logs like

2022-02-15 09:19:54,828 - root - INFO - Done downloading corpus. Contains 5964 elements.
Traceback (most recent call last):
  File "/opt/oss-fuzz/infra/cifuzz/cifuzz_combined_entrypoint.py", line 40, in <module>
    sys.exit(main())
  File "/opt/oss-fuzz/infra/cifuzz/cifuzz_combined_entrypoint.py", line 36, in main
    return run_fuzzers_entrypoint.run_fuzzers_entrypoint()
  File "/opt/oss-fuzz/infra/cifuzz/run_fuzzers_entrypoint.py", line 61, in run_fuzzers_entrypoint
    result = run_fuzzers.run_fuzzers(config)
  File "/opt/oss-fuzz/infra/cifuzz/run_fuzzers.py", line 302, in run_fuzzers
    if not fuzz_target_runner.run_fuzz_targets():
  File "/opt/oss-fuzz/infra/cifuzz/run_fuzzers.py", line 130, in run_fuzz_targets
    result = self.run_fuzz_target(target)
  File "/opt/oss-fuzz/infra/cifuzz/run_fuzzers.py", line 161, in run_fuzz_target
    result = fuzz_target_obj.prune()
  File "/opt/oss-fuzz/infra/cifuzz/fuzz_target.py", line 140, in prune
    result = engine_impl.minimize_corpus(self.target_path, [],
  File "/usr/local/lib/python3.8/dist-packages/clusterfuzz/_internal/bot/fuzzers/libFuzzer/engine.py", line 507, in minimize_corpus
    raise TimeoutError('Merging new testcases timed out\n' + result.output)
TimeoutError: Merging new testcases timed out
INFO: Running with entropic power schedule (0xFF, 100).

Increasing FUZZ_SECONDS does not seem to be enough to fix this...
But I see fuzz_seconds // fuzzers_left_to_run so maybe this fuzzer is a bit long but it should have more time than the others...

Support gitlab CI

Some people have expressed interest in using this with gitlab.
We should support it.

There doesn't seem to be a way to pass "retention-days" to CFLite jobs uploading artifacts

It would be great if by analogy with https://github.com/actions/upload-artifact#retention-period it was possible to pass something like retention-days: 3 to CFLite jobs uploading artifacts. It would be even better if CFLite could automatically delete artifacts that should no longer be kept (like the "latest" builds generated a week ago) but I'm not sure how exactly it should work.

CFLite seems to fail to download coverage data kept as an artifact

I set up a "cron" job pruning corpora and generating coverage reports but PRs don't seem to have access to "coverage" artifacts:

2022-01-21 13:08:25,388 - root - INFO - Removing unaffected fuzz targets.
2022-01-21 13:08:25,601 - root - INFO - Diffing against master.
2022-01-21 13:08:25,604 - root - INFO - Files changed in PR: ['.clusterfuzzlite/bpf-object-fuzzer.c']
2022-01-21 13:08:25,755 - root - ERROR - Request to https://api.github.com/repos/evverx/libbpf/actions/artifacts?per_page=100&page=1 failed. Code: 401. Response: {'message': 'Bad credentials', 'documentation_url': 'https://docs.github.com/rest'}
2022-01-21 13:08:25,756 - root - ERROR - Could not get coverage: Github API request failed..
2022-01-21 13:08:25,756 - root - ERROR - Could not find latest coverage report.

even though they can download the corpus just fine:

2022-01-21 13:08:32,241 - root - INFO - Using address sanitizer.
2022-01-21 13:08:32,253 - root - INFO - Fuzz targets: ['/github/workspace/build-out/bpf-object-fuzzer']
2022-01-21 13:08:32,254 - root - INFO - Running fuzzer: bpf-object-fuzzer.
2022-01-21 13:08:32,254 - root - INFO - Downloading corpus for bpf-object-fuzzer to /github/workspace/cifuzz-corpus/bpf-object-fuzzer.
2022-01-21 13:08:33,281 - root - INFO - Done downloading corpus. Contains 1317 elements.
2022-01-21 13:08:33,281 - root - INFO - Starting fuzzing

The PR where it was tested is evverx/libbpf#4

The coverage data and corpus were generated at https://github.com/evverx/libbpf/actions/runs/1728607427

It could be that I screwed something up there. I'll try to figure out whether the issue is on my side.

Document reproducing crashes.

CFLite seems to fail to download the latest builds due to "Bad credentials"

I'm not sure why it stopped working but it appears CFLite can no longer download the latest builds. In https://github.com/evverx/elfutils/runs/5702214651?check_suite_focus=true it failed with

2022-03-26T08:44:16.2107609Z 2022-03-26 08:44:16,210 - root - INFO - Trying to reproduce crash using: /tmp/tmpzgptwvca/crash-08b22184029412ff41e6015f42fbe90cc6975aea.
2022-03-26T08:44:16.8321170Z 2022-03-26 08:44:16,831 - root - INFO - Reproduce command returned: 77. Reproducible on /github/workspace/build-out/fuzz-libdwfl.
2022-03-26T08:44:16.8328312Z 2022-03-26 08:44:16,832 - root - INFO - Crash is reproducible.
2022-03-26T08:44:16.8448813Z 2022-03-26 08:44:16,844 - root - INFO - Trying to downloading previous build 10b63c5588bf485cf3143b553fad3a9a7c6012f4.
2022-03-26T08:44:16.8952915Z 2022-03-26 08:44:16,893 - root - ERROR - Request to https://api.github.com/repos/evverx/elfutils/actions/artifacts?per_page=100&page=1 failed. Code: 401. Response: {'message': 'Bad credentials', 'documentation_url': 'https://docs.github.com/rest'}
2022-03-26T08:44:16.8954721Z 2022-03-26 08:44:16,893 - root - ERROR - Could not download build for 10b63c5588bf485cf3143b553fad3a9a7c6012f4 because of: Github API request failed.
2022-03-26T08:44:16.8956014Z 2022-03-26 08:44:16,894 - root - INFO - Trying to downloading previous build 392242e2dbe7beddfee283979a81f50a1bf9f18a.
2022-03-26T08:44:16.9439033Z 2022-03-26 08:44:16,942 - root - ERROR - Request to https://api.github.com/repos/evverx/elfutils/actions/artifacts?per_page=100&page=1 failed. Code: 401. Response: {'message': 'Bad credentials', 'documentation_url': 'https://docs.github.com/rest'}
2022-03-26T08:44:16.9440932Z 2022-03-26 08:44:16,942 - root - ERROR - Could not download build for 392242e2dbe7beddfee283979a81f50a1bf9f18a because of: Github API request failed.
2022-03-26T08:44:16.9442233Z 2022-03-26 08:44:16,942 - root - INFO - Trying to downloading previous build 9166adf346d693ea8f40251c601acd1b95a69d40.
2022-03-26T08:44:16.9958385Z 2022-03-26 08:44:16,994 - root - ERROR - Request to https://api.github.com/repos/evverx/elfutils/actions/artifacts?per_page=100&page=1 failed. Code: 401. Response: {'message': 'Bad credentials', 'documentation_url': 'https://docs.github.com/rest'}
2022-03-26T08:44:16.9960886Z 2022-03-26 08:44:16,994 - root - ERROR - Could not download build for 9166adf346d693ea8f40251c601acd1b95a69d40 because of: Github API request failed.
2022-03-26T08:44:16.9963078Z 2022-03-26 08:44:16,995 - root - INFO - Could not run previous build of target to determine if this code change (pr/commit) introduced crash. Assuming crash was newly introduced.

The latest builds can be found at https://github.com/evverx/elfutils/actions/runs/2041191178

CFLite seems to report issues that are reproducible with the latest build

In https://github.com/evverx/elfutils/runs/4719892523?check_suite_focus=true CFlite reported an issue that as far as I can tell should be reproducible with the latest build:

2022-01-05T20:37:26.5170500Z gelf_xlate.h:48:1: runtime error: member access within misaligned address 0x7f08cf690007 for type 'Elf32_Dyn', which requires 4 byte alignment
2022-01-05T20:37:26.5171380Z 0x7f08cf690007: note: pointer points here
2022-01-05T20:37:26.5171925Z  46 01 02 01 80  00 00 ff 01 00 00 54 ff  ff 45 48 48 48 48 04 de  7f 45 27 4e 00 00 00 32  2b 00 48
2022-01-05T20:37:26.5172266Z              ^ 
2022-01-05T20:37:26.5446405Z     #0 0x57fa1a in Elf32_cvt_Dyn /src/elfutils/libelf/./gelf_xlate.h:48:1
2022-01-05T20:37:26.5447286Z     #1 0x57f180 in elf32_xlatetom /src/elfutils/libelf/elf32_xlatetom.c:104:7
2022-01-05T20:37:26.5448229Z     #2 0x53ff9c in dwfl_segment_report_module /src/elfutils/libdwfl/dwfl_segment_report_module.c:848:20
2022-01-05T20:37:26.5455169Z     #3 0x4bad4d in dwfl_core_file_report /src/elfutils/libdwfl/core-file.c:563:17
2022-01-05T20:37:26.5456621Z     #4 0x4b3a22 in LLVMFuzzerTestOneInput /src/elfutils/tests/fuzz-dwfl-core.c:47:7
2022-01-05T20:37:26.5459860Z     #5 0x43d953 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) cxa_noexception.cpp
2022-01-05T20:37:26.5461352Z     #6 0x43d13a in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) cxa_noexception.cpp
2022-01-05T20:37:26.5462476Z     #7 0x43e6e9 in fuzzer::Fuzzer::MutateAndTestOne() cxa_noexception.cpp
2022-01-05T20:37:26.5463658Z     #8 0x43f3b5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) cxa_noexception.cpp
2022-01-05T20:37:26.5464944Z     #9 0x42eccf in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) cxa_noexception.cpp
2022-01-05T20:37:26.5466365Z     #10 0x457bf2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
2022-01-05T20:37:26.5467683Z     #11 0x7f08cf7150b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
2022-01-05T20:37:26.5468709Z     #12 0x407d4d in _start (build-out/fuzz-dwfl-core+0x407d4d)
2022-01-05T20:37:26.5469214Z 
2022-01-05T20:37:26.5470037Z DEDUP_TOKEN: Elf32_cvt_Dyn--elf32_xlatetom--dwfl_segment_report_module
2022-01-05T20:37:26.5471320Z SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior gelf_xlate.h:48:1 in 
2022-01-05T20:37:26.5472764Z MS: 1 CrossOver-; base unit: a17f7d5db59b0763f7cd80ba8193f447f8e454a6
2022-01-05T20:37:26.5474409Z artifact_prefix='/tmp/tmp11sdfotu/'; Test unit written to /tmp/tmp11sdfotu/crash-db46ee9027b1d3bc81d4a4d57752de3c4f841df8
2022-01-05T20:37:26.5475543Z stat::number_of_executed_units: 100496
2022-01-05T20:37:26.5476101Z stat::average_exec_per_sec:     3349
2022-01-05T20:37:26.5476600Z stat::new_units_added:          1627
2022-01-05T20:37:26.5477103Z stat::slowest_unit_time_sec:    0
2022-01-05T20:37:26.5477594Z stat::peak_rss_mb:              44
2022-01-05T20:37:26.5707319Z 2022-01-05 20:37:26,569 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2022-01-05T20:37:26.5708979Z 2022-01-05 20:37:26,570 - root - INFO - Trying to reproduce crash using: /tmp/tmp11sdfotu/crash-db46ee9027b1d3bc81d4a4d57752de3c4f841df8.
2022-01-05T20:37:27.1111894Z 2022-01-05 20:37:27,110 - root - INFO - Reproduce command returned: 77. Reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2022-01-05T20:37:27.1118804Z 2022-01-05 20:37:27,111 - root - INFO - Crash is reproducible.
2022-01-05T20:37:27.1213265Z 2022-01-05 20:37:27,120 - root - INFO - Trying to downloading previous build fc6455f2d50154ac7f4436bc1be3d02fd998b115.
2022-01-05T20:37:28.6243943Z 2022-01-05 20:37:28,623 - root - INFO - Done downloading previous build.
2022-01-05T20:37:28.6246347Z 2022-01-05 20:37:28,623 - root - INFO - Trying to reproduce crash using: /tmp/tmp11sdfotu/crash-db46ee9027b1d3bc81d4a4d57752de3c4f841df8.
2022-01-05T20:37:28.8747938Z 2022-01-05 20:37:28,874 - root - INFO - Reproduce command returned: 0. Not reproducible on /github/workspace/cifuzz-prev-build/fuzz-dwfl-core.

The action is relatively standard in evverx/elfutils#53 apart from REPORT_OOMS: false

run fuzzers with -jobs (or -workers?) for parallelism

github actions provides two cores, but clusterfuzzlite only uses 1. passing -j $(ncpu) gets free extra capacity :-)

There should probably be an easier way to integrate CFLite into projects integrated into OSS-Fuzz

Trying to integrate CFLite into my fork of elfutils in evverx/elfutils#35, I basically just copied all the files from the elfutils directory to $SRC/

FROM gcr.io/oss-fuzz-base/base-builder:v1
RUN apt-get update && \
    apt-get install -y pkg-config make autoconf autopoint zlib1g-dev flex bison gawk
COPY . $SRC/elfutils
RUN git clone --depth 1 https://github.com/google/oss-fuzz && \
    cp oss-fuzz/projects/elfutils/build.sh $SRC/ && \
    cp oss-fuzz/projects/elfutils/fuzz-dwfl-core.c $SRC/ && \
    cp oss-fuzz/projects/elfutils/fuzz-dwfl-core_seed_corpus.zip $SRC/
WORKDIR elfutils

I wonder if there is an easier way to do that?

ClusterfuzzLite does not check out submodules

I tried to add a custom step to fuzzing configuration to check out submodule, but it didn't help.

  - name: Checkout source
    uses: actions/checkout@v2
    id: checkout
    with:
      path: storage/proxy-model.py
      submodules: true

I got an error after all the fuzzers were built:

RuntimeError: Executing command "git rev-parse HEAD" failed with error: fatal: detected dubious ownership in repository at '/github/workspace/storage/proxy-model.py'
To add an exception for this directory, call:

	git config --global --add safe.directory /github/workspace/storage/proxy-model.py

What permissions do the GitHub Actions need?

This repo has two GitHub Actions that use the GITHUB_TOKEN. Can you please tell me what permissions are used for this token? If the permissions are only used in certain conditions, e.g. when a certain input is specified, please share that info as well.

At https://github.com/step-security/secure-workflows we are building a knowledge-base (KB) of permissions needed by different GitHub Actions. When developers try to remediate ossf/Scorecards checks, they use the knowledge-base to secure their GitHub Workflows.

Here is an example of how we store KB for an Action:

name: "GH Release"
github-token:
  action-input:
    input: github_token
    is-default: true
  permissions:
    contents: write
    contents-reason: to create GitHub release #Reference: https://github.com/softprops/action-gh-release/blob/fe9a9bd3295828558c7a3c004f23f3bf77d155b2/README.md?plain=1#L70

Mention in docs that OSS-Fuzz has a lot of examples to learn from

See #94 (comment)_

Spelling mistake - funtionality ---> functionality

Modular funtionality, so you can decide which features you want to use

Issue in build integration

Run fuzzers in local testing step during build integration step takes a lot of time

There doesn't seem to be a way to get CFLite to upload files crashing fuzz targets if they crash the latest build

I've been using CFLite to test various patches and when they introduce new issues CFLite uploads files to the "summary" tab as expected. But if CFLite thinks that bugs aren't introduced in PRs it just skips that step. I think it makes sense in general but in some cases bugs can collide so to speak and crash PRs and the latest build differently. It would be great if I could flip a flag named, say, 'always_upload_crashes` to avoid running something like

./infra/helper.py build_fuzzers --external ~/elfutils
./infra/helper.py run_fuzzer --external ~/elfutils fuzz-dwfl-core

in my VMs and copying files to machines that have access to bug trackers to just attach those files to bug reports.

jazzer_driver not available in local infra/helper.py run_fuzzer command

I have a working cflite jvm demo here: https://github.com/jtpereyda/clusterfuzzlite-demo/tree/188fc3ccfcf51968c9a6961b9989341777c16ce0
With a working build that succesfully runs the fuzzer (finding an example exception): https://github.com/jtpereyda/clusterfuzzlite-demo/runs/6393527402?check_suite_focus=true

However, when I try the recommended run_fuzzer command:

python infra/helper.py run_fuzzer --external --corpus-dir=<path-to-temp-corpus-dir> $PATH_TO_PROJECT <fuzz_target>

I get a /out/jazzer_driver: not found error:

> python infra/helper.py run_fuzzer --external --corpus-dir $PATH_TO_PROJECT/inputs $PATH_TO_PROJECT ExampleFuzzer
INFO:root:Running: docker run --rm --privileged -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e RUN_FUZZER_MODE=interactive -v /path/to/cfl-demo/inputs:/tmp/ExampleFuzzer_corpus -v /path/to/oss-fuzz/build/out/cfl-demo:/out -t gcr.io/oss-fuzz-base/base-runner run_fuzzer ExampleFuzzer.
rm: cannot remove '/tmp/ExampleFuzzer_corpus': Device or resource busy
/out/ExampleFuzzer -rss_limit_mb=2560 -timeout=25 /tmp/ExampleFuzzer_corpus < /dev/null
/out/ExampleFuzzer: 4: /out/jazzer_driver: not found

This led me on a goose chase trying to fix the local run before enabling the Github action and seeing that it was actually working. Still, it'd be super helpful to be able to test locally. I take it something on the server side is including the jazzer_driver executable, but that something is missing from the local run_fuzzer command.

There doesn't seem to be a way to run all fuzz targets in PRs

Both CFLite and CIFuzz exit as soon as they discover a bug but it would be great if it was possible to let them run all fuzz targets and report all the bugs. It would be consistent with how test suites work in general but other than that it should help in scenarios where bugs unrelated to PRs are more likely to pop up because the latest builds aren't available: systemd/systemd@d38363b

Investigate providing reusable workflows for GitHub users

Per: https://docs.github.com/en/actions/using-workflows/reusing-workflows#creating-a-reusable-workflow

This may make our instructions a lot simpler.

"Pruning" jobs seem to upload artifacts with no summaries

In https://github.com/evverx/libbpf/actions/runs/1728290427 the first "pruning" job uploaded a file named crashes-bpf-object-fuzzer:

$ unzip -l crashes-bpf-object-fuzzer.zip
Archive:  crashes-bpf-object-fuzzer.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  01-21-2022 10:59   address/
      417  01-21-2022 10:59   address/crash-1361a2bb8c3090236eff2b9eede58dd6af20bdc2
      269  01-21-2022 10:59   address/crash-560ab4bce5d578289415a41a9f173ab308185a55
     2560  01-21-2022 10:59   address/crash-65c8d13701ba5d2e2686fb2edf502a383f2602f2
      520  01-21-2022 10:59   address/crash-6d9b7d4464f9f2bb3f1f9248873cf07d983338bc
     2624  01-21-2022 10:59   address/crash-a88f6ed52a54dea0797beb168d56999d32d252f4
      520  01-21-2022 10:59   address/crash-c52e9ecec9d012f86b3de6784d6192911da71825
      525  01-21-2022 10:59   address/crash-fd369cf6cc2489d62360426a9d63e487f8df8fec
---------                     -------
     7435                     8 files

I'm not sure whether it should have been uploaded in the first place but artifacts uploaded by CFLite/CIFuzz usually contain summaries and the names of fuzz targets:

$ unzip -l address-artifacts.zip
Archive:  address-artifacts.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  01-23-2022 19:29   fuzz-dhcp-server/
        0  01-23-2022 19:29   fuzz-dhcp-server/address/
     9240  01-23-2022 19:29   fuzz-dhcp-server/address/crash-64c861ad981838e509920e8538640ef46feaae66.summary
      251  01-23-2022 19:29   fuzz-dhcp-server/address/crash-64c861ad981838e509920e8538640ef46feaae66
---------                     -------
     9491                     4 files

Empty Coverage Reports using ClusterFuzzLite with Prow and GO

On the kubernetes/test-infra repo (https://github.com/kubernetes/test-infra) we are running CFL but are receiving blank coverage reports.

Undefined references to C++ standard library symbols

On dependent packages, the docs state:

You should not make any assumptions on the availability of dependent packages in the execution environment and the built fuzzers should have dependencies statically linked.

What is the expected way of pulling the static libraries of dependent packages? Can I use apt packages or do I have to build them on my own using the provided CXX and CXXFLAGS? For example, I have a C++ project that uses Boost and Botan. When trying to link the packages from apt with my fuzz target, I get undefined references to C++ standard libary functions.

[ 50%] Building CXX object CMakeFiles/test_fuzzer.dir/test_fuzzer.cc.o
/usr/local/bin/clang++ -DBOOST_ALL_NO_LIB  -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link -stdlib=libc++ -std=c++17 -MD -MT CMakeFiles/test_fuzzer.dir/test_fuzzer.cc.o -MF CMakeFiles/test_fuzzer.dir/test_fuzzer.cc.o.d -o CMakeFiles/test_fuzzer.dir/test_fuzzer.cc.o -c /src/cflite-example/test_fuzzer.cc
[100%] Linking CXX executable /builds/2286933411/build-out/test_fuzzer
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/test_fuzzer.dir/link.txt --verbose=1
/usr/local/bin/clang++ -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link -stdlib=libc++ -fsanitize=fuzzer -rdynamic CMakeFiles/test_fuzzer.dir/test_fuzzer.cc.o -o /builds/2286933411/build-out/test_fuzzer  /usr/lib/x86_64-linux-gnu/libboost_log.a /usr/lib/x86_64-linux-gnu/libboost_chrono.a /usr/lib/x86_64-linux-gnu/libboost_date_time.a /usr/lib/x86_64-linux-gnu/libboost_filesystem.a /usr/lib/x86_64-linux-gnu/libboost_regex.a /usr/lib/x86_64-linux-gnu/libboost_thread.a /usr/lib/x86_64-linux-gnu/libboost_atomic.a 
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_log.a(core.o): in function `boost::log::v2s_mt_posix::core::get_thread_attributes() const':
(.text+0xea4): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: (.text+0xf02): undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_log.a(core.o): in function `boost::log::v2s_mt_posix::core::add_thread_attribute(boost::log::v2s_mt_posix::attribute_name const&, boost::log::v2s_mt_posix::attribute const&)':
(.text+0x13cb): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: (.text+0x142c): undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_log.a(core.o): in function `boost::log::v2s_mt_posix::core::remove_thread_attribute(boost::log::v2s_mt_posix::attribute_set::iter<false>)':
(.text+0x191b): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: (.text+0x197c): undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_log.a(core.o): in function `boost::log::v2s_mt_posix::core::set_thread_attributes(boost::log::v2s_mt_posix::attribute_set const&)':
(.text+0x2c9b): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)'
/usr/bin/ld: (.text+0x2cfc): undefined reference to `std::out_of_range::out_of_range(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_log.a(core.o): in function `boost::system::system_error::what() const':
...

And more ...

I was able to create a minimal reproducable project that links Boost statically installed via apt.

CFlite doesn't seem to show backtraces by default when bugs aren't reproducible

https://github.com/evverx/elfutils/runs/4541456435?check_suite_focus=true

2021-12-16T00:15:02.4107312Z INFO: Seed: 1337
2021-12-16T00:15:02.4497204Z INFO: Loaded 1 modules   (15250 inline 8-bit counters): 15250 [0x7dee1b, 0x7e29ad), 
2021-12-16T00:15:02.4502091Z INFO: Loaded 1 PC tables (15250 PCs): 15250 [0x747a30,0x783350), 
2021-12-16T00:15:02.4510120Z INFO:        7 files found in /github/workspace/cifuzz-corpus/fuzz-dwfl-core
2021-12-16T00:15:02.4511154Z INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 249851 bytes
2021-12-16T00:15:02.4516748Z INFO: seed corpus: files: 7 min: 568b max: 249851b total: 279535b rss: 70Mb
2021-12-16T00:15:28.6084552Z 2021-12-16 00:15:28,513 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2021-12-16T00:15:28.6095082Z 2021-12-16 00:15:28,573 - root - INFO - Trying to reproduce crash using: /tmp/tmp1zc5yps3/tmp83o6vkm6.
2021-12-16T00:15:29.3282172Z 2021-12-16 00:15:29,325 - root - INFO - Reproduce command returned: 0. Not reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2021-12-16T00:15:29.3303060Z 2021-12-16 00:15:29,326 - root - INFO - Crash is not reproducible.
2021-12-16T00:15:29.3436231Z 2021-12-16 00:15:29,341 - root - INFO - Deleting corpus and seed corpus of fuzz-dwfl-core to save disk.
2021-12-16T00:15:29.3456056Z 2021-12-16 00:15:29,344 - root - INFO - Deleting fuzz target: fuzz-dwfl-core.
2021-12-16T00:15:29.3475358Z 2021-12-16 00:15:29,345 - root - INFO - Done deleting.
2021-12-16T00:15:29.3488252Z 2021-12-16 00:15:29,347 - root - INFO - Fuzzer fuzz-dwfl-core finished running without reportable crashes.
2021-12-16T00:15:29.3543613Z 2021-12-16 00:15:29,353 - root - INFO - No crashes in /github/workspace/out/artifacts. Not uploading.

Looking at "report-unreproducible-crashes: false" it seems I can flip that flag to make it "reportable" but I think it would be great if backtraces were shown by default even when bugs aren't reproducible.

Affordances for "local maxima" in coverage

clusterfuzz has various strategies to try to handle the problem of getting stuck in a local maxima for coverage.

clusterfuzzlite doesn't appear to have these, as a result some fuzzers may find themselves stuck, unable to make progress.

I believe https://github.com/alex/rust-asn1 is experiencing this. The fuzzer is not making much progress, however if I blow away all coverage and run it from the start I can often get it to advance further.

build.sh example for go test -fuzz

I write fuzz tests in Go 1.18 following https://go.dev/doc/fuzz/ and run them with a command such as go test -fuzz=Range. How can I integrate them with ClusterFuzzLite?

I ran python3 infra/helper.py generate --external --language=go $PATH_TO_PROJECT but now I'm stuck. What do I put in build.sh? It would be neat to have an example in the documentation. The current example looks C++ specific https://google.github.io/clusterfuzzlite/build-integration/#buildsh

There doesn't seem to be a way to minimize crashes

I think ideally files triggering new issues should be added to regression testsuites and to make it a bit easier I think it would be great if CFLite could minimize them. For example, in https://github.com/evverx/elfutils/actions/runs/1660223230 the file is kind of huge (15221 bytes) considering that the same issue can be triggered with just 196 bytes. Other than that it's usually much easier to track bugs down and fix them when files triggering them are as small as possible.

allocator_may_return_null doesn't seem to be set to 1 when fuzzers are run under MSan

I added a file to the seed corpus and it triggered an unreproducible MSan crash I reported in #67 (comment). By running ./infra/helper.py I noticed that it failed because the allocator was out of memory. Issues like that are kind of ignored in ASan runs where allocator_may_return_null=1 is passed among ASAN_OPTIONS and propagate until real OOMs are hit, which is handled by the REPORT_OOMS setting.

CFLite reports an issue that CIFuzz can't find

I'm not sure whether it's a CFLite issue or CIFuzz issue but somehow in evverx/elfutils#35 CFLite was able to catch a heap-buffer-overflow in about 40s while at the same time in evverx/elfutils#25 CIFuzz kept fuzzing practically the same code for about three minutes and found nothing. It would be great if they could produce the same results (in this particular case the heap-buffer-overflow should have been found by CIFuzz)

Is someone working on integrating this into Gitlab's CI?

The title.

If not, this issue could be used to implement that change.

Document how to provide dictionaries and seed corpus to fuzzers

Hi, first of all thanks for all the awesome work you've put into this project really appreciate it.

For several of my fuzzers I've created dictionaries and/or have large number of sample files which I obviously would like the fuzzers to take advantage of.

I've looked through the documentation here but was not able to find the answers there so maybe someone here can help me.

Add screenshots to documentation.

We should add some screenshots to the documentation, e.g. UI for GitHub actions.

CFlite artifacts are called "cifuzz-****"

I'm not sure how important this is but looking at https://github.com/evverx/libbpf/actions/runs/1728607427 it seems the artifacts should be called "cflite-***" instead. The only scenario I can imagine where it could break something is when both CFLite and CIFuzz are enabled at the same time though.

google / clusterfuzzlite Goto Github PK

clusterfuzzlite's Issues

Recommend Projects

Recommend Topics

Recommend Org