Possible performance hack about atheris HOT 4 CLOSED

Interesting, we'll have to compare the performance versus the current added instrumentation.
Do you know if this technique would break the line numbers that are printed in exceptions and such?

from atheris.

Do you know if this technique would break the line numbers that are printed in exceptions and such?

It most certainly does https://github.com/risicle/cpytraceafl#doesnt-abusing-lnotab-break-pythons-debugging-mechanisms

from atheris.

I've done some performance benchmarking. For the lnotab strategy, I modified Atheris as follows:

• I replaced Atheris' patch_code function with this solution's rewrite function
• In native code, I called PyEval_SetTrace to install a tracer with no effect (except to enable line tracing).

For Atheris's strategy, I used the benchmarks as written, except that I disabled dataflow tracing. Since the lnotab strategy doesn't have dataflow tracing, this makes the comparison fair.

# lnotab rewrite:
low_cyclomatic	runs=40000	time=0.20s	execs/sec=200006.87
high_cyclomatic	runs=2000	time=0.62s	execs/sec=3234.16

# Atheris rewrite:
low_cyclomatic	runs=40000	time=0.13s	execs/sec=318779.70
high_cyclomatic	runs=2000	time=0.27s	execs/sec=7369.53

The existing strategy used by Atheris is 1.5-2x faster. Atheris was disadvantaged in this benchmark, since my minimal lnotab implementation didn't actually register or increment any counters, reducing the time spent in libfuzzer.

from atheris.

Quite surprised by this - I guess tracehook-calling is not considered a fast-path in cpython (if you're installing any sort of trace hook, I imagine the thought is you're not after top performance), while all the bytecode ops used in the full rewriting are.

from atheris.