Strip release executable? about buck2 HOT 11 CLOSED

FWIW, just compressing the existing --release build on Linux with tar --zstd -cvf takes the binary size from 104M -> 30MB. With lto = true and opt-level = z this can go down to 18MB with zstd (but has a horrible effect on compilation time, because the link step becomes very long).

So if the primary motivation is about saving bandwidth in aggregate, honestly the biggest win would come from just applying compression to the release artifacts. This can be done with very little change. Also, for the open source builds produced by GitHub Actions, the build-time change of setting lto = true may not be that bad — but it also takes a lot more memory to complete, too.

I assume actual changes to the default optimization level e.g. opt-level = z would require "real" profiling to see what the impact is at scale — but at this point, probably the only people in a place to do that are Meta engineers, and my (possibly naive) understanding is that Meta deploys Buck2-built-with-Buck2. So there would need to be some work done here to get things rolling.

Here are some numbers I scribbled into Notepad to get an idea of the effect options had on the build. TL;DR lto = fat is better than thin by a solid margin, z makes a solid impact, codegen-units = 1 seems marginal. These aren't complete and some of the entries I tried are missing, so don't read too much into it. Scripting this would be better.

default:
austin@GANON:~/src/buck2$ ll -h target/release/buck2
-rwxr-xr-x 2 austin austin 104M Apr 10 12:57 target/release/buck2*

fatlto+z:
austin@GANON:~/src/buck2$ ll -h target/release/buck2
-rwxr-xr-x 2 austin austin 50M Apr 10 13:23 target/release/buck2*

fatlto only:
austin@GANON:~/src/buck2$ ll -h target/release/buck2
-rwxr-xr-x 2 austin austin 78M Apr 10 13:49 target/release/buck2*

thinlto+codegen-units=1:
austin@GANON:~/src/buck2$ ll -h target/release/buck2
-rwxr-xr-x 2 austin austin 84M Apr 10 13:59 target/release/buck2*

fatlto+z+codegen-units=1
austin@GANON:~/src/buck2$ ll -h target/release/buck2
-rwxr-xr-x 2 austin austin 48M Apr 10 14:08 target/release/buck2*

from buck2.

Do you expect every user of buck2 to require stack trace functionality? It's not uncommon to ship fully-stripped binaries, and then when investigating issues, attempt to reproduce them on a more instrumented version of the software.

This is a 30% binary size reduction- if you have 1MM buck2 users each downloading it only once, you've saved 1,000,000 * 23MB == 23TB of global internet traffic, and that's not even considering an army of CI robots that will likely download it from somewhere many times per day. If the hope is that 99+% of them won't experience buck2 crashes, that seems like a huge win :)

from buck2.

I'd love to think that Buck2 gets to 1MM, but alas, we are probably a way off for now! At the moment we're definitely keen to get proper stack traces, as that ensures that if/when something goes wrong there isn't a need to try reproducing it first.

For our internal builds we use --strip-debug and -S on Linux/Mac, which strips most debug info (you can't debug into it), but keeps enough for good stack traces. From memory, I think that saved 90% of the space compared to full stripping. That's probably a good point to aim for. CC @themarwhal (see deploy/buck2_upload.py for what we do internally).

from buck2.

Thanks for taking the time to write that response up despite my veering off of the topic for this issue :) I figured the answer was roughly what you described. It's amusing to me that even the fully-stripped undebuggable buck2 executable is the same size as a minimal Debian image! I guess disk space is cheap these days...

from buck2.

Resolved with 5b6e86c and 312fb00 . (latest job: https://github.com/facebook/buck2/actions/runs/4689802059)

from buck2.

Unstripped binaries produce better stack traces (when panicking or when using normal anyhow errors with RUST_BACKTRACE=1). This is not obvious improvement.

from buck2.

My two cents -- maybe add these to Cargo.toml:

[profile.release]
incremental = true
lto = "thin"
strip = "debuginfo"

[profile.min-size-release]
inherits = "release"
codegen-units = 1
lto = "fat"
opt-level = "s"

[profile.release-strip]
inherits = "release"
strip = "symbols"

[profile.min-size-release-strip]
inherits = "min-size-release"
strip = "symbols"

from buck2.

Thanks for the suggestions and stats! I really appreciate it. Taking a look at compressing + some baseline stripping today.

from buck2.

It's slightly off-topic, but I'm curious if there are any binary size targets in general that Buck2 is aiming at. I know it does a lot, but 50+MB for a single native executable is also pretty big!

I don't know if Rust has anything for size profiling or tracking, but tools like https://github.com/google/bloaty have come in very handy for me when auditing binary size in different languages.

from buck2.

The .text section of the binary is about 52MB on my normal release build from earlier (so, 104MB total), the rest going to debug info. Using a tool like cargo-bloat doesn't immediately point anything out to me (actually the biggest symbol is actually a sqlite3 function!); one issue of course is Rust's strong reliance on monomorphization means that a lot of individual symbols will end up being very small, similar copies and so won't stick out like a sore thumb in a "top N" list, it's just death by a thousand cuts. This is one reason why LTO and outlining from opt-level = z are relatively effective on any-ol-average Rust (and C++!) codebase in my experience, because those many tiny symbols get deduplicated/outlined/merged/dead-code-eliminated more effectively thanks to global visibility over the code graph. However there're definitely cases where monomorphization is best undone and calls should be left to sit behind a dereference. It's hard to see that sometimes from raw histogram of symbol names, though...

Honestly, text section size is something an afterthought for many things today, not just Rust projects — specifics of things like monomorphization or larger dependency chains (that have to be more aggressively optimized away with something like LTO) aside. So I'd just guess it was somewhat low on the list. But there are at least a couple easy immediate wins. More aggressive ones would be things like boxing/dyn'ing some dispatches, or even replacing dependencies, but that's probably out of scope for this issue.

from buck2.

We have no specific targets for size - stripping and compression seems worthwhile. If you happen to stumble on something that improves code quality and reduces size (and typically reduces compile time - which is always welcome!) then we'd be delighted to take a diff. But we aren't particularly focused on it, and there's no specific target size at which we worry.

from buck2.