Darwin installer: create a case sensitive APFS volume about nix HOT 4 OPEN

The next time I upgrade my system, I probably won’t use a case-sensitive store unless I’m given no choice because it doesn’t solve case-sensitivity problems while building (because /tmp is case-insensitive) while adding extra hassles.

I’ve experienced two sets of issues with using it:

• Meson occasionally has problems with case-sensitivity. See:
  • mesonbuild/meson#11820
  • mesonbuild/meson#12587
  • mesonbuild/meson#13038
• gnatcoll-python3 fails to build even though the casing of the file is actually correct (see NixOS/nixpkgs#310450 (comment)). I have no idea why. 🫤

If the store is made case-sensitive by default, then I hope the builders are converted immediately. Otherwise, that will hide compatibility issues that will cause problems for users when they try or need to build things from source.

from nix.

Since I'm floating some ~challenges, I'll start by saying that I would like to see this happen myself, and took explicit action in the installer to make this easier for people to fiddle with so that the adventurous among us could start discovering the downsides.

I'm not one of these people, but the few things I'm aware of are:

• The utility of a case-sensitive store may be constrained if builds are still done in /tmp on the default-insensitive root volume. (Maybe this work can be moved under /nix, but then we don't get to lean on the OS cleaning it up on reboot and probably need to actively manage it? It sounds like your notes above mean something along these lines, but I don't see clean-up addressed.)
• @reckenrode has noted (and AFAICT has upstreamed fixes for) some meson issues on a case-sensitive store. Hopefully he can elaborate as to whether that's an ongoing or solved problem.
• If we start migrating the install base, over time we'll lose eyes on namespace clashes that cause trouble here, but there may still be some meaningful fraction of the install base with old case-insensitive installs. (This is probably just a thing to keep in mind, but maybe Nix could also be doing some explicit detection of these and erroring/warning on them during a transition period before we finally shrug and suggest people with insensitive stores uninstall/reinstall?) Sounds like you are just thinking we use the warnings to nudge people in this boat to fix up. That should be fine.
• This one's speculative/theoretical. Some people work around problems we have getting nix-installed Mac apps in a number of ways, some of which include copying the .app files out of the store and into an Applications directory (because symlinks cause trouble). I wonder a little bit if we'll see any weird behavior plop out if the apps (which probably aren't rigorously tested on case-sensitive file systems) build on a case-sensitive FS and then get copied out to an insensitive one.

I'll mention this in the macOS room, but I would also recommend searching the macOS room for case-sensitive and clicking through the discussions around this, since it's been discussed at least a few times over the past few years.

from nix.

Sorry--I'm a little scattered from storm-related displacement in Houston and did a poor job of absorbing implications of bullets in OP. Edited a little to improve but probably still rehashes stuff y'all already have in mind.

One more, though:

Provide documentation for making /nix case sensitive and link to it from the warnings

When we've seen this come up in the macOS room, I think it was reported (maybe by @lilyball?) that there isn't a way to convert the volume. On the up side, this probably means that the "instructions" can just be 2 steps linking to uninstall and reinstall instructions?

from nix.

Lightly edited reproduction of further Matrix discussion on the tmpfile cleanup angle:

Randy Eckenrode

I missed the last bullet. I don’t think warning about TMPDIR is enough. Nix needs to pick a new location because TMPDIR will always be case-insensitive on Darwin. It also needs to make sure that location gets cleaned
regularly because /tmp accumulates a ton of build cruft.

Huh. When did macOS drop daily tmpfile cleaning?
/etc/periodic/daily/110.clean-tmps is present on my wife’s iMac (running macOS 13.whatever) but missing on my > MBP (running macOS 14.5).
macOS 14. I checked another Mac that was upgraded from 13.x, and the file is also gone.

For cleanup, couldn’t nix just install its own clean-tmps for the nix build location?

pxc@pxc:matrix.org

does it no longer clean those files, or just rely on a different mechanism for it?

Randy Eckenrode

There’s a tmp_cleaner launch daemon that runs /usr/libexec/tmp_cleaner. It doesn’t appear to be configurable.
Oh, it’s actually in the diskdev_cmds source release.
It’s just a shell script and definitely not configurable.

https://github.com/apple-oss-distributions/diskdev_cmds/blob/main/tmp_cleaner/tmp_cleaner
Once diskdev_cmds is updated, it might be possible to use that script to do the cleanup. It would require > setting up a launch agent for it and patching the script with the new store’s tmp path.

That said, the tmp_cleaner script itself looks fairly simple and leans on both sed/find, so it'd probably make more sense to just build this into Nix than to depend on patching what's in diskdev_cmds.

from nix.