New API about impermanence HOT 18 CLOSED

Yeah, I think branches and PRs are the way to go here - they allow for fast iteration while keeping the stable version clean.

from impermanence.

Alright, I'll move the current new module into a branch

from impermanence.

Alright, @etu, after discussing with @talyz I now understand their issue better.

In the current next setup, when a file does not exist in the source (state) directory it is created during activation and then bind mounted to the target (ephemeral) directory. If, however, that file is meant to be populated for the first time, such as /etc/machine-id on a new box, most services will look at the (empty) file and get confused as they expect it to either not exist, or to exist with valid content. The same doesn't happen with a symlink which would just point to a non-existent file to be populated.

Alternatives are:

1. We symlink files, as opposed to bind mounting
2. We don't initialize missing bind-mounted files and warn about it during activation. Users then have to manually initialize their state based on what the service populated in the ephemeral fs.
3. We do (2.) but we automate the copying on newly initialized state.

I think 1. might be our best bet for now, and it is also aided by the work done in #3 to correctly build directory trees.

from impermanence.

Yep, I think 1, while not perfect, is our best bet right now.

Additionally, I would advocate for keeping the split as it is right now and doing this work (i.e. adding support for users in the NixOS module) in nixos.nix instead, for a few reasons:

• I quite like the split - at least as long as home-manager is the best way to manage user settings. I already put other file settings in my home-manager config, so I don't see why persistent state should be managed in a different place, as long as it doesn't come with obvious benefits. I would be fine with adding this user-specific functionality to the nixos.nix-module, thereby extending the API - people have different workflows and tastes ;)

• Doing the work in one place instead of starting over in new files keeps the history cleaner and, in this case, some of the improvements made in impermanence.nix could have been more easily added to nixos.nix when bind mounting turned out to not be viable.

• The name impermanence doesn't really correctly convey what we're trying to reach, which is actually the opposite - persistence. While I think it's fine as a name for the repo / project, where we can explain that the name refers to managing ephemeral systems, it feels a bit confounding to me to use the option environment.impermanence."/blah" to point to something I want to keep, when impermanence makes it sound like the referenced paths will be cleaned out.

Also, on a more personal note, I spent quite some time and effort getting these modules together, and more recently getting them readable by others to make cooperation easier. It doesn't feel great when all that code gets copied to a new file with modifications, the original files are deleted and the history thereby gets partially erased. I understand that this was not at all done with bad intentions, and I'm sorry if I'm being overly sensitive and whiny about this, but I figured it was best that I said something so we can find a solution that works for everyone.

from impermanence.

I also like the split as it allows me to keep my state configuration right next to the service/program configuration for the user, a few thoughts on this:

1. It's fine to use the home-manager infra for symlinking files
2. I'd like to use proper bind-mounting infra for directories, even for the user. Is that possible?
3. Maybe we can let users choose whether they want something linked or bound. Something along the lines of:

{
  environment.state."/state" = {
    bind = {
      files = [ "..." ];
      dirs = [ "..." ];
    };
    link = {
      files = [ "..." ];
      dirs = [ "..." ];
    };
  };
}

As it currently stands I see next as "let's try out ideas and see what sticks". I, for one, have no intention of doing git merge next. Successful things from there should make their way into master more individually. For example, we found that bind mounting files isn't viable for some things, so that won't get into master as-is, on the other hand we wrote some code to correctly replicate the permissions of dir structures, that could be brought onto master.

I agree with you about naming, impermanence is hip, but just calling it environment.state or something like that is way better for clarity.

Lastly, regarding authorship, credit, and so on, I don't think any of us want to erase or hide any of your work; I can say for my part I'm really grateful you were willing to donate your personal work to this effort. To be clear, without your initial modules and movement on the IRC this project would not exist at all. Again, I don't want to merge next, I'd much prefer for it to be used as a place to test new ideas.

from impermanence.

If you don't want to use the home-manager module, I guess the only thing missing from the NixOS module is handling files outside /etc, so that you could do

{
  environment.persistence."/state" = {
    directories = [
      "/var/lib/iwd"
      "/var/lib/bluetooth"
      "/home/bemeurer/pictures"
      "/home/bemeurer/src"
    ];
    files = [
      "/etc/machine-id"
      "/home/bemeurer/.gist"
      "/home/bemeurer/.zsh_history"
    ];
  };
}

We could add users.<user> as a shortcut, but it wouldn't be strictly necessary, and possibly a bit confusing when we have the home-manager module..

from impermanence.

I think the users.<user> part can be quite good to have, because with that we could make sure that the created files are owned by the right user.

from impermanence.

Yeah, the reason I formatted it like that was to ensure ownership, as opposed to having to do some hacky path-based detection (e.g. assuming all paths starting with /home/$user are owned by $user)

Another alternative is to get rid of the distinctions altogether and have something like this:

{ config, ... }: {
  environment.state = {
    directory = "/state";
    entities = [
      { target = "/var/lib/iwd"; type = "d"; }
      { target = "/home/bemeurer/.gist"; owner = config.users.users.bemeurer; type = "f"; }
    ];
  };
}

from impermanence.

Yeah, that's true. That's handled automatically with the home-manager module, since it runs as the user and links don't actually need to point to a file.

from impermanence.

@lovesegfault Hm, I would prefer the users.<user> one, since the second one is very verbose and gets close to systemd-tmpfiles.

from impermanence.

Um, why a new module? Can we make incremental improvements to the existing ones instead?

Also, not to be a bummer, but I tried impermanence.nix and it doesn't work - not because the code is bad or anything, but the concept doesn't work, sadly. My test case was /etc/machine-id - I end up with an empty file and broken systemd journal support. The problem with bind mounting files like this is that for many applications, the main check for whether to create a state file or not is simply going to be "does the file exist already", which is going to be true for bind mounts and false for symlinks on first run.

from impermanence.

I liked that a new module was created because it's easier not to clobber yours, which actually works, while we hack on the new one. Live a v1/v2 living side-by-side.

from impermanence.

Maybe a better approach would be to utilize git branches? E.g. leave master as-is with the currently-functional module(s), and work on a next branch or something where this new module slowly (or quickly) accumulates the features present on master, and then PR it when ready.

from impermanence.

IMHO for a project this early and this small with <5 people working on it the overhead of branches takes more than gives.

from impermanence.

Done, impermanence now lives in next and I've rebased #3 there.

from impermanence.

2. I'd like to use proper bind-mounting infra for directories, even for the user. Is that possible?

I'm currently experimenting with getting something working using bindfs and it works, but is a bit problematic in its current form, since it spawns fuse processes in the activation script, a oneshot service, which isn't ideal. I'll try to use a combination of setup in the activation script and user services to get around this.

3. Maybe we can let users choose whether they want something linked or bound. Something along the lines of:

{
  environment.state."/state" = {
    bind = {
      files = [ "..." ];
      dirs = [ "..." ];
    };
    link = {
      files = [ "..." ];
      dirs = [ "..." ];
    };
  };
}

Yeah, that sounds like a reasonable compromise. I suppose we could also do runtime detection of which method is preferable; if no file exists, link, otherwise bind mount.

As it currently stands I see next as "let's try out ideas and see what sticks". I, for one, have no intention of doing git merge next. Successful things from there should make their way into master more individually. For example, we found that bind mounting files isn't viable for some things, so that won't get into master as-is, on the other hand we wrote some code to correctly replicate the permissions of dir structures, that could be brought onto master.

Sounds good to me 👍

Lastly, regarding authorship, credit, and so on, I don't think any of us want to erase or hide any of your work

No, I get that :)

I can say for my part I'm really grateful you were willing to donate your personal work to this effort. To be clear, without your initial modules and movement on the IRC this project would not exist at all.

I'm really happy that someone other than me finds it useful and wants to work on it. Thank you for all your work and encouragement! :)

from impermanence.

Yeah, that sounds like a reasonable compromise. I suppose we could also do runtime detection of which method is preferable; if no file exists, link, otherwise bind mount.

How would runtime detection work? I was straying away from it b/c I wasn't sure how to get the filesystems to be dynamic (i.e. decided at activation time)

I really like the idea of "link fallback"

from impermanence.

I was thinking, for every file or directory, we could generate a systemd service which either bind mounts or links, and have them run as early as possible at system boot. I don't know how feasible this is, but it's similar to my plans for home-manager bind mounts, so that should give us an indication.

from impermanence.