Provide a way to cache the installation about setup-msys2 HOT 22 CLOSED

I'll open a PR once I have something, not sure re branches, let's see how it works out.

from setup-msys2.

Related to this there was #56 and #55 to handle corner cases that are more likely with longer living installations. I think this is good to go now.

from setup-msys2.

Thanks for clarification.

Is there some specific use case that we are not covering yet?

I'll report if I found anything

from setup-msys2.

Afair we currently only cache the downloaded packages, which doesn't save much. The reinstall will always happen.

from setup-msys2.

Oh, are there reasons against caching the installations too?

from setup-msys2.

Assuming you install some software or change some files in the installation we'd restore that on the next run, but ideally a cache would be transparent. Ideas welcome. Think like you'd want to cache "/" in an Arch Linux install.

from setup-msys2.

I'll look into enabling package caching by default. That will at least avoid the confusion regarding the "cache" option we provide (by removing it..)

from setup-msys2.

As an end user who just uses MSYS2 to compile C files using CMake, I don't have too much knowledge about what paths there are overall.
I guess running CMake and gcc/clang should not change files in the installation directory at all, right?

Thank you for looking into it! Really appreciated.

from setup-msys2.

I guess running CMake and gcc/clang should not change files in the installation directory at all, right?

Unless you install with it, no. It might still change /home and /tmp. Point is we can't cover all cases because the user can do everything. We'll probably have to expose the cache functionality to the user and let the user decide if and when keeping the whole directory around is OK.

from setup-msys2.

I've created #51 for removing the cache option

from setup-msys2.

I think a good strategy for a full install cache would be to expose a cache-key and cache-restore-keys option which we prefix and pass through to @action/cache. This way the user controls when and between which runs things are cached and we just take care of the "what" and "how".

Open question if we go that way: "pacman -Scc" would be a good idea to clear the package cache, so we don't cache twice. And maybe we should nuke /home on restore? And what to do if an update after the cache restore fails? Maybe then just nuke everything and start new. -> all in all, tricky, but not impossible.

from setup-msys2.

Sounds good, I guess in my use case I would just pick a constant key like cache-key: deadbeef, no?

from setup-msys2.

Yeah, probably. Or include the architecture if you want a separate cache for 32bit and 64bit etc.

@eine any thoughts on my suggestion #50 (comment) ?

from setup-msys2.

I'm not sure... My initial try was to use a single key and retrieve/update it every time. That didn't work. Then, I thought about letting users specify a key. But that was not intuitive. The point, as commented in #23, is that it seems not possible to share a cache between jobs or workflows. It is neither possible to update a key. Hence, I don't know what's the namespace where the keys are being used. That is, I have no idea if the cache that is restored makes any sense.

from setup-msys2.

I got confused again with my comments above.. we store the cache before anything uses it, and not after, so we have perfect control of what we store anyway, so please forget everything I said :/

Re sharing: Based on my experiments in #51 sharing caches between different jobs works fine. Here is one run that gets a cache from a different config: https://github.com/msys2/setup-msys2/runs/888874485?check_suite_focus=true According to https://docs.github.com/en/actions/configuring-and-managing-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache caches can also be read from PRs, but updates will be created in the PR scope.

Re key: I think the current approach is similar to what we'd need: Have one key which is used as a restore and one that is unique. The restore should be a combination of all settings, the action version, the installer URL, so you can be sure you just get an older version of a "similar" run. For the unique key we can hash /var/lib/pacman which contains info about what is currently installed and the package database. If the unique changes we create a new cache.

from setup-msys2.

@lazka, as far as I understand the proposal is to cache the installed dependencies. Does that mean caching / or some specific dir? If / is cached, it wouldn't make much sense to download the installer in future runs. Hence, maybe the version of the installer should be part of the cache restore key. What do you think?

from setup-msys2.

Hence, maybe the version of the installer should be part of the cache restore key. What do you think?

Yeah, that's what I had in mind by adding "the installer URL".

from setup-msys2.

I like it, specially because retrieving the full installation from the cache will significantly reduce the traffic in msys2's servers. Users will retrieve the installer once only. Do you want to prototype it in a PR and I will release a "temporal" branch for users to try before merging it to master?

from setup-msys2.

Is there any sample on this?

from setup-msys2.

@tavrez, caching of packages is done by default. What do you mean with "a sample"?

from setup-msys2.

I thought we can use actions/cache with proper keys to cache the whole setup process, and wanted to see a sample workflow with setup-msys2 and cache with keys and cache folders

from setup-msys2.

Since June (see https://github.com/msys2/setup-msys2/blob/master/CHANGELOG.md#110---20200605) the packages are cached by default (using actions/cache) (see #51, #54). For some time, both packages and the full installation were cached. However, there were some issues with lost files (#61), and we decided to disable it (temporaly, at least) (see #63). As a result, this action is already caching as much of the setup as possible, there is no need for users to handle keys and folders explicitly. See, for example: https://github.com/ghdl/ghdl/runs/1170192304?check_suite_focus=true#step:2:104. You see a similar output anywhere you use this action as explained in Usage. Is there some specific use case that we are not covering yet?

from setup-msys2.