image bazelbuild/buildfarm-worker:v2.7.0 fails to start with "libfuse.so.2: cannot open shared object file: No such file or directory" about bazel-buildfarm HOT 8 CLOSED

I've just run a worker with an omitted storages entry in the config, and the default config properly kicks in to identify a single storage with type: FILESYSTEM that does not go through that codepath of createFuseExecFileSystem.
Are you able to paste your config to confirm that it does not contain a storages in the container? The worker logs also dump a blurb on startup which guarantees enumeration of the actual content loaded.

from bazel-buildfarm.

backplane:
  queues:
    - name: "cpu"
      properties:
        - name: "min-cores"
          value: "*"
        - name: "max-cores"
          value: "*"
worker:
  storages:
    - type: GRPC
      target: "garlic.nz1.lmax:1985"

from bazel-buildfarm.

Also, it appears I cannot start a worker with out the storages or worker keys (or an empty storages map) in the config as the options parser either NPE's or fails to parse an empty map.

from bazel-buildfarm.

The docs are a little light-weight on what a valid configuration is (over the minimal example).

I note that there is a FUSE enum but only GRPC and FILESYSTEM are noted in the docs.

from bazel-buildfarm.

The FUSE storage is not recommended for real installations. FILESYSTEM is the supported enum type and must appear first in a list of storages for the worker to provide value - either the worker is a CAS shard and needs its own storage, or it must have a filesystem to present inputs to actions for execution (on a filesystem). If you have a secondary GRPC type storage, the FILESYSTEM storage will read through it and expire into it if configured.

Why are you set on not declaring a FILESYSTEM for the worker?

from bazel-buildfarm.

Why are you set on not declaring a FILESYSTEM for the worker?

I'm not, but I already have a GRPC cache in place. The docs are very vague on what the actual required configuration is. It seemed like I could just put a GRPC configuration in.

The FUSE storage is not recommended for real installations.

So why is it trying to configure FUSE storage at all? Especially when it is not configured in the configuration file. It's just noise in the error logs that confuses anyone already confused by the vagueness of the docs.

FILESYSTEM is the supported enum type and must appear first in a list of storages for the worker to provide value - either the worker is a CAS shard and needs its own storage, or it must have a filesystem to present inputs to actions for execution (on a filesystem). If you have a secondary GRPC type storage, the FILESYSTEM storage will read through it and expire into it if configured.

This is exactly the information that should be in the docs!

from bazel-buildfarm.

I'm not, but I already have a GRPC cache in place. The docs are very vague on what the actual required configuration is. It seemed like I could just put a GRPC configuration in.

You definitely can use your existing GRPC cache, as detailed in the (admittedly poorly annotated) specification for a FILESYSTEM storage delegating (read-through/expiration) to a GRPC endpoint in the examples here: https://bazelbuild.github.io/bazel-buildfarm/docs/configuration/configuration/#worker-cas. Some better diagrams detailing the myriad of options we have for configuring behavior are definitely in order.

So why is it trying to configure FUSE storage at all? Especially when it is not configured in the configuration file. It's just noise in the error logs that confuses anyone already confused by the vagueness of the docs.

It happens to be the else clause from a check for the FILESYSTEM type storage, which selects its own ExecFileSystem, then just bails to start a Fuse-type EFS. Legacy upon legacy, and I've filed #1529 to track the cleanup I've wanted to do for a while here.

There actually was some mention of the FuseCAS in https://bazelbuild.github.io/bazel-buildfarm/docs/architecture/workers/#fuse, but it doesn't speak to why it is getting applied in your case, nor speak loudly enough about how it shouldn't be used.

This is exactly the information that should be in the docs!

Agreed. I've updated the architecture/workers and configuration base descriptions to try to improve the situation. Thanks for keeping us honest!

from bazel-buildfarm.

TYVM!

from bazel-buildfarm.