Volume on Host / Containers not reflecting Bucket Contents about docker-s3fs-client HOT 10 CLOSED

@TheNexter thanks :) i already figured it out tho, the issue was me using the compose plugin for docker instead of standalone docker-compose... Unfortunately setting propagation just plain didn't work with the compose plugin

from docker-s3fs-client.

Are you running this against something else than AWS? In that case, you would need to specify the URL at which to contact the S3 API, e.g. https://s3.yourprovider.com or similar. Tell me if it helps.

from docker-s3fs-client.

@efrecon I had the same problem. Inside container files are existing. But in volume nothing. I tried to use docker volumes and volume mapping to host machine. The same behaviour.

AWS_S3_BUCKET=BYBUCKET
AWS_S3_ACCESS_KEY_ID=MYID
AWS_S3_SECRET_ACCESS_KEY=MYKEY
AWS_DEFAULT_REGION=MYREGION
AWS_S3_URL=https://s3.provider
S3FS_ARGS=use_path_request_style

  s3fs:
    cap_add:
    - SYS_ADMIN
    security_opt:
      - apparmor:unconfined
    privileged: true
    image: efrecon/s3fs:1.91
    restart: unless-stopped
    env_file: .env
    volumes:
      - s3data:/opt/s3fs/bucket:rshared
  test:
    image: bash:latest
    restart: unless-stopped
    depends_on:
      - s3fs
    # Just so this container won't die and you can test the bucket from within
    command: sleep infinity
    volumes:
      - s3data:/data:rshared
volumes:
  s3data:

from docker-s3fs-client.

I did an experiment. I opened shell in s3fs and umount /opt/s3fs/bucket. Then I tried to create a folder. And this folder was showed in another container.

Then I deleted folder 'test' and run tini -g -- docker-entrypoint.sh again. And got that in second container nothing

Also I build a new image from ubuntu latest and there is the same problem

from docker-s3fs-client.

@logicalor heya! did you manage to figure out a fix to this? I have the same issue, when i exec into the container and modify the contents of the bucket folder, it works, but not when i mount it to the host and then mount things from the host into the other containers

from docker-s3fs-client.

It seems as if the volume from the container is not getting mapped to the host container from what i'm seeing, 'cause all the other services mount properly to the host, except s3fs

from docker-s3fs-client.

@logicalor I figured out the issue with the help of my friend and trial / error... There is a bug in docker compose plugin it seems docker/compose#9380 so when you run it using docker compose up it just wont work since the propagation doesn't get updated no matter what you put in docker-compose, you can check it by docker inspect {container_name} and check propagation under the moutns section.

Fixed by uninstalling docker-compose-plugin and installing standalone docker-compose

from docker-s3fs-client.

Enjoy everyone : https://github.com/TheNexter/Rclone-Mount-Docker-Compose
@truesteps @nrukavkov @logicalor

from docker-s3fs-client.

Just some tips from my experience:

• Mount propagation doesn't work on Docker Desktop (Windows machines I believe). The project will still build and run fine but the mount will only be accessible within the s3fs container.
• If you're deploying this docker as a container in a Compose project to a cloud service, be careful! Using AWS EC2 and Elastic Beanstalk I could not for the life of me get the bucket to be propagated into other containers. EC2 and services that involve deploys have their own procedure for building and running Dockers, which involves many intermediate steps that might not be documented.

Possible workarounds:

• Build a single Dockerfile (no point in using Compose) and multi-stage builds to FROM this docker-s3fs-client image from Dockerhub, then copy in these files into your container. You will need to use this entrypoint.sh and related files, so I'd copy them into your root folder and then add the running command as CMD. This would result in a longer 1st build time but it'll be optimized using the Buildkit for future builds. A single container approach works on AWS and when built locally on machines with Docker Desktop
• aws s3 sync CLI command
• AWS DataSync service

Thanks to the package contributors for your efforts!

from docker-s3fs-client.

Thanks for figuring this out. I have added a mention of this issue in the main README.

from docker-s3fs-client.