Metadata Caching in case of not existing file about mountpoint-s3 HOT 12 OPEN

Hey @nguyenminhdungpg. Mountpoint will indeed cache the non-existence of a file. We've sometimes described this as 'negative caching'. This behavior was introduced in Mountpoint v1.5.0.

It sounds like you would prefer that Mountpoint would not cache these entries, and instead go to S3 each time if it has not seen that the file exists. Can you tell us a bit more about your use case? What problems does caching the negative entries introduce for you?

from mountpoint-s3.

Hi @dannycjones thank you for your information.

My usecase for using disable_noobj_cache option of s3fs at this time is very weird and it is not good.

There are 2 services run in parallel and there is a chance that one requests file in s3fs mounted folder before the other finished adding the file to S3 using AWS SDK for S3.
If I don't use disable_noobj_cache, then the non-existences of the file and maybe its parent folders are cached, requests to the file with high frequency can make it (and its parent folders) seem to be not-existed forever. This also breaks a logic in my service that copies files from or create new files in the seem-to-be-not-existed-forever folders in s3fs mounted folder.

On another hand, I am wondering without 'negative caching', can an attacker brute-force requests to non-existing files and increase my S3 cost?

from mountpoint-s3.