gcsfs: Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object about gcsfs HOT 3 OPEN

GCP credentials block using the service account credentials file

What does this mean?

You can specify token= when instantiating GCSFileSystem, which can point to any gcloud JSON file, it sounds like this might be what you need.

from gcsfs.

This is an example of what it looks like creating a Prefect block programtically. Prefect is a workflow orchestration tool

from prefect_gcp import GcpCredentials

# replace this PLACEHOLDER dict with your own service account info
service_account_info = {
  "type": "service_account",
  "project_id": "PROJECT_ID",
  "private_key_id": "KEY_ID",
  "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n",
  "client_email": "SERVICE_ACCOUNT_EMAIL",
  "client_id": "CLIENT_ID",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL"
}

GcpCredentials(
    service_account_info=service_account_info
).save("BLOCK-NAME-PLACEHOLDER")

After persisting this on Prefect Cloud, I can reference the keys inside a flow.

You can specify token= when instantiating GCSFileSystem, which can point to any gcloud JSON file, it sounds like this might be what you need.

I tried using project = credentials
where credentials are loaded from Prefect GCP Credentials block
Hope that makes sense

from gcsfs.

It is not project but token that you need to set to the saved credential file's location. You should also provide project= ideally (this can matter for some operations).

from gcsfs.