djc / gcp_auth Goto Github PK
View Code? Open in Web Editor NEWMinimal authentication library for Google Cloud Platform (GCP)
License: Other
Minimal authentication library for Google Cloud Platform (GCP)
License: Other
I have discovered a lot of entries such as the following in my log files:
2022-05-20T08:29:28.053898Z DEBUG refresh_token: hyper::client::connect::http: connecting to 142.250.74.138:443 self=CustomServiceAccount { credentials: ApplicationCredentials { type: Some("service_account"), ...
The problem here, is that ApplicationCredentials
contains a plaintext private key which is getting printed repeatedly to our log files, as well as a lot of other information one might not choose to expose so liberally such as service-account email and id, etc.
Perhaps using something like #[instrument(skip_all)]
would be safer in this case?
When invoking Cloud Run endpoints, token needs to have audience of the endpoint url.
On 0.12.1
, my application started failing to get the credentials. Upon investigation, I noticed that 0.11.1
wasn't using the application_default_credentials.json
file, as I expected but using the gcloud
command to get the authorized user (via GCloudAuthorizedUser
).
The reason for failing to load application_default_credentials.json
is that, according to the Google Cloud documentation, on Windows the application_default_credentials.json
file is located at %APPDATA%\gcloud\application_default_credentials.json
, but the the application tries to load it from %HOMEPATH%/.config/gcloud/application_default_credentials.json
(looking at this line and this line.
It seems that GCloudAuthorizedUser::from_string
defaults the token's expiry_time
to None
, which causes Token::has_expired
to always return false. This results in AuthenticationManager::get_token
never refreshing a token that was retrieved via a GCloudAuthorizedUser
.
Are these tokens not intended to be refreshed? Would it be unreasonable to default the expiry_time
to one hour from when from_string
is called? Something like:
expires_at: Some(OffsetDateTime::now_utc() + Duration::hours(1))
I don't see anything about extending the token's expiry time in the documentation for the CLI command used to fetch the token, but I suppose it's possible the expiry time's default could be overridden somewhere else.
gcloud auth
appears to no longer write a user's application credentials to application_default_credentials.json
but instead to a user specific path under .config/gcloud/legacy_credentials/
. I believe the change was done when the authors moved to use SQLite to manage the tokens.
I'd like to add support for using gcloud auth print-access-token
directly but wanted to get feedback before committing to the process. Are there concerns or opinions about adding support for gcloud
?
Hi there, thanks for creating this nice package.
Does it support all kinds of different key jsons.
I tried to use the the json generated via
gcloud auth application-default login
and the key is located at /Users/fuyangl/.config/gcloud/application_default_credentials.json
CustomServiceAccountCredentials(Error("missing field `private_key`", line: 11, column: 1))
The key as "client_id", "client_secret", "refresh_token" and "type" as "authorized_user", that is all the fields there.
Best wishes for the New Year ๐
For one of my projects I need to generate tokens to interact with the GCP API. It looks like this is exactly what I need, especially since I'm looking for something simple instead of a full SDK.
In my case the JSON secrets file is stored in a database. Unless I have missed something, I believe reading the file from disk is currently the only option to use your library. Would you consider adding a function like:
gcp_auth::from_credentials_str
In this case the given string or slice of bytes already contains the content of the JSON file.
Maybe I'm just dumb and didn't understand how to use the library. Sorry if that's the case, but I'm perplexed about this output.
Output:
ya29.c.b0AT7lpjA9DDSaBgwjFjWyjZ6iXNU3Lmok9NQmStz4hCc3RuPOw2GQB2x5rHiKqNkNyEJHlLzEUbkJ1NF_g7wNSgCXeLnLJonMLSfmi1Sr8ZrMFCG1nDZV1tymD5dnYWX2S3g2fECKqXvJrUwxDcwe2WRhuPZmse-rFTGAm3xUdk9RpLQYLpwzVHz8rLvg33ULx7agLUcuGLf81s5GW9M82PjiEcDIgjk........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
You might want to fix that up. :)
It looks like ring was upgraded in main branch but the latest release in crates.io hasn't picked this up and pins to "0.16.20"
I am using the latest version (0.4.0)
I am getrring token from the following lines:
let authentication_manager = gcp_auth::init().await.unwrap();
authentication_manager.get_token(scopes).await.unwrap();
Execution never returns from this point.
On 0.12.0
and 0.12.1
, gcp_auth
is unable to find the cloud
command. This regression may be caused by the removal of the which
crate, which was being used to locate the command on 0.11.1
, but is not used on 0.12.0
and 0.12.1
2024-05-29T07:58:59.067572Z DEBUG provider: gcp_auth: initializing gcp_auth
2024-05-29T07:58:59.067780Z DEBUG provider: gcp_auth::custom_service_account: check for GOOGLE_APPLICATION_CREDENTIALS env var
2024-05-29T07:58:59.072484Z DEBUG provider: gcp_auth::config_default_credentials: try to load credentials from .config/gcloud/application_default_credentials.json
2024-05-29T07:58:59.072926Z DEBUG provider: gcp_auth::metadata_service_account: try to fetch token from GCP instance metadata server
2024-05-29T07:58:59.073294Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.073929Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.078290Z WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=0
2024-05-29T07:58:59.079130Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.079422Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.079742Z WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=1
2024-05-29T07:58:59.080038Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.080306Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.080620Z WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=2
2024-05-29T07:58:59.080921Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.081178Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.081468Z WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=3
2024-05-29T07:58:59.081753Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.082011Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.082268Z WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=4
2024-05-29T07:58:59.082537Z DEBUG provider: gcp_auth::gcloud_authorized_user: try to print access token via `gcloud`
Error: NoAuthMethod(Io("failed to run `gcloud`", Error { kind: NotFound, message: "program not found" }), Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))), Io("failed to open application credentials file", Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }))
2024-05-28T19:58:57.622868Z DEBUG new: gcp_auth::authentication_manager: Initializing gcp_auth
2024-05-28T19:58:57.627715Z DEBUG new: gcp_auth::default_authorized_user: Loading user credentials file
2024-05-28T19:58:57.628029Z DEBUG new:get_token{client=Client}: gcp_auth::default_service_account: Getting token from GCP instance metadata server
2024-05-28T19:58:57.628913Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.645776Z WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.646494Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.646779Z WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.646919Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.647205Z WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.647400Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.647647Z WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.647834Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.648058Z WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:59.817829Z DEBUG new: gcp_auth::authentication_manager: Using GCloudAuthorizedUser
2024-05-28T19:58:59.819169Z DEBUG reqwest::connect: starting new connection: https://us-central1-aiplatform.googleapis.com/
2024-05-28T19:58:59.819376Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="us-central1-aiplatform.googleapis.com"
2024-05-28T19:58:59.820883Z DEBUG hyper_util::client::legacy::connect::http: connecting to 142.250.179.234:443
2024-05-28T19:58:59.829227Z DEBUG hyper_util::client::legacy::connect::http: connected to 142.250.179.234:443
2024-05-28T19:59:00.487182Z DEBUG hyper_util::client::legacy::pool: pooling idle connection for ("https", us-central1-aiplatform.googleapis.com)
We're seeing that refreshing tokens from outside of a GCP env is slow, often around 1 second for us. Because of this we'd like to get the refresh done in background outside of the get_token
call. This is possible but awkward currently as there's no direct way for a consumer to trigger refresh other than to call get_token
in the last 20 seconds of the token lifetime.
Ideally the library would offer a background refresh option, or if not that, the ability to force a refresh.
I can put together a PR for either of these approaches if this is something you're interested in.
Hello!
I would like to know if there are any examples on how to use AuthenticationManager
in a multithreaded context?
The absence of the environment variable GOOGLE_APPLICATION_CREDENTIALS
should not preclude using DefaultServiceAccount
and DefaultAuthorizedUser
.
Hi!
Thanks for a great library, I am using it to build a small hobby project and so far everything has been running smoothly.
I noticed however, that the order of the credentials being loaded is not the same as other tools.
For examle, the Google documentation called Authenticating as a service account states:
ADC looks for service account credentials in the following order:
If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set, ADC uses the service account file that the variable points to.
If the environment variable GOOGLE_APPLICATION_CREDENTIALS isn't set, ADC uses the default service account that Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions provide.
If ADC can't use either of the above credentials, an error occurs.
Version 0.2.2 of gcp_auth
loads the credentials in this order:
Would you consider changing the order to be more in line with other google tools?
I would personally prefer this order:
Thanks!
A number of the functions in this library are instrumented with tracing::instrument
, but that defaults to an INFO
logging level. It would be helpful to set these to something higher like DEBUG
so we can manage whether these low level methods will show up in the logs when we instrument our own applications.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.