azure / kubelogin Goto Github PK
View Code? Open in Web Editor NEWA Kubernetes credential (exec) plugin implementing azure authentication
Home Page: https://azure.github.io/kubelogin/
License: MIT License
A Kubernetes credential (exec) plugin implementing azure authentication
Home Page: https://azure.github.io/kubelogin/
License: MIT License
The README has a lot of AKS references but no examples for unmanaged clusters. This is a plugin for all Kubernetes clusters on Azure, not just AKS, README should reflect that.
examples of AKS specific mentions in README:
If you are using kubeconfig from AKS AADv1 clusters
To configure the role binding on Azure Kubernetes Service
myAKSAutomationServicePrincipal
In addition, it would be good to add a generic set up guide that doesn't assume AKS AADv2 doing everything for you, kind of like https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/client-go/plugin/pkg/client/auth/azure/README.md but for out-of-tree (how to set kubeconfig context for a new cluster, etc.)
Related to #24, I use Windows primarily and to solve lensapp/lens#649 kubelogin was the perfect solution. I looked at the releases tab and aside from the source, a single zip is present. I almost glossed over this as a solution because there is no indication that the build artifact contains Windows, Linux and Mac binaries. It would be helpful to indicate it is for all platforms, or separate these into platform specific files.
Thanks!
I noticed the linux binary from Releases is dynamically linked which makes it incompatible with some linux distributions such as alpine.
Of course, I can compile my own version but it would be very useful if the release version was already statically linked. Especially for use in lightweight docker containers.
Other Go tools such as kubectl are distributed as fully static binaries, so I assumed kubelogin would be too.
Hi,
Will there be support for AKS cluster created with managed identity (--enable-managed-identity ) instead of service principle ? in this case doesn't exist.
kubelogin v0.0.11, psversion 7.2.0 core, windows 10.
I expected that when $env:kubeconfig is set, kubelogin would respect the kubeconfig environment variable in the same way that kubectl does.
Test:
> $env:kubeconfig="C:\Projects\.kube\config"
> az aks get-credentials --resource-group myRG --name myCluster --file $env:kubeconfig
The behavior of this command has been altered by the following extension: aks-preview
Merged "myCluster " as current context in C:\Projects\.kube\config
> kubelogin convert-kubeconfig -l azurecli
This results in a new config file being created at %userprofile%\.kube\config but I was expecting the config file at $env:kubeconfig to be updated.
If $env:kubeconfig is not used/set, kubectl and kubelogin updates the file located at %userprofile%\.kube\config
How can I get kubelogin to update the config file located at a specific location, namely $env:kubeconfig ?
thanks
Sorry for opening an issue for a usage question - I'm not sure if there's a better forum? GitHub Discussions don't seem to be enabled here.
In any case, if I wanted to connect kubectl
to our AAD enabled AKS cluster I would (naively?) think to create a manged identity, assign it the AKS RBAC Writer role and run the pod using that identity (using the AAD Pod Identity project).
Inside the pod I would connect to the Azure CLI:
az login --identity
...and then use az aks get-credentials
to connect kubectl
to our AKS.
Is there some problem with doing that? Would that not work? If that does work then where does kubelogin
fit into the picture?
I am using a main service to authenticate with the Kubernetes API in an environment of continuous integration.
My cluster is set up and enabled to integrate with Azure AD and everything works fine for over a year.
Currently I created a main service that has permission Azure Kubernetes Service Cluster User Role
to my AKS.
Right at the azure cli normally and I can get kubeconfig to start interacting with the Kubernetes API.
az login --service-principal -u <spn_id> -p <spn_secret>--tenant XXXXXXX
az aks get-credentials --name xxxx --resource-group xxxx --subscription xxxx --overwrite-existing
Then I call kube login to convert my kubeconfig to the expected model.
kubelogin convert-kubeconfig --client-id <spn_id> --client-secret <spn_secret> --tenant-id XXXXXXX --legacy -l spn
At this point, I already have the ClusterRoleBinding created and using the ObjectId
of this SPN.
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: sp-role-binding
subjects:
- kind: User
apiGroup: rbac.authorization.k8s.io
name: <spn_object_id>
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
My kubeconfig after conversion is something like:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: xxxxxx
server: https://<uri>:443
name: xxxxxx
contexts:
- context:
cluster: xxxxxx
user: clusterUser_xxxxxx
name: xxxxxx
current-context: xxxxxx
kind: Config
preferences: {}
users:
- name: clusterUser_xxxxxx
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- get-token
- --environment
- AzurePublicCloud
- --server-id
- <server_id>
- --client-id
- <spn_id>
- --tenant-id
- XXXXXXX
- --legacy
- --client-secret
- <spn_secret>
- --login
- spn
command: kubelogin
env: null
After that, having a simple command to search for running pods and object always unauthorized.
I0312 13:23:15.244292 27174 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.18.8 (linux/amd64) kubernetes/9f2892a" 'https://<uri>:443/api?timeout=32s'
I0312 13:23:16.993746 27174 round_trippers.go:443] GET https://<uri>o:443/api?timeout=32s 401 Unauthorized in 1749 milliseconds
I0312 13:23:16.993817 27174 round_trippers.go:449] Response Headers:
I0312 13:23:16.993834 27174 round_trippers.go:452] Audit-Id: xxxxxxx
I0312 13:23:16.993847 27174 round_trippers.go:452] Cache-Control: no-cache, private
I0312 13:23:16.993859 27174 round_trippers.go:452] Content-Type: application/json
I0312 13:23:16.993870 27174 round_trippers.go:452] Content-Length: 129
I0312 13:23:16.993881 27174 round_trippers.go:452] Date: Fri, 12 Mar 2021 13:23:16 GMT
I0312 13:23:16.996236 27174 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
I0312 13:23:16.998572 27174 cached_discovery.go:121] skipped caching discovery info due to Unauthorized
In contrast, if I try to generate the token manually by executing the following command, I get the token normally.:
kubelogin get-token --environment AzurePublicCloud --server-id <server_id> --client-id <spn_id> --tenant-id XXXXXXX --client-secret <spn_secret> --login spn
Am I doing something wrong? Is there a step I need to take that I didn't take? I am looking for help to solve this problem.
I try to use Azure CLI token login (non interactive) from jenkins pipeline to login to AKS with non interactive but got the following error:
+ az login --service-principal --username **** --password **** --tenant mytenant -o none
[Pipeline] sh
+ az aks get-credentials --name mycluster -g mykaas
Merged "mycluster" as current context in /root/.kube/config
[Pipeline] sh
+ export 'KUBECONFIG=/root/.kube/config'
[Pipeline] sh
+ kubelogin convert-kubeconfig -l azurecli
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ kubectl get pods -n fm-gtv
E0728 10:11:50.667941 139 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper
E0728 10:12:22.668147 139 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper
Unable to connect to the server: context deadline exceeded
Who can help? Is there something missing?
The old Azure Active Directory plugin describes using two AAD application, and the same approach seems to be applicable to kubelogin too based on the --server-id: <AAD server app ID>
and --client-id: <AAD client app ID>
parameters.
For AKS clusters one is fixed:
The AAD server app ID of AKS Managed AAD is always
6dae42f8-4368-4678-94ff-3960e28e3630
in any environments.
however for non-AKS clusters (e.g. EKS) is it okay to use the same AAD app for both, or is there some (security) issues with doing so?
It would be nice if kubelogin supported converting a single cluster in a kubeconfig file, and let the other cluster configurations be as-is to reduce the blast radius.
Hi.
I caught such bugs:
$ make
rm -f bin/linux_amd64/kubelogin
CGO_ENABLED=0 go build -o bin/linux_amd64/kubelogin -ldflags "-X main.version=v0.0.6/1ee0da905a5b9ce9dceacff7e914b3c3c1e198ea -X main.goVersion=go1.15.2 -X main.buildTime=2020-10-13T09:09:19Z"
go: downloading k8s.io/client-go v0.17.4
go: downloading github.com/spf13/cobra v0.0.6
go: downloading k8s.io/cli-runtime v0.17.4
go: downloading github.com/Azure/go-autorest v14.2.0+incompatible
go: downloading k8s.io/apimachinery v0.17.4
go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.1
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading github.com/Azure/go-autorest/tracing v0.6.0
go: downloading github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d
go: downloading k8s.io/api v0.17.4
go: downloading github.com/imdario/mergo v0.3.5
go: downloading k8s.io/utils v0.0.0-20191114184206-e782cd3c129f
go: downloading golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456
go: downloading github.com/evanphx/json-patch v4.2.0+incompatible
go: downloading k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: downloading golang.org/x/net v0.0.0-20191004110552-13f9640d40b9
go: downloading github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d
go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0
go build: when using gccgo toolchain, please pass linker flags using -gccgoflags, not -ldflagsgithub.com/Azure/kubelogin
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:55: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:30: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:23: error: undefined reference to 'gccgoRealSyscallNoError'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:37: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:44: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:50: error: undefined reference to 'gccgoRealSyscallNoError'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:74: error: undefined reference to 'reflect.typelinks'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:78: error: undefined reference to 'reflect.resolveTypeOff'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:42: error: undefined reference to 'reflect.typelinks'
make: *** [Makefile:33: kubelogin] Error 2
I am using
$ go version
go version go1.15.2 gollvm LLVM 12.0.0git linux/amd64
and my here is my environment:
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/oceanfish81/.cache/go-build"
GOENV="/home/oceanfish81/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/oceanfish81/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/oceanfish81/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/oceanfish81/gollvm_dist"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/oceanfish81/gollvm_dist/tools"
GCCGO="/home/oceanfish81/gollvm_dist/bin/llvm-goc"
AR="ar"
CC="/usr/bin/clang"
CXX="/usr/bin/clang++"
CGO_ENABLED="1"
GOMOD="/home/oceanfish81/go_projects/kubelogin/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build215526650=/tmp/go-build -gno-record-gcc-switches -funwind-tables"
Logging in with type "MSI" (i.e. -l msi
) later fails with parameter 'identityResourceID' cannot be empty
. These commands are being run on an Azure VMSS with a system-assigned identity.
Repro commands:
az login --identity
az account set -s {subscription}
sudo az aks get-credentials --resource-group {resourceGroup} --name {clusterName} --overwrite-existing
sudo kubelogin convert-kubeconfig -l msi
sudo kubectl get nodes
At this point we are seeing this error consistently when using any kubectl commands that access the cluster.
Error: failed to get token: failed to create service principal from managed identity for token refresh: parameter 'identityResourceID' cannot be empty
Hi, we are currently using kubelogin to authenticate against Azure AD and then use the JWT groups claim to authenticate against an EKS cluster in AWS.
Currently we have no Assignment required restrictions on the enterprise application, this way everyone from our organization can get a valid JWT token by using kubelogin. We do this because this saves us quite the administrative hassle of adding additional new groups to that enterprise application before they can sign in. We only check if a user is part of our domain and has valid credentials and some conditional access policies.
With that valid token you are able to use kubectl and when you know the cluster_ca and the cluster_url you are allowed to do calls to the kubernetes api. On the cluster itself we then have clusterrolebindings & rolebindings to further delegate permissions with regular kubernetes RBAC based of the objectid of a (A)AD group that needs to have permissions on the clusterresource(s).
For some of our users we have the well known 200 groups limit in the JWT token issue. Based of that I've got the following question, is it possible to bypass this by having kubelogin directly query the graph url that's in the JWT and inject the groupid's in the JWT? groups:src1
This in itself doesn't seem like a complex function to add in kubelogin but maybe there is some hidden complexitity that I don't know of.
Hello,
it would be great if there were checksums available directly so we can validate during download.
e.g. additional assets:
e.g. compare to oidc-login
assets.
https://github.com/int128/kubelogin/releases/tag/v1.24.0
Thank you!
It doesn't appear that there is a way to configure the kubectl exec plugin when using spn for authentication. Are there updates required for this plugin to work? If this requires testing, I'm definitely willing to help out as I am building automated processes for spn authentication against AKS Managed AAD clusters.
Why exactly are there only instructions for brew?
This should be on winget and instructions should be added ASAP. (and az cli should install it automatically)
let's improve code coverage
Support Managed Identity Token with kubelogin
The limitation is documented here.
The fix is to adopt auth code grant flow
Sample code: https://github.com/Azure/azure-sdk-for-go/blob/master/sdk/azidentity/interactive_browser_credential.go
With the MS ADAL library going out of support in June, is there work already underway by someone to switch from the ADAL to the MSAL libraries?
Hi,
I create a clusterrolebinding in my cluster for SPN giving it admin role at k8s level.
after that I used kubelogin and ran the below commands
kubectl convert-kubeconfig
set AAD_SERVICE_PRINCIPAL_CLIENT_ID = <>
set AAD_SERVICE_PRINCIPAL_CLIENT_SECRET = <>
after that when I am trying to use
kubectl get nodes
kubectl get pods
its giving below error
error: you must be logged into the server(unauthorized)
please if any guidance what am I doing wrong
I have test kubelogin with Managed Service Identity with below scenario:
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: test-msi-rolebinding
subjects:
Am I doing something wrong or it is current behaviour?
After following kube config the conversion script I have updated kubeconfig
:
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- get-token
- --environment
- AzurePublicCloud
- --server-id
- xxx
- --client-id
- xxx
- --tenant-id
- xxx
- --legacy
command: kubelogin
env: null
After running a kubectl command and confirming my identity I have credentials stored in ~/.kube/cache/kubelogin/azure.json
.
When I tried to run another kubectl command and it asked me to confirm my identity again.
I tried to not use --legacy
and authenticate again and I got from k8s api server (obviously non-legacy type of credentials is not accepted):
error: You must be logged in to the server (the server has asked for the client to provide credentials)
When I tried to switch back to --legacy
and still got the same error message.
The error disappeared when I cleared the cache and reauthenticated but I was back to having to authenticate every kubectl
execution.
Helllo:
Would we be able to release this binary for linux/arm64
architecture?
Currently, I see we only have the following:
My team uses this binary as dependency for another project, and we would like to start supporting ARM64 architecture, such that the following URL resolves:
https://github.com/Azure/kubelogin/releases/download/v${KUBELOGIN_VERSION}/kubelogin-linux-arm64.zip
Can we get a signed version of kubelogin.exe, we are only allowed to use signed binaries and exe in our production systems.
I followed the instructions in the Service principal login flow (non interactive) section, and I am able to successfully run kubelogin convert-kubeconfig -l spn
and set the environmental variables as instructed. Upon trying to execute any command such as kubectl get no
or even kubectl get namespace
, I receive the following error:
Error from server (Forbidden): nodes is forbidden: User "<service-principal-object-id>" cannot list resource "nodes" in API group "" at the cluster scope
I created the ClusterRoleBinding as instructed in the readme, and, when I login to kubectl
interactively, I can confirm that the CRB is definitely in the cluster:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: sp-role-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: <service-principal-object-id>
What am I missing here, that is preventing me from accessing the AKS cluster using non-interactive service principal?
The environment variables used here are different than those used by the other MSFT SDKs (dotnet, golang, java)
All of them use:
AZURE_CLIENT_ID
AZURE_TENANT_ID
AZURE_CLIENT_SECRET
AZURE_USERNAME
AZURE_PASSWORD
Here, you need to define entirely different variables names like AAD_SERVICE_PRINCIPAL_CLIENT_ID
. Is there a reason for this inconsistency? Having it means that any execution environment that needs to use both kubelogin
and an azure SDK must re-define the same values under two different names.
Current implementation decodes PKCS12 with empty password, however there are flows where the PFX might be password protected.
https://github.com/Azure/kubelogin/blob/master/pkg/token/serviceprincipaltoken.go#L89
Another thing thing I've noticied is that it should be pretty straight forward to also support PEMs (just like the Azure CLI supports).
https://github.com/Azure/kubelogin/blob/master/pkg/token/serviceprincipaltoken.go#L200
Kubelogin today supports getting AAD token using client id and client secret. The request here is to get AAD access token using client ID and certificate.
can support this adal api: adal.NewServicePrincipalTokenFromCertificate
Maybe I'm not following the directions correctly, but I'm prompted for an interactive login when using
$ kubelogin convert-kubeconfig -l spn
$ kubectl get no
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FN4N26MMT to authenticate.
Environment is set
$ env |grep AAD
AAD_SERVICE_PRINCIPAL_CLIENT_ID=foo
AAD_SERVICE_PRINCIPAL_CLIENT_SECRET=bar
Kubeconfig
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <snip>
server: <snip>
name: aks-westus-test
contexts:
- context:
cluster: aks-westus-test
user: clusterUser_rg-aks-westus-test_aks-westus-test
name: aks-westus-test
current-context: aks-westus-test
kind: Config
preferences: {}
users:
- name: clusterUser_rg-aks-westus-test_aks-westus-test
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- get-token
- --environment
- AzurePublicCloud
- --server-id
- <snip>
- --client-id
- <snip>
- --tenant-id
- <snip>
command: kubelogin
env: null
Although kubelogin
addresses kubernetes/kubernetes#86410 to remove spn:
prefix in audience
claim, you may encounter the same issue while using kubelogin
with Managed Service Identity (MSI) in Azure Container Instance (ACI).
For example
az login --identity --username ${MSI_CLIENT_ID}
kubelogin convert-kubeconfig -l azurecli
kubectl get pods
error: You must be logged in to the server (Unauthorized)
## If `guard` log is enabled in AKS control plane, you will see
## failed to verify token for azure: oidc: expected audience "6dae42f8-4368-4678-94ff-3960e28e3630" got ["spn:6dae42f8-4368-4678-94ff-3960e28e3630"]
The issue is in ACI where they have different infrastrucutres such that the returned MSI token on legacy one will include spn:
prefix in audience
claim. You can verify by examining the AAD token by kubelogin get-token --server-id 6dae42f8-4368-4678-94ff-3960e28e3630 --client-id ${MSI_CLIENT_ID} --login msi
and paste the access token to https://jwt.ms and look at the audience
claim. If it has spn:
, your ACI is running on the legacy infrastructure.
Please open support request to ACI team to migrate your subscription
We encountered an issue today in our build pipeline. The build script installs the kubectl
as part of az aks install-cli
. However, the latest version seems to be not available yet at the time of this posting. The following link returns 404.
https://github.com/Azure/kubelogin/releases/download/v0.0.10/kubelogin.zip
I am using Kubelogin plugin and run command kubelogin convert-kubeconfig -l azurecli
to refresh aks token. but got the following error:
Any direction on how I can grant the permission or any insight on the issue?
It does successfully refresh the aks token but sometimes the kubectl commands are not working
We're want to run run Kubernetes deployments from Container Instances with attached managed identities, but are hitting an issue with MSI authentication. We are running kubelogin
in the Managed Service Identity (non interactive) setup, but after the call to kubelogin convert-kubeconfig -l msi
, any kubectl
command returns the error:
error: You must be logged in to the server (the server has asked for the client to provide credentials)
When running the same script from a VM using the same managed identity, the kubectl commands succeed after running kubelogin
. This leads us to believe that our cluster/managed identity setup is correct, but that the issue lies with managed identities in container instances.
Hi,
I have a Azure AD enabled AKS cluster. I am trying non interactive login with a SP that has Cluster User role assigned to it. But when I execute via kubectl I get:
I0723 17:27:13.956018 6753 execCredentialPlugin.go:111] acquire new token
I0723 17:27:15.046242 6763 execCredentialPlugin.go:111] acquire new token
I0723 17:27:15.789685 6774 execCredentialPlugin.go:111] acquire new token
I0723 17:27:16.445092 6784 execCredentialPlugin.go:111] acquire new token
I0723 17:27:17.195349 6795 execCredentialPlugin.go:111] acquire new token
I0723 17:27:17.926715 6805 execCredentialPlugin.go:111] acquire new token
error: You must be logged in to the server (Unauthorized)
But if I use kubelogin get-token I get the token
An offcial container image would be great. I don't know if kubelogin
can be used with AKS like aws-iam-authenticatior
is used with EKS (node authentication) but I know use cases where people use container images for packaging their tooling.
I'm using Docker based Azure App Service as a Azure Devops deployment agent for reaching our private network. In this app service enabled Identity and I want to use this identity for accessing AKS.
In AKS granted ClusterAdmin role for this identity:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: <binding name>
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: <managed identity object id>
After configuration I tried to run deployment, but got next error:
W0209 12:49:50.510666 1246 transport.go:260] Unable to cancel request for *exec.roundTripper Error: failed to get token: adal: Failed to execute the refresh request. Error = 'Get http://169.254.169.254/metadata/identity/oauth2/token?api-version=2017-09-01&resource=<id>: dial tcp 169.254.169.254:80: i/o timeout' Unable to connect to the server: getting credentials: exec: executable kubelogin failed with exit code 1 (Client.Timeout exceeded while awaiting headers)
Kubelogin tries to get token from 169.254.169.254, but this endpoint works only under VM's, for App Services there is available "MSI_ENDPOINT" environment variable and kubelogin should use that endpoint. Also in the kubelogin this endpoint is hardcoded: https://github.com/Azure/kubelogin/blob/master/pkg/token/msi.go#L39
Continuing investigation I've found that in a newer version of library adal GetMSIVMEndpoint()
is deprecated and NewServicePrincipalTokenFromMSI()
can automatically detect correct MSI endpoint: https://github.com/Azure/go-autorest/blob/master/autorest/adal/token.go#L714
Would like to propose the following contract in kubeConfig for a cluster that mandates access tokens with proof of posession (PoP) semantics.
kind: Config
preferences: {}
users:
- name: user-name
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
command: kubelogin
args:
- get-token
- ...... specific auth flow parameters
- --with-pop
provideClusterInfo: true
To indicate that the returned token should be of type "pop" conforming to SHR format with API server's host name (conveyed via provideClusterInfo) captured as "u" claim.
Packaging as a krew
plugin would solve #25 and streamline the user experience. There are however some existing generic ODIC krew
plugins and Homebrew
repositories using the same name:
https://github.com/int128/kubelogin
https://github.com/Nordstrom/kubelogin
As this project is specific to Azure OIDC authentication, would it not be more appropriate to name as such? For example:
azure-kubelogin
azure-oidc-login
I would like to help with the krew
packaging but need direction on naming first.
It would be nicer if the linux binaries were presented in tar.gz format to avoid having to install zip to extract them as they appear to be multiple entry zip archives.
subj
support multiple cached tokens based on environment, serverID, clientID, tenantID, and legacy
Hi! I'm trying to use the Service principal login flow to login to my AAD enabled connected cluster to enable non interactive login for Azure/aks-set-context action. However, I've been getting this error:
server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server has asked for the client to provide credentials",
"reason": "Unauthorized",
"details": {
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "Security token validation failed."
}
]
},
"code": 401
}]
Kubelogin works fine with interactive flow, but doesn't seem to work with service principals.
Also, I could not find any executables in the latest release. So, I used the makefile in the source code to get kubelogin.exe. Could that be a reason for this issue? Or is there anything that I am missing?
I would like to run kubectl commands using a user-assigned managed identity against an AKS cluster.
I have created a user-assigned managed identity (az identity create
) that I would like to use to call kubectl
commands against my AKS cluster (setup with Managed ID and managed AAD enabled, cluster version is 1.18.6). Here the steps I did to get the kubeconfig file updated using kubelogin
:
az aks get-credentials ... // create a kubeconfig file with my user's info
kubelogin convert-kubeconfig -l msi --client-id <clientid-of-my-managed-identity>
kubectl get pods // This fails (see below)
The kubeconfig file looks as follows after running kubelogin
:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: <cert-blob>
server: https://mycluster.hcp.westeurope.azmk8s.io:443
name: mycluster
contexts:
- context:
cluster: mycluster
user: clusterUser_myresourcegroup_mycluster
name: mycluster
current-context: mycluster
kind: Config
preferences: {}
users:
- name: clusterUser_myresourcegroup_mycluster
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
args:
- get-token
- --server-id
- <...>
- --client-id
- <clientid-of-my-managed-identity>
- --login
- msi
command: kubelogin
env: null
When running kubectl get pods
command, nothing happens for several seconds and then the following output is printed to the console:
Error: failed to get token: adal: Failed to execute the refresh request. Error = 'Get http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=<...>&resource=<...>: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.'
Unable to connect to the server: getting credentials: exec: executable kubelogin failed with exit code 1
I also tried to assign the MSI the following roles:
Azure Kubernetes Service Cluster User Role
AKS rolecluster-admin
role in the cluster as described in the kubelogin README.mekubectl
command executes successfully
Any suggestions what I'm missing? Would be great if you could point me to some E2E sample.
Can this please be published to packages.microsoft.com so apt install can be used to obtain it?
Hello, everything on the kubelogin side looks to be working perfectly.
Not having much luck with the above error which I am getting in the kube-apisever log.
I can see there is a parameter --oidc-ca-file which is probably what I need. Where I am at we have security which rewrites SSL. I have the ca-bundle for those, which had to be installed to worker nodes in order to successfully pull images. I've tried the same ca-bundle via the --oidc-ca-file parameter without success.
Not sure how to resolve this. Since many using kubelogin (azure version) may have also run into this, I thought this would be a good place to post the question as many may later on also come here and search for this error message.
Is there a certificate specific to azure's identity provider which I need to download, and configure via --oidc-ca-file?
I'm nearly certain this is user error so apologies in advance.
I have an AKS cluster running 1.23.5. I can auth to it fine after running az aks get-credentials. However I keep getting the warning when I run any Kubectl commands as follows:
I have installed the AKS CLI and added both the the environment variables to my path. Is this expected to continue to get this warning when I run any command? If so, is there a way to supress it?
It would be very useful to get a homebrew integration
I am trying to use kubelogin from CI system and I am facing the following error while trying to authenticate.
2021-04-21 18:48:36.443474: apiVersion: v1
2021-04-21 18:48:36.443524: clusters:
2021-04-21 18:48:36.443613: - cluster:
2021-04-21 18:48:36.443699: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUS....RklDQVRFLS0tLS0K
2021-04-21 18:48:36.443779: server: https://xx-xx-xx-xx.hcp.eastus.azmk8s.io:443
2021-04-21 18:48:36.443800: name: coxxxx-aks-ppe-1
2021-04-21 18:48:36.443809: contexts:
2021-04-21 18:48:36.443865: - context:
2021-04-21 18:48:36.443880: cluster: coxxxx-aks-ppe-1
2021-04-21 18:48:36.443888: user: clusterUser_ConxxxxRG_conxxxx-aks-ppe-1
2021-04-21 18:48:36.443894: name: conxxxx-aks-ppe-1
2021-04-21 18:48:36.443900: current-context: conxxxx-aks-ppe-1
2021-04-21 18:48:36.443955: kind: Config
2021-04-21 18:48:36.443985: preferences: {}
2021-04-21 18:48:36.443994: users:
2021-04-21 18:48:36.444037: - name: clusterUser_ConxxxxRG_conxxxx-aks-ppe-1
2021-04-21 18:48:36.444044: user:
2021-04-21 18:48:36.444050: exec:
2021-04-21 18:48:36.444056: apiVersion: client.authentication.k8s.io/v1beta1
2021-04-21 18:48:36.444065: args:
2021-04-21 18:48:36.444071: - get-token
2021-04-21 18:48:36.444077: - --server-id
2021-04-21 18:48:36.444108: - 6dxxxx-4368-xxxx-94ff-xxxx
2021-04-21 18:48:36.444126: - --login
2021-04-21 18:48:36.444133: - azurecli
2021-04-21 18:48:36.444139: command: kubelogin
2021-04-21 18:48:36.444145: env: null
2021-04-21 18:48:36.444156: helm version
2021-04-21 18:48:36.499299: version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTreeState:"clean", GoVersion:"go1.14.13"}
2021-04-21 18:48:36.500722: kubectl version
2021-04-21 18:48:38.130123: Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
2021-04-21 18:48:38.130861: error: You must be logged in to the server (the server has asked for the client to provide credentials)
The cluster is AAD RBAC enabled and has MSI as well.
Here is the script that executes az cli comands followed by kubelogin
n=0
SLEEP_TIME=2
while true; do
echo "Try # $n"
az login --identity
signInExitCode=$?
if [[ $signInExitCode -eq 0 || $n -ge 5 ]]; then break; fi
n=$((n + 1))
echo "sleeping for $SLEEP_TIME"
sleep $SLEEP_TIME
SLEEP_TIME=$(( SLEEP_TIME * 2 ))
done
if [ $signInExitCode -eq 0 ]; then
echo "authenticated to azure"
else
echo "failed to authenticate to azure"
exit 1
fi
az account set --subscription $deploymentSubscription
az aks get-credentials --resource-group $deploymentResourceGroup --name $targetAKSCluster
apt update && apt install -y unzip >&-
curl -sSLO https://github.com/Azure/kubelogin/releases/download/v0.0.9/kubelogin-linux-amd64.zip && unzip kubelogin-linux-amd64.zip
mv ./bin/linux_amd64/kubelogin /usr/local/bin/kubelogin
echo "kubelogin install completed"
echo ${KUBECONFIG}
echo "kubelogin version"
kubelogin --version
echo "kube config file"
cat ~/.kube/config
export KUBECONFIG=~/.kube/config
kubelogin convert-kubeconfig -l azurecli
Hello,
thanks for this project. Is it foreseen to integration with "service connections" in Azure devops?
Our pipelines use kubectl tasks: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops . These tasks use a "service connection". It would be nice if kubelogin becomes one of the "service connection" choices for kubernetes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.