[Question] Understanding how many calls are made to KeyVault? about secrets-store-csi-driver-provider-azure HOT 7 CLOSED

This issue is stale because it has been open 14 days with no activity. Please comment or this will be closed in 7 days.

from secrets-store-csi-driver-provider-azure.

hey @aramase - wanted to follow up re the above question.

from secrets-store-csi-driver-provider-azure.

@kchilka-msft It's a GET call to Key vault per secret per pod. So, in the load test scenario, it is 20k calls to KV as it's the same 2 secret per pod and there are 10k pods.

In case of rotation, the list of all existing pods is generated and the contents in the mount are rotated linearly. A workqueue is created in the driver, all 10k pods added to queue and then the queue is processed. The AKV rate limit is 4000 GET/10s (xref: https://learn.microsoft.com/en-us/azure/key-vault/general/service-limits#secrets-managed-storage-account-keys-and-vault-transactions). As we're not making all 20k calls at once, the rate limit is not hit with the load tests. Having said this , if there is a burst of pods all requesting the secrets at the same time, the rate limit with be hit.

from secrets-store-csi-driver-provider-azure.

@aramase thanks for the explanation!

So in every rotation, the number of calls to AKV will be (number of pods * number of secrets)? But in a linear fashion so AKV throttling limit isnt breached?

Also, is there any explicit check you have that ensures calls to AKV are 4000 GET/10s?

from secrets-store-csi-driver-provider-azure.

So in every rotation, the number of calls to AKV will be (number of pods * number of secrets)? But in a linear fashion so AKV throttling limit isnt breached?

Yes!

Also, is there any explicit check you have that ensures calls to AKV are 4000 GET/10s?

There is no explicit check. The retries are in the SDK and in case of 429 error, the sdk returns error without retry.

from secrets-store-csi-driver-provider-azure.

This issue is stale because it has been open 14 days with no activity. Please comment or this will be closed in 7 days.

from secrets-store-csi-driver-provider-azure.

This issue was closed because it has been stalled for 21 days with no activity. Feel free to re-open if you are experiencing the issue again.

from secrets-store-csi-driver-provider-azure.