Azure Blob Storage: DVC uses (wrong) environment variable instead of SAS token about dvc HOT 3 CLOSED

Environment variables take precedence over DVC and Azure file-based configs (.dvc/config, .dvc/config.local ~/.azure/config, etc). So, I think this behavior is correct.

In this case, I would assume the AZURE_STORAGE_ACCOUNT_KEY doesn't have the key in the proper format (e.g. "hello" is not a correct base64 or something and that's why some error from the Azure lib is thrown). I can't find the reference to this env variable in Azure libraries docs right away, but my guess it is the same as the AZURE_ACCOUNT_KEY (I have tested this and both variables work the same way for me).

So, in this case, if you want to have that env var you need to get the Account key

If you feel there is still a concern, please reopen and provide a bit more details:

• please add more details on why this is an issue for you (why do you need both SAS and the KEY set)
• run dvc pull -v and include the full stack trace
• please upgrade DVC - it's quite old and outdated and might be bringing some outdated Azure libraries with it.

from dvc.

For reference, support for the AZURE_STORAGE_ACCOUNT_KEY env var comes from the underlying library DVC uses for accessing azure blob storage (https://github.com/fsspec/adlfs). As noted by @shcheklein, it's a historical equivalent for the current Microsoft/Azure documented AZURE_ACCOUNT_KEY

The following environmental variables can also be set and picked up for authentication:

• "AZURE_STORAGE_CONNECTION_STRING"
• "AZURE_STORAGE_ACCOUNT_NAME"
• "AZURE_STORAGE_ACCOUNT_KEY"
• "AZURE_STORAGE_SAS_TOKEN"
• "AZURE_STORAGE_TENANT_ID"
• "AZURE_STORAGE_CLIENT_ID"
• "AZURE_STORAGE_CLIENT_SECRET"

from dvc.

Thanks Ivan and Peter for your explanations. This subject in not a bug indeed, and you provided very useful explanation that maybe could be included in the documentation?

For context:
- please add more details on why this is an issue for you (why do you need both SAS and the KEY set)
I need the KEY set for another Azure Storage access, that has nothing to do with DVC. (hence the "hello" value in my example, as useless for DVC as the real value I use).

- please upgrade DVC - it's quite old and outdated and might be bringing some outdated Azure libraries with it.
As said, I used version 3.39 first, and the problem was the same, I downgraded to the version my colleagues use as an attempt to solve my issue. Happy to go back to latest version now :-)

I propose to update the documentation to:

• clarify the precedence order of credentials. For now, the doc states the contrary of the real behavior (i.e. env. var first):

For custom authentication, you can set the following config params with dvc remote modify --local, use environment variables, or an Azure CLI config file (in that order).

• explicitly warn users about environment variables names that should be avoided when working with DVC, including the namesadlfs uses.

from dvc.