Error using Service Principal with Certificate about azure-kusto-spark HOT 4 OPEN

Hello @vsaroopchand

Thanks for confirming with a screengrab here. Let me set the context , workarounds and next steps here ( Noam mentioned this earlier, just adding the full context around this)

Context:

The Synapse spark connector is a wrapper that provides abstractions over Kusto spark connector and authentication using the Synapse libraries. Internal to the Synapse auth repos are a set of libraries called TokenLibrary (https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python) . To make it easier for users, the Synapse spark connector calls this TokenLibrary API , gets the token and then passes it when calling Kusto.

The TokenLibrary auth part is broken unfortunately for ManagedIdentity. See illustration below where it fails with a BadRequest for MSI and Certificate based auth and works for an AAD app-based Auth. This is the exception you see bubble up into the notebook execution

Workaround:

• Easiest is to use AAD app based auth in the LinkedService, this will work.
• Using the AZ-Identity/MSAL libraries, get a Token and pass it in as a "accessToken" option in spark.

Next steps:

An ICM perhaps with the Synapse team will help in looking at taking their help and fixing the issue with the TokenLibrary API ..

from azure-kusto-spark.

Thank you @ag-ramachandran - can you provide more context on the first workaround "Easiest is to use AAD app based auth in the LinkedService, this will work."

from azure-kusto-spark.

Hello @vsaroopchand , You will have to create an APP Id / App Secret (Using App registration) and use that in the Synapse Wizard (The UI will request for AppId,AppKey and Tenant).

from azure-kusto-spark.

Thank you. Unfortunately I cannot use Client/Secret. I'll explore the MSAL option.

from azure-kusto-spark.