Cannot write to ADX from Azure Databricks using Kusto connector for pyspark "com.microsoft.kusto.spark.datasource" about azure-kusto-spark HOT 6 CLOSED

Hello @pk2k14

What is your networking strategy? Are you running Kusto in a private network ? (or) Azure Databricks in a seperate Subnet

Regards
Ram

from azure-kusto-spark.

Hi @ag-ramachandran,

Yes, Kusto is running in a private network, but we have all the network peerings configured between ADX and ADB.

from azure-kusto-spark.

Hi @pk2k14

The connector uses internal storage accounts of Kusto (managed by Kusto) to be used during ingestion.

To find the storage accounts out connect to the Kusto ingest endpoint on Kusto explorer

Execute : .show ingestion resources

You will get the below set of storage accounts being used

Whitelist traffic to these storage accounts , without these ingestion will not work.

Query with ForceSingleMode will work, if you use the readMode as ForceDistributedMode, that will fail as well (as it accesses storage that is blocked)

from azure-kusto-spark.

Assigning contributor access on the ADX should enable it to write to underlying storage as well, correct?

from azure-kusto-spark.

@pk2k14 the question is not permissions, it is the route to that host (network).

I think there is a seperate thread you are following up with MS in a seperate channel as well ? Would you like to continue there ? ramacg is my handle at MS

from azure-kusto-spark.

Yeah sure, we can discuss it over there.

from azure-kusto-spark.