DataServiceException: IOError when trying to retrieve CloudInfo about azure-kusto-spark HOT 4 CLOSED

Hello @Sri-Bhashini , Raises the suspicion that it may be not related to connectivity of the connector and may be something specific to LA (query limits). We'd check this out and see if we can replicate

from azure-kusto-spark.

It is working fine when I limit to 5000 rows. However, for a kusto cluster, the connector is able to retrieve large volume of data. Only log analytics is throwing this issue. Is it limited to 5000 only rows?

from azure-kusto-spark.

Are there any limits to the LogAnalytics workspaces. As I understand, the connector can connect and retrieve data, but fails if there are more than 5000 rows. Is this a right statement ?

The limits are 500k rows or 104MB of data for query API on LA/AzureMonitor. Could we perhaps try and retrieve a handful of columns to limit size of data and may be start investigating from there ?

https://learn.microsoft.com/en-us/azure/azure-monitor/service-limits

from azure-kusto-spark.

Yes, that's the right statement. I have tried retrieving only 1 column and also tried bringing all columns. In both the cases, its allowing only 5000 rows. I dont think it is related to size.

from azure-kusto-spark.