Unable to Write or Read (un-encrypted) data from Cosmos DB Containers created via Cosmos DB SDK Encryption about azure-cosmosdb-spark HOT 1 OPEN

Hello @ssharma444. You are correct that not all Cosmos DB features are supported in all Cosmos DB client libraries. In this case, demand for the feature was much higher in our SDKs than in our Spark Connector, where demand for this feature has been very low. With that said, this is still on our backlog, and currently our plan is to deliver encryption support for the Spark 3 Connector within the next 3-6 months.

Meanwhile, I want to point out that encryption support will not be delivered in the version of the Spark Connector that this repo relates to. This is the repo for the older Cosmos Spark Connector for Spark 2 (note: Spark 2 itself will soon be end of life on Databricks). This connector and is also based on deprecated versions of our Java SDK.

We strongly recommend upgrading to the Spark 3 OLTP Connector if possible, as this is where we will be adding encryption support. Feel free to raise this issue in the repo where our latest Spark Connector lives: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3-2_2-12. We will track that issue against our work items for completion of this feature.

from azure-cosmosdb-spark.