Blog post proposal: repartition Delta Table (with generated partition column) about website HOT 5 CLOSED

@keen85 - yea, sounds like an interesting post, but think Liquid is the real solution here. Did you try Liquid?

from website.

@MrPowers yes, Liquid Clustering will be the better solution, making things much easier in future.
But in Delta OSS 3.1 Liquid it is still "in preview"; I think it will be "production ready" in Delta 3.2 (delta-io/delta#1874).

I'm on Azure Synapse / Microsoft Fabric; stuck with Delta 2.2 (soon 2.4) for some more time 😕.

So, I think a blog post might be helpful nonetheless for Delta users who still use hive-style partitioning and who face the same issue in production now.

from website.

@MrPowers this is my first draft for the blog post:
https://github.com/keen85/delta-io-website/blob/blog/387_partition_evolution/src/blog/2024-04-xx-partition-evolution/index.mdx

Should I create a pull request so you can add some comments?

from website.

@keen85 - draft looks good.

Would you be OK just adding those sections to this post on the pros & cons of Hive-style partitioning?

Would be happy to add you as a co-author of course!

from website.

@MrPowers, sure; I'll try to incorporate it there.

from website.