Journal event deletion behavior difference about akka.persistence.sql HOT 10 CLOSED

This would not be a problem if we remove the logical-delete and include-deleted flag from the HOCON settings. The fake entry would not be read during recovery and query operation because it is marked as deleted. It will break thing if users are actually reading the journal table directly though.

from akka.persistence.sql.

cc @to11mtm

from akka.persistence.sql.

@Aaronontheweb So this is one of the many vaunted compatibility mode switches 😅

• The way JVM/Linq2Db do things is better because it minimizes the total number of pages written on commit.
  • Also, it's one less page being read on read.
• FWIW, Persistence.Linq2db -should- be able to handle both.
  • but to be clear it's kinda gnarly code, and there may be some edge cases around 'converting' to the JVM method.

I can say from experience that Linq2Db/JVM's approach handles frequent deletes -much- better than Sql.Common's JournalMetadata paradigm on MSSQL.

https://github.com/akkadotnet/Akka.Persistence.Linq2Db/blob/dev/src/Akka.Persistence.Sql.Linq2Db/persistence.conf#L23 <-- Switch

https://github.com/akkadotnet/Akka.Persistence.Linq2Db/blob/dev/src/Akka.Persistence.Sql.Linq2Db/Journal/DAO/BaseByteArrayJournalDao.cs#L385 <-- Compat Mode Query

https://github.com/akkadotnet/Akka.Persistence.Linq2Db/blob/dev/src/Akka.Persistence.Sql.Linq2Db/Journal/DAO/BaseByteArrayJournalDao.cs#L373 <-- JVM-style Native Query

We also handle it on the actual Delete of course, but the logic is a bit uglier.

This may or may not be another migration step to consider, or making this compat flag ternary for better forward-compatibility.

from akka.persistence.sql.

Thanks @to11mtm - this is very helpful. This issue was mostly created as a discussion point so we can reason about the backwards compat specs and what to do going forward.

from akka.persistence.sql.

Thanks @to11mtm - this is very helpful. This issue was mostly created as a discussion point so we can reason about the backwards compat specs and what to do going forward.

For sure; it's also worthwhile to have some level of documentation (even if it's in this issue) around the overall rationale for the change.

Realistically, a 'seamless' migration would require at minimum a migration script; the 'tricky' thing is that we would need some 'fake' event to put into the journal tables based on migration.

I suppose the other option would be the aforementioned ternary; it would at least minimize pain on 'new' persistenceIds (as they would elide at least the writes to Metadata) but would still have a read check+union (although, if metadata is properly indexed, should be fast to see it's not there.)

from akka.persistence.sql.

This would not be a problem if we remove the logical-delete and include-deleted flag from the HOCON settings. The fake entry would not be read during recovery and query operation because it is marked as deleted. It will break thing if users are actually reading the journal table directly though.

Any reason we shouldn't do this @to11mtm ? I think this approach makes a lot of sense.

from akka.persistence.sql.

My take is that we'll be implementing both model, but the metadata table will only be supported on a legacy system?
We will need to clean up the HOCON configuration for linq2db, its a bit too convoluted right now, especially with the addition of TagTable

from akka.persistence.sql.

@Arkatufus was this resolved in yesterday's PR? #145

from akka.persistence.sql.

@Aaronontheweb @Arkatufus Can we close this?

from akka.persistence.sql.

@to11mtm yep!

from akka.persistence.sql.