At present, if you supply an unfold function, for each write, its output:
a) travels on the wire to the stored procedure (this does not cost extra, so optimizing that is not really a concern)
b) replaces the current unfolds in the Tip document (as usual, you pay for a read with the original size and a write with the new size (not forgetting the cost of the read per active ChangeFeedProcessor) in terms of Request Charges)

Pros and cons:

• if you stop writing an unfold, it will get removed on the next write, saving storage space, and RUs on the write, subsequent reads, and induced reads by ChangeFeedProcessor(s)

• if you are doing blue/green deploys, old and new writers can interleave - the loser needs to rebuild their state unless the competing writer happens to also have written the same one (and then potentially remove the competitor's one iff it writes)

A small change to the JS can improve the semantics by
a) removing unfolds that are being updated and replacing with the supplied values
b) retaining any that are not being touched (perhaps subject to some retention criteria, i.e. only keep it if it's <= N events behind Tip)
c) allowing the writer to indicate a set of caseNames that should be removed regardless of retention rules

Related: there's a tip-isa-batch branch which stores events in the Tip - the competing writers scenario's efficiency would be greatly improved by this (any conflict will yield the competing events cleanly, and any unfolds that are behind will typically see both their unfold and successor events from the single point-read roundtrip)

In the spirit of #62, this ticket tracks the considerations for implementing a storage adapter for Amazon DynamoDb; It absolutely does not represent a roadmapped task or anything that's an organic need relevant to a current usage scenario.

The scheme used by Equinox.Cosmos seems to map to the [new DynamoDb transactional API) (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/transaction-apis.html)

Not 100% about how best to manage OCC - the client Position tracks the etag in the Cosmos impl, and a detailed read and research of guarantees provided is obviously warranted. One thing that does seem clear is that it It seems the client token idempotency is definitely not relevant. Wild guess: Maintaining expectedVersion either inside Tip or as metadata within it and then doing the writes conditional on that should work.

One key difference is the 400K document size upper limit, which means:

• Unfolds can't practically be maintained directly in Tip - e.g. if you ever need to support a transition in a snapshot schema in a blue/green deploy scenario, that halves the 400K. This means that an equivalent of #61 would need to be implemented
• Likely one should explore re-introducing the [Tip isa Batch] semantics removed in #58 as that is in general an efficient scheme in terms of reducing document counts, contention and Request Capacity Init consumption Should follow the Equinox.CosmosStore v3 impl, which implements Tip isa Batch semantics
• The 400K may make #31 more relevant than it has proven for CosmosStore - client side mechanisms about managing how one is faring wrt the limit will be big consideration given its 3MB vs 400K - the former is effectively unlimited for most reasonable sets of events and/or unfolds one might be maintaining.

Wrt #19, @dongdongcai makes the point that the re-read of the conflicting data can be implemented neatly using CosmosDb intrinsic facilities without involving a backoff.

The unfolds mechanism, as documented in #50 provides the ability to efficiently stash relevant data facilitating reading of state based on a point read.

However, this is can be suboptimal in the cases of larger snapshot states:-

• the data gets compressed and travels each time
• writing/updating occasions read and write costs within the sync stored procedure each time
• competing writers pay all but the update case in the case of a re-run of the same transaction
• readers not interested in/unable to process a specific snapshot schema still have to pay the price to transport it
• each update goes to the changefeed triggering more read costs
• the size and cost of the snapshots is harder to separate out
• snapshots contribute to partition size limits

Thus it makes sense to do some, or all of the following in the case where there is a projector in play and there is an aux collection (in the case of multiple writers one might even consider making an aux collection even if you don't have a projector):

• maintain the snapshots in the aux collection, where one is not paying for changefeed re-reads of the same data
• reduce the write frequency for snapshots (when the write happens should be deterministic for competing writers)
• implement ability for stored proc to maintain events since snapshot in tip when not updating snapshot
• write to a nominated collection (typically aux), with a deterministic guid based on the case / event type tag of the union as a point write
• refer to the snapshot in the unfolds; automate loading of the referenced snapshot if picked by isOrigin
• expose the ability to customize the snapshot loading process (but batteries should be included)
  • (perhaps do a sample showing parallel reads of the Snapshots and the Tip?)

In order to simplify backend store migrations, we should default to rejecting messages over the limit of any commonly used backend in order to avoid necessitating e.g. requiring a DocDb-based backend to split messages.

The mechanism should likely enforce the rejection via an exception - applications will be required to ensure that e.g. compaction events guarantee to fit within the constraints implied by this limit (i.e. shedding or truncating of state should be managed at application level)

(ES and DocumentDb respectively have 4Mb and 2Mb message size limits; DynamoDB max item is 400KB.)

Via @sorinoboroceanu
Using the caching layer when each event represents a full set of state by using a window size of 1 and a tautologous compaction predicate (see https://github.com/jet/foldunk/blob/master/Samples/Store/Domain/ContactPreferences.fs) currently has suboptimal behavior if there has been >1 competing write -- uncached, it would read exactly one event (backwards) whereas in this impl, we read all the events to catch up, but will then only use one

As flagged by @jakzale, its critical that consumers be confronted early in the experience with decision making processes around caching and/or unfolds and/or out-of-collection snapshots early in the game.

🤔 we could simply make the caching argument not be optional, at least until #61 is fleshed out more

This also could benefit from more docs/articles and/or tutorials and/or samples ref #57

SqlStreamStore is a SQL-focused library addressing many of the concerns Equinox does, with support for SQL Server and Postgres. Given the common EventStore (and DDD-CQRS-ES slack!)-influenced designs, there's a good chance there'd be minimal work required on either side to add an Equinox.SqlStreamStore adapter

Doing this work would:

• improve the breed wrt generalizing the store interfaces in Equinox (SqlStreamStore has a wealth of experience behind it)
• provide an avenue for folks looking for a hosted solutions without having to concern themselves with request charges (likely giving up some perf, but the CLI benchmark facility will tell all)

Initial implementation should probably take a dependency on a Sql LocalDb, but there's an obvious need for it to work well with Azure SQL (the CLI would likely manage the provisioning etc. see #59)

#19 improved the correctness of conflict handling wrt ESDB when using a twin connection, by reading from a Leader connection after a conflict has been flagged in the course of a write.

Fixing this in full in Equinox as a whole means:

• Ensuring we have a solution for Cosmos (see #21)
• ensuring there's a good answer for DynamoDB (there is as of June 2023, but the impl does not yet lean on it, see https://stackoverflow.com/a/76609239/11635)
• Having an optimal solution for EventStore (see EventStore/EventStore#1652)

We use minver, which derives versions from tags
When a build is triggered (based on a refs/branches/X or refs/tags/X), the newest tag encountered in that branch dictates the name
As long as you push the tag first, a random PR or branch CI build will pick up the correct version
Before releasing, sanity check the artifacts includes nupkg files with the release numbers you anticipated
the Release button (if you are in the relevant Pipelines org), then sends those artifacts to nuget.org
After that, it should show on the nuget landing page per artifact
subsequent to this (you'll see a message on the package nuget page before its indexed), it'll be indexed in 3-30 mins
when the indexing has completed, nuget update checks will pick up the new version

once a tag has been pushed, from which the build will pick up the version, the github Releases tab is used to add release notes (no automation at present, would be delighted to have a PR with support for some as long as it does not pull in a boatload of dependencies)

Nuget conventions/tricks/hacks esp when adding new packages:

• packages should be owned by jet.com, but for safety the pipelines api keys for a release pipeline should only allow uploads of new versions of existing packages, thus the process for when a new package is added is:
  0. never try to upload if there's a new package in the artifacts as it'll mean a partial upload with associated mess (and no clean way to remediate aside from doing manual uploads of all packages after the first one in the upload order)
  1. manual upload using your mouse on nuget.org
  2. invite jet.com as owner
  3. when accepted, run the release with a new version (noting step 0)
  4. remove yourself as the owner of the new packages

While the CLI needs to remain runnable from full FW in order to be able to run benchmarks in full framework (so we can't just go in and change the CLI to [only] be a dotnet tool), the provisioning and running facilities are environment independent. We should thus wra the CLI as a global tool (and make that be available via nuget)

Ensure we have tests for batch capacities

As part of an ETL process (see eqxetl template in https://github.com/jet/dotnet-templates), it's necessary to be able to propagate the correct initial creation timestamp of the event such that logic depending on it's correctness can be ported to Equinox.Cosmos

It should be possible to apply this to Equinox.EventStore too

The name DeprecatedRawName was chosen in haste; I was seeking to:
a) give a pejorative name to make people think before putting string concats composing a name into their app logic - AggregateId provides a clean way to compose a category name literal with an identifier in code
b) give a strong nudge to use AggregateId as it ensures that the out-of-the-box default separator used by EventStore ("-") gets used which makes $ec- streams etc work
c) provides scope to map AggregateId to something appropriate for any new store

On reflection, while there's nothing wrong with the above opinionated stance, the reality is that plenty existing codebases already compose names in a variety of ways, and will pick a backend and stick with it.

So, for the final v2 release, I'm open to a name change if we can settle on one. Open to any thoughts...

• needs to log WRN if not being applied immediately

There are various scenarios where it makes sense to allocate RUs across collections when managing notifications via the ChangeFeedProcessor

Relevant Docs:

• https://docs.microsoft.com/en-us/azure/cosmos-db/set-throughput
• https://azure.microsoft.com/en-us/blog/sharing-provisioned-throughput-across-multiple-containers-in-azure-cosmosdb/

The base implementation can map to a catchall case, but it does not yet support indicating what the unknown state was (for diagnostic purposes).

It should be possible to capture the discriminator field and perhaps trap the entire payload as a JObject or simila to facilitate dynamic handling

Current behavior is to attempt a re-read without a backoff delay; in prod this is regularly failing despite 3 retries (it's a short stream and the typical timespan is <10ms so the conflict never resolves and the app is yielding excessive 500s as a result).

Considerations:

• Ideally the API will accommodate varying the backoff per transaction (similarly to how the current attempts count is per-transaction not connection-wide)
• If there is a way to request a read from the master node (perhaps after the first retry?), we should do that (a secondary connection might be overkill)

See EventStore/EventStore#1626

While the normal Stream API short circuits on writing zero events, the lower level Equinox.Comsoms.Core.Events APIs do not, which leads to a confusing error message.

This should be addressed by throwing an exception at an appropriate point in proceedings.

Also ensure acceptance test does not do duplicate any content

When one has just added allocated a new Guid-based index value, you can be confident the child stream is empty and hence it makes sense to attempt writing with an expected version of -1 (in GES terms).

Atm, the implementation does not afford a way to singnal "it's empty, no read to read it, when writing, assume its empty from a consistency control perspective"

Will likely defer addressing this until further generalizations to the interface contracts to facilitate store switching techniques are completed.

Easier said that done as EventStore.ClientAPI.NetCore is very far behind - need multitargeting to have a workable solution

For this repo, a central tenet has always been ./build should produce a clean build with no warnings

Atm it says:
warning FS0988: Main module of program is empty: nothing will happen when it is run

I'm OK with it remaining in the short term if there is a tooling fix in the works

The Azure portal, MS docs and V3 SDK all use Container in preference to Collection consistently.

Based on this, I feel all Cosmos related APIs should reflect that - this would affect things with Collection in the name.

I'm also thinking that the EQUINOX_COSMOS_COLLECTION env var used by eqx and dotenet-templates should also be changed at this time.

See changes queued in #144

NB this is long and needs lots of editing.

Storage model (see source)

Batches

Events are stored in immutable batches consisting of:

• partitionKey: string // stream identifier
• index: int64 // base index of this batch (0 for first event in a stream)
• id: string // same as i (CosmosDb forces every doc to have one, and it must be a string)
• events: Event[] // (see next section) typically there is one item in the array (can be many if events are small, for RU and perf-efficiency reasons)
• ts // CosmosDb-intrinsic last updated date for this record (changes when replicated etc, hence see t below)

Events

Per Event, we have the following:

• case - the case of this union in the Discriminated Union of events this stream bears (aka Event Type)
• data - json data (CosmosDb maintains it as actual json; it can be indexed and queried if desired)
• metadata - carries ancillary information for an event
• t - creation timestamp

Tip Batch

The tip is readable via a point-read, as the id has a fixed known value (-1). It uses the same base layout as an Event-Batch, but adds the following:

• _etag: CosmosDb-managed field updated per-touch (facilitates NotModified result, see below)
• id: always -1 so one can reference it in a point-read GET request and not pay the cost and latency associated with a full query
• u: Array of _unfold_ed events based on a point-in-time state (see State, Snapshots, Events and Unfolds, Unfolded Events and unfold in the programming model section)

State, Snapshots, Events and Unfolds

In an Event Sourced system, we typically distinguish between the following basic elements

• Events - Domain Events representing actual real world events that have occurred, reflecting the domain as understood by domain experts - see Event Storming. The customer favorited the item, the customer saved SKU Y for later, $200 got charged with transaction id X.

• State - derived representations established from Events. A given set of code in an environment will, in service of some decision making process, interpret those events as implying a particular state in a model. If we change the code slightly or add a field, you wouldn't necessarily expect a version of your code from a year ago to generate you equivalent state that you can simply blast into your object model and go. (But you could easily hold a copy in memory as long as your process runs)

• Snapshots - A snapshot is an intentionally roundtrippable version of a State, which can be saved and restored. Typically one would do this to save the cost of loading all the Events in a long running sequence of Events to re-establish the State. The EventStore folks have a great walkthrough on Rolling Snapshots.

• Projections - the term projection is heavily overloaded, meaning anything from the proceeds of a SELECT statement, the result of a map operation, an EventStore projection, an event being propagated via Kafka (no, further examples are not required).

.... and:

• Unfolds - the term unfold is based on the FP function of that name, bearing the signature 'state -> 'event seq. When using Equinox.Cosmos, the unfold produces projections, represented as _event_s to snapshot the state at a position in the stream.

Generating and saving unfolded events

Periodically, along with writing the events that a decision function yields to represent the implications of a command given the present state, we also unfold the resulting state' and supply those to the sync function too. The unfold function takes the state and projects one or more snapshot-events which can be used to reestablish the same state we have thus far derived from watching the events on the stream. Unlike normal events, unfolded events do not get replicated to other systems, and can also be thrown away at will (we also compress them rather than storing them as fully expanded json).

Reading from the Storage Model

Most reads request tip with anIfNoneMatch precondition citing the `etag it bore when we last saw it, which, when combined with a cache means one of the following happens when a reader is trying to establish the state of a stream prior to processing a Command:

• NotModified (depending on workload, can be the dominant case) - for 1 RU, minimal latency and close-to-0 network bandwidth, we know the present state
• NotFound (there's nothing in the stream) - for equivalently low cost, we know the state is initial
• Found - (if there are multiple writers and/or we don't have a cached version) - for the minimal possible cost (a point read, not a query), we have all we need to establish the state:-
  i: a version number
  e: events since that version number
  u: unfolded auxiliary events computed at the same time as the batch of events was sent (aka projections/snapshots) - (these enable us to establish the state without further queries or roundtrips to load and fold all preceding events)

Building a state from the Storage Model and/or the Cache

Given a stream with:

{ id:0, i:0, e: [{c:c1, d:d1}]},
{ id:1, i:1, e: [{c:c2, d:d2}]}, 
{ id:2, i:2, e: [{c:c2, d:d3}]}, 
{ id:3, i:3, e: [{c:c1, d:d4}]}, 
{ id:-1,
  i:4,
  e: [{i:4, c:c3, d:d5}],
  u: [{i:4, c:s1, d:s5Compressed}, {i:3, c:s2, d:s4Compressed}],
  _etag: "etagXYZ"
}  

If we have state4 based on the events up to {i:3, c:c1, d: d4} and the index document, we can produce the state by folding in a variety of ways:

• fold initial [ C1 d1; C2 d2; C3 d3; C1 d4; C3 d5 ] (but would need a query to load the first 2 batches, with associated RUs and roundtrips)
• fold state4 [ C3 d5 ] (only need to pay to transport the tip document as a point read)
• (if isStart (S1 s5) = true): fold initial [S1 s5] (point read + transport + decompress s5)
• (if isStart (S2 s4) = true): fold initial [S2 s4; C3 d5] (only need to pay to transport the tip document as a point read and decompress s4 and s5)

If we have state3 based on the events up to {i:3, c:c1, d: d4}, we can produce the state by folding in a variety of ways:

• fold initial [ C1 d1; C2 d2; C3 d3; C1 d4; C3 d5 ] (but query, roundtrips)
• fold state3 [C1 d4 C3 d5] (only pay for point read+transport)
• fold initial [S2 s4; C3 d5] (only pay for point read+transport)
• (if isStart (S1 s5) = true): fold initial [S1 s5] (point read + transport + decompress s5)
• (if isStart (S2 s4) = true): fold initial [S2 s4; C3 d5] (only need to pay to transport the tip document as a point read and decompress s4 and s5)

If we have state5 based on the events up to C3 d5, and (being the writer, or a recent reader), have the etag: etagXYZ, we can do a HTTP GET with etag: IfNoneMatch etagXYZ, which will return 302 Not Modified with < 1K of data, and a charge of 1.00 RU allowing us to derive the state as:

• state5

Programming model

In F#, the Equinox programming model involves, per aggregation of events on a given category of stream:

• 'state: the state required to support the decision or query being supported (not serializable or stored; can be held in a .NET MemoryCache)
• initial: 'state: the implied state of an empty stream
• 'event: a discriminated union representing all the possible Events from which a state be evolved (see e and u in the data model). Typically the mapping of the json to an 'event case is driven by a UnionContractEncoder
• fold : 'state -> 'event seq -> 'state: function used to fold events (real ones and/or unfolded ones) into the running 'state
• evolve: state -> 'event -> 'state - the folder function from which fold is built, representing the application of the delta the 'event implies for the model to the state
• decide: 'state -> 'command -> event' list: responsible for (in an idempotent manner) interpreting a command in the context of a state as the events that should be written to the stream to record the decision

When using the Equinox.Cosmos adapter, one will typically implement two further functions in order to avoid having to have every 'event in the stream having to be loaded and processed in order to build the 'state (versus a single cheap point read from CosmosDb to read the tip):

• unfold: 'state -> 'event seq: function used to render events representing the state which facilitate quickly re-establishing a state without needing to go back to the first event that occurred on a stream
• isStart: 'event -> bool: predicate indicating whether a given 'event is sufficient as a starting point e.g.

High level Command Processing flow

When running a decision process, we thus have the following stages:

1. establish a known 'state (based on a given position in the stream of Events)
2. let the decide function look at the request/command and yield a set of events (or none) that represent the effect of that decision in terms of events
3. update the stream _contingent on the stream still being in the same State/Position it was in step 1
   3a. if there is no conflict (nobody else decided anything since we decided what we'd do), append the events to the stream (record the new position and etag)
   3b. if there is a conflict, take the conflicting events that other writers have produced since step 1, fold them into our state, and go back to 2 (the CosmosDb stored procedure sends them back immediately at zero cost or latency)
4. if it makes sense for our scenario, hold the state, position and etag in our cache. When a reader comes along, do a point-read of the tip and jump straight to step 2 if nothing has been modified.

Sync stored procedure high level flow (see source)

The sync stored procedure takes a document as input which is almost identical to the format of the tip batch (in fact, if the stream is found to be empty, it forms the template for the first document created in the stream). The request includes the following elements:

• expectedVesion: the position the requestor is basing their proposed batch of events on (no, an etag would not be relevant)
• e: array of Events (see Event, above) to append if the expectedVersion check is fulfilled
• u: array of unfolded events which supersede items with equivalent case values (aka snapshots, projectiosn)
• maxEvents: the maximum number of events to record in an individual batch. For example:
  • if e contains 2 events, the tip document's e has 2 documents and the maxEvents is 5, the events get merged into the tip
  • if maxEvents is 1, the tip gets frozen as a Batch, and the new request becomes the tip (as an atomic transaction on the server side)
• (PROPOSAL/FUTURE) thirdPartyUnfoldRetention: how many events to keep before the base (i) of the batch if required by lagging unfolds which would otherwise fall out of scope as a result of the appends in this batch (this will default to 0, so for example if a writer says maxEvents 10 and there is an unfold based on an event more than 10 old it will be removed as part of the appending process)

Example

The following example is a minimal version of the Favorites model, with shortcuts for brevity (yes, and imperfect performance characteristics):

(* Event schemas *)

type Item = { id: int; name: string; added: DateTimeOffset } 
type Event =
    | Added of Item
    | Removed of itemId
    | Compacted of items: Item[]

(* State types *)

type State = Item list

let contains state id = state |> List.exists (fun x -> x.id=id)

(* Folding functions to build state from events *)

let evolve state event =
    match event with
    | Compacted items -> List.ofArray items
    | Added item -> item :: state
    | Removed id -> List.filter (not (contains state id)) 
let fold state events = Seq.fold evolve state events 

(* Decision Processing *)

type Command =
    | Add item
    | Remove itemId: int

let decide command state =
    match command with
    | Add (id, name, date) ->
        if contains id then [] else [Added {id=id; name=name; date=date}]
    | Remove id -> 
        if contains id then [Removed id] else []

(* Equinox.Cosmos Unfold Functions to allow loading without queries *)

let unfold state =
    [Event.Compacted state]
let isOrigin = function
    | Compacted _ -> true
    | _ -> false

Atm, Todo.fs and Aggregate.fs are the only real examples (aside from the samples/ folder) of what one might do to connect the basic domain logic to a running app.

Documenting the sort of things you put into the Handler and/or Service and how to balance that is a key piece of documentation that's been pointed out as missing....

Patterns to be covered:

• Command with no response with straight call in Service (todo does this)
• Command with decision result from Interpret function feeding out (don't think that's covered either here or in the Equinox /samples folder)
• Command with projection from final folded result post command application (todo does this)
• Query (see todo and aggregate)

General advice on what to do that the tutorial should cover:

• there's definitely more - the tutorial should also mention some general concepts around CQRS and provide advice beyond "here are 11 techniques, you fail if you on't use at least 9 of them"
• address the key point is that one size does not fit all and its important that the Handler and Service be well thought through
  • if you end up cut and pasting exactly the same one as boilerplate per aggregate its a bad sign
  • if you end up using more than 2-3 of the techniques above, that's also a bad sign

Deferred:

• Command calling out to external decision process (e.g. if a decision process needs to examine the folded state, then propose a relevant command and/or have a longer chain of calls which you want to wrap an optimistic retry loop around)
  • There are Async overloads for prominent functions which result from specific cases in existing codebase. They will not be documented in #96 as they don't constitute best practice worth highlighting (@eiriktsarpalis argues for removal of such temptations from the house entirely; for now they remain)

Questions

Some of these have likely been answered already in the Readme, if so consider making these broad points more obvious. I read the Readme a few times but it was frequently challenging to tease out what was some very specific point, or a more broad objective.

• What kind of project is this for, are there other easier ways to build event sourcing for F#, or is this the easiest? (as in does the compatibility layer make things easier to use)
• What is the scope of Equinox, or what features have been consciously omitted?
• Broadly speaking what are the goals of using this instead of using one particular kind of EventStore database directly? Does it just allow me to swap out "event stores" or does it do more?
• Should I reach for Equinox first if I'm doing DDD + CQRS + ES, even if I may not need to change the event store database?
• Is this layer constructed for ease of use, or does it make things more challenging toward some end?
• Do you recommend this for the average person who is building an event sourcing project with F# or should I only use Equinox if I need massive scalability?
• What are some of the other good options for F# event sourcing, is this the only fleshed out option?
• How do I get started? (development vs production)
• How do I use Equinox? (development vs production)
• Is there an API documentation? Where can I find it?
• You say I can use volatile memory for integration tests, could this also be used for learning how to get started building event sourcing programs with equinox?
• Is there a guide to building the simplest possible hello world "counter" sample, that simply counts with an add and a subtract event?

At present, the OOTB codecs only present the data as UTF-8 to encode/tryDecode functions; need to extend to enable:

• generically wrapping events as an Envelope<T>
• combining UnionEncoder with grabbing context from the Event Data
• some degree of uniformity between consumption afforded by Equinox.Cosmos and Equinox.EventStore

Needs to consider #79 to some degree

🤔 Likely will involve introducing an IEvent into the codec module

cc @ameier38

While I elided #71 from master post-merge (it was only really half-working) in the interests of time, MinVer just makes sense given how we'll be managing all Releases based on tags too.

This will involve removing the powershell VersionSuffix logic and replacing it with MSBuild derivation using the Azure env vars as per the MinVer readme.

At present, master targets the full framework. The main blocker on migrating it to netstandard is that the caching impl in feature/ges-caching branch is implemented against the full framework `System.Runtime.Caching. The .NET Core / .NET Standard impl has a different interface AFAICT.

We've yet to put significant load through the current cache impl to base impl to make a call as to whether using a different impl is a loss or a gain.

If this can be resolved, a likely fast follow would be to retarget the Foldunk.EventStore in master against EventStore.Client.NetCore (EDIT: EventStore.Client.NetCore is not being maintained; current thinking is to target a multi-targetted version of the ES client)

Trying to append to non-initialized collection returns AppendResult.Conflict(0, [||]) which tells us that there are conflict on appending event with index=0 and there are no conflicts :)

We should add another AppendResult for that case.

At present, EventStore, MemoryStore and Cosmos all predicate the concurrency check on a stream index aka expectedVersion. This makes sense as it's a well proven methodology for OCC.

Equinox.Cosmos's unfolds (snapshots etc) mechanism allow one to combine the writing of a set of events with associated snapshot updates as an atomic upsert.

Normally, if no event is being written, it doesnt makes sense to write a snapshot update either.

For rolling updates where there isn't a useful event one would want to add each time (e.g. maintaining a checkpoint), you want to have
a) a way to check you're up to date and/or read the value a competing writer has applied
b) be able to do an OCC for the write to detect and/or react to conflicts
c) be able to efficiently avoid incurring a write cost in the case of an idempotent write
d) minimize the number of events written permanently (and hence docs ChangFeedProcessor sees (esp when running a replay))

How? at present we track the current version and etag per stream in the cache. The etag is used to make a cached read cheap. However, when writing, for consistency with the other stores, the "is a null change" predicate is based on the expected version. The aforementioned can be achieved by allowing a write to requested that is instead based on the etag not having changed.

An associated mechanism is having a way to write logic based on events, but allow a post processing mechanism to map what would ordinarily be an OCC event append to an etag-contingent snapshot update only, while allowing the logic to be written in the normal manner.

Example of a stream that would benefit from this: https://github.com/jet/propulsion/blob/master/src/Propulsion.EventStore/Checkpoint.fs#L40

Shall root namespace be jet.equinox to avoid potential collisions?

As has recently been applied to the EventStore logging, we should stop emitting syncAttempt: 1 in the logs on the [normal case] first attempt at the execution

Rendering a Meta field containing [||] yields invalid Json when rendered with JsonConverter.SerializeObject
(notably problematic when coupled with reading empty metatara arrays in jet/dotnet-templates#16)

Need to apply suggestions from @enricosada in #40 (comment)

Test to pin the nature of the re-read behavior and the idempotency of the retry

In the sample project we have SkuId as a value object implemented as a class hat inherits a Comparable

/// Endows any type that inherits this class with standard .NET comparison semantics using a supplied token identifier
[<AbstractClass>]
type Comparable<'TComp, 'Token when 'TComp :> Comparable<'TComp, 'Token> and 'Token : comparison>(token : 'Token) =
    member private __.Token = token // I can haz protected?
    override x.Equals y = match y with :? Comparable<'TComp, 'Token> as y -> x.Token = y.Token | _ -> false
    override __.GetHashCode() = hash token
    interface IComparable with
        member x.CompareTo y =
            match y with
            | :? Comparable<'TComp, 'Token> as y -> compare x.Token y.Token
            | _ -> invalidArg "y" "invalid comparand"

/// SkuId strongly typed id
[<Sealed; JsonConverter(typeof<SkuIdJsonConverter>); AutoSerializable(false); StructuredFormatDisplay("{Value}")>]
// (Internally a string for most efficient copying semantics)
type SkuId private (id : string) =
    inherit Comparable<SkuId, string>(id)
    [<IgnoreDataMember>] // Prevent swashbuckle inferring there's a "value" field
    member __.Value = id
    override __.ToString () = id
    new (guid: Guid) = SkuId (guid.ToString("N"))
    // NB tests (specifically, empty) lean on having a ctor of this shape
    new() = SkuId(Guid.NewGuid())
    // NB for validation [and XSS] purposes we prove it translatable to a Guid
    static member Parse(input: string) = SkuId (Guid.Parse input)
/// Represent as a Guid.ToString("N") output externally
and private SkuIdJsonConverter() =
    inherit JsonIsomorphism<SkuId, string>()
    /// Renders as per Guid.ToString("N")
    override __.Pickle value = value.Value
    /// Input must be a Guid.Parseable value
    override __.UnPickle input = SkuId.Parse input

Can this be simplified to a record type like this?

[<JsonConverter(typeof<SkuIdJsonConverter>)>]
type SkuId = private { [<IgnoreDataMember>] Value: Guid } with
    static member Create (value: string): SkuId =
        if (String.IsNullOrWhiteSpace(value)) then
            invalidArg "value" "cannot be whitespace"
        { SkuId.Value = Guid.Parse(value) }
type SkuIdJsonConverter() =
    inherit JsonIsomorphism<SkuId, string>()
    override __.Pickle value = value.Value.ToString("N")
    override __.UnPickle input = SkuId.Create(input)

• Redundant events read count/bytes
• Events not read count/bytes

The follow queries provide key store metrics which would be very useful to be able to provide in the eqx tool:

• event count: SELECT VALUE SUM(c.n) FROM c WHERE c.id="-1"
• stream count SELECT VALUE COUNT(1) FROM c WHERE c.id="-1" (will need to become a distinct count operation if/when #109 is implemented)
• document count SELECT VALUE COUNT(1) FROM c

Provisioning of the stored procedure is presently an explicitly separated act, driven by Equinox.Cli init, which fits in with DDL/DML separation concepts etc.

However, in practice
a) keys are often provisioned with both DML (manipulate data) and DDL (create proc) permissions
b) as stored procedures are revised, new editions may need to be added

For the above, reasons, the connector should gain a ?provisionStoredProcNow parameter that attempts to add if not present when requested

I'm trying to integrate Prometheus to instrument the events related to each of these retries.

Atm unions with string payloads don't correctly roundtrip in Equinox.Cosmos with the UnionEncoder.

Also need to validate round-tripping of tupled union cases and/or whether it's sensitive to the order of the [named] elements in the tuple (as https://github.com/jet/jet.jsonnet.converters isnt)

cc @jakzale

I've been researching the library and it looks really neat!

I've been wondering if there are plans to allow the customization if cache implementation for GES and CosmosDB providers.
At the moment it uses System.Runtime.Caching.MemoryCache instance.

Microsoft.Extensions.Caching provides multiple abstractions (IMemoryCache and IDistributedCache), and the possibility to customize the implementation seems like a valuable option. E.g. in our system MemoryCache from Microsoft.Extensions.Caching is used heavily, and having multiple caching providers doesn't make much sense.

• Should not delete the database
• should probably be [<Hidden>]
• maybe require a -Yes ?

There's a tip-isa-batch branch, which generalizes the storage scheme to provide even greater efficiency in terms of read and write costs.

This work was shelved for prioritisation purposes in order to avoid having to make projectors deal cleanly with seeing an event multiple times.

Given a projector that does some practical de-duplication of events, its pretty feasible to re-introduce this storage optimization; the performance and RU cost reductions can be inspected by using Equinox.Tool run and/or dotnet new eqxtestbed against said branch.

As noted in #108, this storage generalization also provides benefits wrt competing writer scenarios.

At present, Equinox.Cli init will create a collection and add a stored proc. For the aux collection used by Change Feed Processors, adding the stored procedure should be made optional as this is step is not necessary for the Aux collection

See also #59 - in some cases users may not be interested in adding the stored procedure at the point of provisioning the collection in any case.

See also #61 - snapshots have different (no?) indexing requirements

We'll probably also make ./build provision an aux