from @jessegrosjean in Discord chat:

since ObjIds are not typed, it might make API easier to read if actions are named based on type they can happen to. Right now some are, some are not. And it gets a little confusing since you have to switch on prop for some to determine what kind of object you are working with. So maybe more consistent would be action names:

• MapPut
• MapDelete
• MapConflict
• ListSplice
• ListPut
• ListConflict
• TextSplice
• TextMarks
• CounterIncrement

Update the example in the docs to include how to add the dependency to a local project, both an Xcode project and a standard Swift package.

As done in automerge-swift, the binary can be packaged and deployed into a separate GitHub repo, which helps to reduce the size of this repository

I saw that Automerge-rs released an updated 0.4.0 crate, and went and took a quick stab at updating this repo to use it.

Unfortunately, with the version update there's a few breaking changes to the internal pieces that enable the UniFFI overlay. When I updated to 0.4.0, I received the following errors when invoking cargo build:

error[E0050]: method `insert` has 5 parameters but the declaration in trait `automerge::OpObserver::insert` has 6
  --> src/patches/observer.rs:90:9
   |
90 | /         &mut self,
91 | |         doc: &R,
92 | |         obj: am::ObjId,
93 | |         index: usize,
94 | |         tagged_value: (am::Value<'_>, am::ObjId),
   | |________________________________________________^ expected 6 parameters, found 5
   |
   = note: `insert` from trait: `fn(&mut Self, &R, automerge::ObjId, usize, (automerge::Value<'_>, automerge::ObjId), bool)`

error[E0046]: not all trait items implemented, missing: `mark`, `unmark`
  --> src/patches/observer.rs:88:1
   |
88 | impl am::OpObserver for Observer {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `mark`, `unmark` in implementation
   |
   = help: implement the missing item: `fn mark<R, M>(&mut self, _: &'a R, _: automerge::ObjId, _: M) where R: ReadDoc, M: Iterator, std::iter::Iterator::Item = automerge::marks::Mark<'a> { todo!() }`
   = help: implement the missing item: `fn unmark<R>(&mut self, _: &R, _: automerge::ObjId, _: &str, _: usize, _: usize) where R: ReadDoc { todo!() }`

Since I'm only minimally proficient at Rust, I thought I'd wait until @alexjg was available to help make this migration, as I'm uncertain of the implications as well as implementation changes needed.

Using the log-enables version has a default log levels that's shouting into my app debug logs. The library doesn't set a default level, which it needs to do - and ideally can be configurable so that the additional debugging CAN be enabled, but isn't in default library usage.

Working with demo app code, and on decode of existing files I'm getting the following generic error on file load

error: The data couldn’t be read because it is missing.

Adding more detailed logging, it appears as though a key is missing from the Automerge doc:

Key 'CodingKeys(stringValue: "discussion", intValue: nil)' not found: No value associated with key CodingKeys(stringValue: "discussion", intValue: nil) ("discussion").
codingPath: [CodingKeys(stringValue: "agendas", intValue: nil), [0]]

And dumping the raw automerge content using a CLI tool shows that is indeed the case:

{
  "agendas" : {
    [
      "id" :String(DD1E2812-0280-48CF-9FBD-12B6124E40B1)
      "title" :String(agenda item 1)
    ]
  }
  "attendees" : {
    []
  }
  "title" :String(Example Meeting ...)
}

The "agendas" should each have an id (UUID), title (String), and discussion (AutomergeText) - but the text content isn't included within the encoded results, and what had been there previously was removed/the file corrupted for decoding with this model.

Core library 0.5.6 added a Text API update_text - but it's not yet exposed through the Swift bindings. While swift can use the diffing mechanisms built into the standard library (CollectionDifference in updateText for example), the low-level API should be exposed for completeness.

The update with #54 causes a rather unfortunate side effect with the AutomergeEncoder method encode<T: Encodable>(_ value: T, at path: [CodingKey]). The goal of this particular API is to allow a far more targeted encoding of values into an Automerge document, but the new cleanup removes keys and array elements around the pathing that isn't written.

The implementation behind this API should be updated to only clean up the portion of the Automerge document that matches elements from the path provided and lower within the document.

When an Automerge doc using the AutomergeEncoding updates a counter, it's setting a value explicitly rather than incremented (or decrementing) by a value, resulting in updates between documents acting akin to "LWW Ints" rather than combining the increment values from the base value set.

Ultimately, Counter ergonomics would benefit from Counter becoming a reference type instead of a value type, but at the base level this is a bug with the 0.3.x release branch, and the reference type updates (#63) should only apply to later updates.

Sorting through an issue I had using AutomergeRepo and Meetingnotes, I found that while direct changes to AutomergeText were reflected, there isn't a path to signal when an underlying Document changes. With this missing, the internal content of a document could update but never be reflected if, for example, you had a SwiftUI view with just an instance of AutomergeText.

Splice is very useful to build all kinds of functionality (append, remove, ...). How would I splice elements which are objects/codable objects.

Right now Automerge.Document uses this pattern for modifications:

try sync {
    try self.doc.wrapErrors {
        sendObjectWillChange()
        ...
    }
}

The problem is this means there's no way to read document state from the objectWillChange callback (unless you debounce, but then you will be getting post change state), since reading also requires access to the same queue. So for example there's no way to record the heads() before the document changes.

This will fail waiting for queue:

automergeDoc.objectWillChange.sink {
    automergeDoc.heads()
}

Does it make sense to move sendObjectWillChange outside the sync block?

For docs (Rust) ref: https://automerge.org/automerge/automerge/struct.Automerge.html#method.get_marks

Get marks at a given position or Cursor should allow us to retrieve the markers without traversing all the ops of a document.

Digging in with ludicrous tracing enabled, I found that in a number of places there are change notifications sent at ANY time that call is made that could change the document. In the case where the value that's set is identical, there shouldn't be a notification, as the document treats that as a no-op.

Baseline: the notifications should only be emitted when the set of changesets returned from heads() is actually different.

With #150, AutomergeText and Counter (as ObservableObjects) reflect a signal that "something's changed" when ANY change happens to a document. Perhaps with #148 in place, update that process (and assocaited tests) so that the their "objectWillChange" signals are only triggered when the content they point to has changed.

The easiest cut would be to add additional tracking in AutomergeText to hold a "current value hash" and compare when the Doc.objectWillChange() comes in, potentially reflecting updated content in the Automerge document.

A more optimal solution would be to leverage the diff API (#148) and have the instances watch for only changes of their objId, which would potentially mean a different Combine published exposed that provides more granular data than objectWillChange(), ideally with information about what changes parsed from the patches returned by the diff API.

There's variations on this that might include instances of AutomergeText and Counter registering a callback function that a Document could be responsible for triggering, which would move the logic for this filtering and triggering from AutomergeText and Counter into Document itself, which could be notably more efficient in the scenario of lots of these instances, a deep document, and sparse change updates via merge, sync, etc.

Found while working the demo app. Shrinking an array and then "encoding" that array doesn't appear to be trimming the last values as expected. Instead, the encoding logic appears to be extending the array. The sequence of encode -> update -> encode isn't resulting in consistent results within the Automerge document.

Often I have a pattern like this:

let changes = try document.encodeChangesSince(heads: lastSavedHeads)
lastSavedHeads = document.heads()

But I think it's possible that the document will be modified between those two calls, so the heads that I'm saving might be newer than the ones used to encode the changes. And data will be lost.

To solve I think both document.save and document.encodeChangesSince should return both the data and the heads used to create that data. I guess something like this:

struct Changes {
    from: Set<ChangeHash>?
    to: Set<ChangeHash>
    data: Data
}

I'm not sure if from is strictly needed in API, but maybe convenient to include? Anyway to is what is needed I think.

I'm currently building an app based on Automerge and I'm using this Package for my iOS app. It has been working really well so far.

Now I'm about to start developing the Watch App counterpart for my app and it seems that automergeFFI.xcframework is not yet built for watchOS and watchOS simulator.

I don't know the Rust ecosystem well, but it seems that watchOS is supported as compile target: https://doc.rust-lang.org/rustc/platform-support/apple-watchos.html

Would it be possible to also compile the framework for these platforms?

Actor ID is missing a public init. I would like to init the Actor with a given custom string

In establishing the signals to notifiy instances of the type AutomergeText and Counter of changes from an Automerge document update, those classes are no exhibiting some Sendable warnings - in particular, referencing the class from within a Task. These need to get resolved.

In addition to objectWillChange I often feel myself wanting an objectDidChange event. I know SwiftUI doesn't need it, but generally didChange events seems like the useful default. Does it make sense to add this event to doc. And also fire it outside the sync block so that document state can be read immediately when receiving the event?

AutomergeText is meant to work with the encoding/decoding setup, but that implies a heavier document structure than may be warranted in some cases. It would be nice to have an option to set up an AutomergeText instance on a document where the creation of the AutomergeText instance would also immediately create the relevant structure in the document, if the structure wasn't already in place. If the document already has structure where AutomergeText is attempting to be created, it should throw an error or fail initialization - haven't thought throw the call site usage on that point.

Currently you need to:

1. create the Automerge document
2. establish the schema into the document (either directly, or by encoding a structure into it)
3. then establish AutomergeText so that it binds to the schema you created.

Even more optimized might be creating a document as well as the structure, so that you've got a one-shot setup for a document with a single text instance for collaborating editing of that text value.

As far as I know, every change encodes a wall clock time stamp. Is this already exposed?

While exploring a possible "view history" feature of an example app, I realized that uniffi heads() returns a list, but in converting it to a set - we loose all the ordering of that history to show a sequence of changes.

I believe a better result is either a straight up Array, preserving the ordering from the underlying library - or potentially bringing on Swift Collections OrderedSet to preserve the ordering from the underlying library.

The Text type, a Swift placeholder type that encapsulates the notion of the collaborative .Text object within Automerge, is poorly named being both a generic name and a type in extremely common use in SwiftUI. (The Text type is intended to vend both String and AttributedString from the underlying Automerge object representation with the upcoming fuller Pretext support.)

While putting together a sample application, I've had to reference the type directly a couple of times - in SwiftUI views no less - which resulted in some really annoying syntax juggling to make sure the compiler knew which Text type I was referring to. It's a breaking change in this API, but it would be a LOT more sane to have a different name for this type.

Variations for names that I've noodled:

• MultiText
• CollaborativeString
• SyncString
• SyncText
• MergeString
• AutomergeString

The type needs a different, non-conflicting name - and the documentation needs relevant updates, including example usage.

When updating documentation with #81, the post-merge build reported a failure on tests.

The relevant failure:

Test Case '-[AutomergeTests.PatchesTestCase testReceiveSyncMessageWithPatches]' started.
/Users/runner/work/automerge-swift/automerge-swift/Tests/AutomergeTests/TestPatches.swift:43: error: -[AutomergeTests.PatchesTestCase testReceiveSyncMessageWithPatches] : XCTAssertEqual failed: ("[]") is not equal to ("[Automerge.Patch(action: Automerge.PatchAction.Put(ObjId.ROOT, Automerge.Prop.Key("key2"), ScalarValue<String(value2)>), path: [])]")

I verified locally again that this test wasn't causing a failure with the current main branch, but I think it's worth digging deeper to see if this is a flaky test or perhaps some other issue (my system is more updated than CI, so there could be a lingering issue there?)

For the purposes of syncing, it would be immensely useful to be able to get notifications from a Document instance when it was updated. For the simplest case, just a signal that "something changed" so that any observer can react - sync, inspect, gossip, etc.

Add ObservableObject conformance to the base Document type.

For "bound" types - Counter and AutomergeText, only the AutomergeText initializer that takes a "path" will create the schema (if it doesn't exist and doesn't conflict). That logic is a bit of mess, and really needs to be extracted, and a similiar mechanism used for Counter - which still requires schema in an Automerge Document to be created before invoking bind() on it.

Wire through the the diff API sets through the UniFFI and Rust library code to expose that API into the Swift library.

Upstream docs for Rust: https://docs.rs/automerge/latest/automerge/struct.AutoCommit.html#method.diff

• diff_cursor
• diff
• diff_incremental

After a bit of holiday, I came back to the codebase and noted that clean builds of the XCFramework were failing. Multiple pieces appear to have shifted forward to the point that they're incompatible with Rust prior to version 1.70. In #18, @munhitsu pinned Rust to version 1.69 and a specific nightly version to enable Catalyst support, and now it looks like we might need to review how that all goes together.

#75 is a related PR that was updating to the latest Automerge crate, which also experienced a variation of this, but the issue also happens when just working from the main branch of the repo and invoking ./scripts/build-xcframework.sh

Update the release process (and hosting) to build the WebAssembly archive of libuniffi_automerge.a and store it alongside the XCFramework binary target.

cargo build --manifest-path $AUTOMERGE_SWIFT_SOURCE_DIR/rust/Cargo.toml --target wasm32-wasi --release

example repo: https://github.com/kateinoigakukun/automerge-swift-wasm/tree/main

Follow up to #92

The old automerge-swift stored Timestamp in milliseconds (not sure why), resulting in wrong resulting Dates when importing it with the new automerge-swift. Any idea we can mitigate this?

I think we could deprecate automerge-swift and replace it with automerge-swifter. It guess automerge-swift is the better name.

Following up from initial cursor support in #69:

• concept of a a default cursor for tracking AutomergeText
• worth noting that UITextField thinks in terms of NSRange. Potentially track this with two cursors - start and end - to allow range to expand/shrink with applied updates.
• look at spliceText (in the Rust layer/swift interface) and consider allowing a cursor as an argument in addition to an index, for convenience/developer ergonomics.
• allow a swift-oriented initialized for a cursor on .Text that starts with a String.Index to set the position of the cursor (translate String.Index to UTF8-view position and retrieve cursor reference from that)

First, thanks again for all this work!

Second I'm going to dwell on the non-perfect parts of automerge-swift experience (for me), but generally it seems to do what I want and I plan to use it... (assuming I can ever actually progress my own app code far enough). I'm not sure that I've ever followed an open source projects development quite so closely. I love the core, and also can't wait for more packaged versions of the networking stuff. The local-first architecture enabled by automerge really excites me.

My general thought is that I wonder if automerge-swift is trying to do too much. When I first started looking at the package I was expecting expecting something along the lines of Document Data Model, but with a Swift API.

Then I started reading the Five Minute Quick Start and got very confused. It didn't look like JSON, there was this new concept of encoders. The code didn't fit the JSON automerge model that I was expecting. Also I wasn't sure what to do with AutomergeText. I wondered if maybe lists and maps weren't yet supported by Swift wrapper.

Once I realized that those things were all extras built over the Document everything got much easier to use and understand. I think these extras might be useful, but I feel they also distract from the core of what automerge does and how it can be most efficiently used.

Small issue, but I'm also unsure about adding Observable to Document. I understand it makes working with SwiftUI easier, but it also (to me) distracts from the core API. I'm not sure it can scale usefully if your document has many items, it seems to lead API in wrong direction. Adds a new concept that isn't really core to automerge. I think of automerge more like a magic data structure, that does diffs and patches, and application code is responsible for wrapping those changes up into events.

Should you actually change anything? Very possibly not :) Extracting package seems like a lot of work to me, and it doesn't really affect functionality. I would maybe add a somewhere early in the documentation (such as in Five Minute QuickStart) that there are two levels of API in the project... use Document directly for low level API.

What I'm really interested in now is the network stuff, so I hesitate to post any of this and possibly slow you down. :/

Thanks,
Jesse

In the setter for the Binding supplied by AutomergeText.textBinding():

updateText() calls sendObjectWillChange():

But no change notification is sent on the code path that updates _unboundStorage. This confused the heck out of me for a while, trying to figure out why the text binding just wasn't working in an ultra-simple test app (literally just a TextField and an AutomergeText is what I was using).

As illustrated in #123, encoding an empty dictionary at the top level of an Automerge doc throws an error rather than completing without effect.

The idea being ultimately replacing the DispatchQueue protection, and shifting to fully embracing swift's async/await structure. With Swift 6 (fully embracing this safety marker) coming this June (2024), we'll want to make sure at a minimum we pass all Sendable checking.

In addition, the more recent updates to UniFFI (2.6.1) have started to embrace coordinating async calls between Rust and Swift. We're not taking advantage of these yet, but it may prove useful down the road.

Since I think we'll want to maintain backward compatibility for a while, a parallel type to Document would make a lot of sense, and we can slowly shift/deprecate the current version as the experiments vet themselves out in practical (app) use.

After having worked with getting things operational with the demo application MeetingNotes, it looks like it would be notably useful to have built-in conformance for an Automerge.Document to the Transferrable protocol.

This is macOS 13+, ios16+ stuff for the library, but would be a big convenience for supporting future app interactions that move or share data, like the Share button, drag and drop, and copy and paste. For an overview of what Transferrable is about, see Core Transferable.

For docs (Rust) ref: https://docs.rs/automerge/latest/automerge/struct.Automerge.html#method.diff

The differ should allow us to compare two commits in a document's history. It is quite valuable for example to implement an in-house undo manager.

From #191 :

In the automerge crate we have a TryFrom<&[u8]> implementation for ChangeHash. This is the supported way of converting bytes into a ChangeHash and is where we would handle future inconsistencies in hash format (not that I expect any at this point). If possible I think it would be best to expose this method up through the Uniffi UDL rather than assuming that any 32 byte array is a valid change hash. (This also applies to the serialization step, except we're already using ChangeHash::as_bytes when we generate the bytes which we pass to the Swift side).

The core API:

• split_block
• join_block
• update_block
• spans
• updateSpans

For docs (Rust) ref: automerge/automerge@806ef43

The rust commit that added most of the core: automerge/automerge@1d987cd
The commit that exposed through to WASM: automerge/automerge@5068143
The commit that exposed through to JavaScript: automerge/automerge@e7b090a

The blocks API should allow us to provide mapping from some common JavaScript-based rich-text editors through to Swift's AttributedString type.