What is the difference between OT-json0 and OT-json1

doc

{
  startTime: 1
}

when A and B both submit op with replaceOp(['startTime'], 2) and replaceOp(['startTime'], 3) will conflict and throw error.

but can we add extra infomation to resolve this problem ? example we add a 'timestamp' or random number

op object

['startTime', { r: 2, weight: Date.now() }]
['startTime', { r: 3, weight: Date.now() }]

This can keeps the two operations consistent。we only simple compare weight to compose two operation.

Context:
I encode colors as strings in json. To ensure, they are always consisted 6 character colors, I decided to use {r: oldColor} + {i: newColor} instead of {es: newColor}. (To my understanding from reading the documentation, two simultaneous {es: newColor} operations could lead to a longer string e.g. FF0000444444, correct?)

Call:
json1.makeInvertible(["color", {r: true}, {i: "FF0000"}], {color: "00FF00"}])

Throws:
Error: Invalid JSON key

Expected result:
["abc", {r: "00FF00"}, {i: "FF0000"}], {color: "00FF00"}]

I saw this TODO in source code.

// My TODO list:
// - compose preserve invertable data
// - makeInvertable and invert()

is there any plan to implement the invert() method? Because I really need this to implement undo and redo functions in my editor.

Thanks for your great work.

I am working on adding collaboration to SlateJS.

It represents edits as a series of batched operations.

https://docs.slatejs.org/concepts/05-operations

What would be an efficient way to use json1 to add collab editing capability?

https://github.com/ianstormtaylor/slate/blob/master/packages/slate/src/interfaces/operation.ts

Do I have to manually map each operation to a json1 equivalent? Slate provides a document structure in the form of JSON but the ops are done through the editor, do I have to mirror the document structure in json1?

As a newbie to ottypes + sharedb, which library should be used json0 or json1(I have been working with json1)?

First, thanks for creating this library!

This isn't really an issue, more just a share of a utility I made to convert between json1 ops and JSON patch ops: https://gist.github.com/adorsk/67a27968aeb9cc534057c424ee39e63e .

Sharing it here in case other folks are hoping to use JSON patches with this library. I hope that this can help them get started :) .

i have found https://github.com/kbadk/json0-ot-diff ，it can be support json0，but i can not found the lib support for json1

Hi,

Again thanks for this great library.
While testing the library with composition and inserts, I stumbled on this behavior.

Is this intended?

json1.type.apply(
  [0],
  json1.type.compose(
    json1.insertOp([2], { key: "value" }),
    json1.type.compose(
      json1.insertOp([2, "anotherKey"], "anotherValue"),
      json1.insertOp([2, "lastKey"], "lastValue")
    )
  )
);
=> 
[
  0,
  { key: 'value' },
  { key: 'value', anotherKey: 'anotherValue', lastKey: 'lastValue' }
]

Having { key: 'value' } at position 1 is creating a lot of confusion.

Based on the discussion in this ShareDB issue it looks like it'd be useful to have an isNoop method in OT types.

JSON1 uses null as its noop sentinal value, which is fine and easy, but subtypes can also generate noops from transform and compose.

So:

• isNoop(null) returns true
• isNoop([{r:true}]) returns false

But what about:

• isNoop([{es:[]}])? [] is a no-op in the text-unicode type, so this operation has no effect. Should we detect this sort of thing in isNoop?
• normalize([{es:[]}])? Should this return [{es:[]}] or null?
• compose([{es:['hi']}], [{es:[{d:'hi'}]}])? These two embedded operations cancel each other out. Should we do the naive thing and return [{es:[]}] or flatten the result out to null?
• ... and the same in transform.

As I see it there's ways we could play this:

1. isNoop must scan the operation recursively, looking for an operation component which does something.
2. Embedded noops generated by subtypes are considered invalid. They get stripped in all methods that interact with operations. (So normalize([{es:[]}]) and all the other similar examples above return null)
3. We ignore embedded noops, and a JSON operation which only has an embedded noop is not considered a noop in turn. This is a little weird.

Its a bit obscure no matter which way we play this, and I can't think of any serious consequences of preserving embedded noops or not. There's a little bit of runtime type checking which comes from applying an embedded edit (even if its null); but I don't think it matters much. I can't think of any other way this will affect the output of transform, compose or apply.

I'm leaning toward (2) and stripping any embedded noops we can detect in all cases.

I've been following this project since last year and am really excited about the possibilities this brings to general purpose OT.

It seems that there hasn't been much activity on this and I was wondering if you are still working on it in private or if you have lost interest and moved on to other things?

Bad case:

const json1 = require('ot-json1')

const doc = [{
    text: '-'
}]

const op1 = json1.editOp([0, 'text'], 'text-unicode', [{ d: '-' }])
const op2 = json1.removeOp([0], { text: '' })

const op = json1.type.compose(op1, op2)

const result = json1.type.apply(doc, op)

console.log(result) // []

const invertedOp = json1.type.invert(op)

const result2 = json1.type.apply(result, invertedOp)

console.log(result2) // [ { text: '' } ]

Expect result2 equal to doc ? But they are not equal.

When I have 100 ot，need 700ms
1000 ot, 5000ms

how to fix it ?

Hi Joseph!

As always, thanks for this library. I'm having a bit of trouble knowing if I encountered a bug or I just use the library badly. What do you think about this example?

const doc = {
  object: 'one',
  nodes: [
    {
      object: 'two',
      nodes: [
        {
          object: 'three',
          nodes: [
            {
              object: 'four',
              nodes: [{ object: 'five' }],
            },
          ],
        },
      ],
    },
  ],
}

const op = [
  'nodes',
  0,
  { r: true, d: 0 },
  'nodes',
  [0, { p: 0, i: { object: 'six' } }],
  [2, { i: { object: 'seven' } }],
]

const appliedDoc = {
  "object": "one",
  "nodes": [
    {
      "object": "three",
      "nodes": [
        {
          "object": "six"
        },
        {
          "object": "four",
          "nodes": [
            {
              "object": "five"
            }
          ]
        },
        {
          "object": "seven"
        }
      ]
    }
  ]
}

const path = ['nodes', 0, 'nodes' ,0, 'nodes', 0, 'nodes', 0]

json1.type.transformPosition(path, op)
// => [ 'nodes', 0, 'nodes', 0, 'nodes', 1 ] which seems buggy

Thanks for your time.

It would be useful to have a UMD browser build for this library.

It might be as simple as adding a flag to tsc here https://github.com/ottypes/json1/blob/master/Makefile#L20

Perhaps tsc --module=umd? Though I'm not sure how to specify the name of the browser global introduced.

FWIW I gave the ShareDB Client a browser build treatment that looks like this: https://github.com/vizhub-core/sharedb-client-browser#readme

One option is to do something similar for JSON1, or even bundle JSON1 into that build, but it would be neat if JSON1 had its own browser build "out of the box".

Thanks!

Greetings,

I'm evaluating feasibility of migrating from JSON0 to JSON1 for VizHub, a browser based collaborative code editor. One feature of the JSON0 ecosystem that I have been relying on heavily is computing a diff. I have been using kbadk/json0-ot-diff for this.

I'm opening this issue to ask, has anyone using JSON1 had any success trying to compute diffs between documents and generating JSON1 ops as output?

To be clear, I'm not proposing to extend this JSON1 codebase to compute diffs, that can live externally to this library. Just probing the space to see if anyone else is doing this. I may try to modify json0-ot-diff to work with JSON1, but wanted to first see if anyone else has thought about this.

Related issue json0-ot-diff: JSON1 OT Diff.

Thanks!

I'm wondering, is the type invertable?

I see the following in

// - makeInvertable and invert()

What would be some first steps to implementing/verifying this? I suppose we could start with some failing tests.

Also the related to the ideas from the spec:

(quoting from there)

The JSON1 OT type is designed to be optionally invertible. There are no constraints on the content of r: components, so the contents of the removed elements can be placed there.

To implement this, we need a (unwritten) function which takes an operation and a document snapshot and augments all r:X components with the content to be removed.

Then an invert function can be written which:

• Swaps i and r components
• Swaps p and d components
• Move all edits to the item's source location
• Invert all embedded edits using the embedded type

In the current implementation the extra information in r:X components will be lost if the operation is transformed.

const op = [ json1.insertOp([], {title: '', contents: '', public: false}), json1.editOp(['title'], 'text-unicode', ['My cool blog entry']), json1.replaceOp(['public', false, true]) ].reduce(json1.type.compose, null)

Should be
const op = [ json1.insertOp([], {title: '', contents: '', public: false}), json1.editOp(['title'], 'text-unicode', ['My cool blog entry']), json1.replaceOp(['public'], false, true) ].reduce(json1.type.compose, null)

Hey! First of all, thanks for creating this awesome library!

I have a question about using this library for JSON data types that has to comply with a schema: on the server side, the server can reject changes that may break the schema, but what should the client do?

For example, if the schema imposes that an array can have at most 1 element. Both Alice and Bob now simultaneously insert an element to the array without knowing each other's change. The server may decide to reject Bob's change since Alice's request is handled first. On the client side, how should Bob handle server-sent update containing Alice's operation (when applied, will break the array length limit)? And if server rejected Bob's operation, should it send any operation regarding this rejection to Bob?

I'm about to embark on an integration of JSON1 with CodeMirror 6.

Has anyone else already done this? Is anyone else interested in this work?

https://github.com/kbadk/json0-ot-diff

^Can this feature be added to the json1 codebase?

When moving an array element into a subkey of a later element,
I believe the path could use adjusting when the picked up element is
on a previous index.

Illustration:
https://codesandbox.io/s/happy-moon-3b295?file=/src/index.js

This issue is about improving the approach taken in the tests for checking immutability.

A quote from the original discussion of this idea in #8:

A better way to verify immutability would be to recursively call Object.freeze on all inputs to all the functions in all tests. The nice thing about that approach is that if any tests inappropriately modify data, the stack trace will point to the exact source of the error. If we do that, we probably shouldn't need special immutability test cases at all. --@josephg

It seems that JSON1 and ShareDB will not play well together see here.

I don't know if we should rename invert to something else (which is a big breaking change) but I think I've reached a dead end.

What do you think we should do?

Hi @josephg and welcome back :-)

I took a quick read-through of the spec and think the pick-up / put-down concept is really novel and clever. Here are some questions and thoughts:

Pick-up and put-down across ops
With regards to the pick-up 'slots' can you clarify whether a valid op may leave a sub-tree in suspense in the slot storage? In other words, can I have a pick-up operation to a named / numbered register that does not have a corresponding put-down operation in the same op?

It seems to me that allowing this sort of behaviour would just be asking for crazy corner-cases and challenges down the road. An alternative would be to encourage consumers of the api to create their own 'storage areas' within the json tree for disassociated trees if and when their app logic requires it.

So, assuming the behaviour is disallowed, that means some stronger validation can be created for ops and you avoid the issue of potential conflicts on 'pick-up' operations to the same slot.

List splicing vs overwriting
In your example of moving something between two items in a list, the item is inserted. Are the semantics to splice the item in or to overwrite the item at that position? It seems to me (perhaps naively) that in the former case, overwriting items will be non-trivial (or the pick-up/delete operation in a list context needs to be specialized to splice vs set to undefined or delete) while in the latter case, inserting items into the middle of a list will be non-trivial. What are your thoughts?

I may be going out on a limb here, but what's your take on presence?

I've embarked on adding presence to json0.

Related proposal for OT spec changes: ottypes/docs#27

Currently the library allows any key path to be used to make and apply operations. If the operation is created by a user (which it usually will be), we should disallow obviously unsafe object keys from being assigned to.

The canonical example is assigning to obj.__proto__.

I propose we make __proto__ an invalid key for operations to contain. Are there others we should explicitly disallow?

There are several Python translations of the delta library from Quilljs, but there's no Python translation of an OT library to work on JSON objects.

Before I try reinventing the wheel, are you aware of any Python port of JSON1? Is that where you would recommend getting started?

Hi Joseph.
What do you think about using Ethereum public ledger as the content server? Ethereum is a decentralised p2p public ledger - similar to Bitcoin, but one that allows to include turing complete scripts in transactions and thus allows sophisticated built-in support for smart contracts.
In this way - creating a new document will be similar to creating a new contract. And adding new content would be similar to sending transaction to this contract. Since the transactions are stored in public ledger - everyone can access them and thus read the document content.
I think this suggestion solves some of the issues:

1. Monetisation - whoever wants to create a new document and/or add content pays for it in ether - and this mechanism is supported by Ethereum.
2. User addresses - will be standard ethereum addresses.
   Also you don't have to host the content - it is hosted on ethereum blockchain, and paid by users, you just provide a front-end that can read the info from the ledger.
   Disclaimer: I am not familiar very deep with Ethereum technology - just read about it and tried the tutorial, so maybe it's harder that I think.

The background is this, in the process of using slate, there is a basic operation called split_node，which is：

{
   type: 'split_node',
   position: 4,
   path: [0, 0],
   properties: { bold: true },
}

and original document like this:

[
  {
     type: 'paragraph',
     children: [{ text: 'some text!' }]
  }
]

The final expected result after the operation is as follows：

[
  {
     type: 'paragraph',
     children: [
         { text: 'some' }
         { text: ' text!' }
     ]
  }
]

In order to use ot-json1, I transform the origin path to [0, 'children', 0, 'text'].

My idea is to use editOp to truncate the previous result and use insertOp to insert the latter result, but I encountered a problem that there is no field in the original op that represents the entire string.

So my idea is to first copy the previous result to the position of the second result, and then use string truncation to achieve the final effect.

So I want to ask if it is possible to provide a basic copyOp(formPath, toPath) to achieve this effect? Or is there any way I haven't thought of, I am very willing to learn！

As per my understanding ShareJS is a full collaboration system, complete with Transformation Functions and the Networking algorithm.

ottypes like json0 and json1 are the transformation function themselves extracted out of ShareJS?

So these are basically algorithms? I'd really like my team & I to contribute to these projects. We are literate is JS / TS / CofeeScript / ES6 etc. but in order to contribute to an algorithmic project such as this what can be some reference material?

Are the algorithms being used here out of one or more specific research papers?

Are there general OT algorithms in the wild? Other implementations / Academic study / research papers?

Are you creating your own algorithm here? If so, can you specify some mathematical / theoretical foundation?

It would be great if you could revert for json1 and ottypes/json0

I noticed the following quote in #4 (comment), and thought I'd make an issue for it:

(And I'd like to port the tests away from coffeescript too!)

This issue can be closed when all the tests are ported to JavaScript.

I am thrilled to see this project at a steady slow burn. Really appreciate your long-term efforts on this!