antics (u/antics) - Readit News

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

Kjue · a day ago

Am I correctly understanding that you (Moment) have chosen to use Prosemirror and that with that using Yjs was the hard part? Or did you mean to say in the article that you used Yjs directly? It would be less prone to misunderstanding if it read "why we don't use y-prosemirror" and you would lose a lot of potential audience for the post.

I tried to understand what was wrong in Yjs, as I'm using it myself, but your point is not really with Yjs it seems but on how the interaction is with Prosemirror in your use case. I can see why you're bringing up your points against Yjs and I'm having a hard time understanding why you don't consider alternatives to Prosemirror directly. Put another way, "because this integration was bad the source system must also be bad". I do not condone this part of your article. Seems like a sunken cost fallacy to me and reasoning about it at anothers expense, but perhaps not. Hoping to hear back from you.

antics · 19 hours ago

So, we are basically making two points.

First, the fact that Yjs bindings, by design (for ~6 years), replace the entire document, does in my opinion, indicate a fundamental misunderstand what rich text editors need to perform well in any circumstance, not just collaborative ones. As I say in the article... I hope to be able to write another article that this has changed and they now "get" it, but for now I do not think it is appropriate to trust Yjs with this task for production-grade editors. I'm sorry to write this, but I do think it's true! I'm not trying to bag on anyone!

Second, and more material: to deploy Yjs to production in the centralized case, I think you are very much swimming against the current of its architecture. Just one example is permissions. There is no established way to determine which peers in a truly-p2p architecture have permissions to add comments vs edit, so you will end up using a centralized server for that. But that's not free, CRDTs are mechanically much more complicated! For example, you have to figure out how to disallow a user to make mark-only edits if they have "commenter" access, but allow editing the whole doc for "editor" access. This is trivial in `prosemirror-collab` (say) but it's very hard in Yjs because you have to map it "through" their XML transformations model.

I'm happy to talk more about this if it's helpful. But yes, we are trying to say some stuff about Yjs specifically, and some stuff about CRDTs generally.

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

miloignis · a day ago

(Xpost from my lobsters comment since the Author's active over here):

I really disagree with this article - despite protestation, I feel like their issue is with Yjs, not CRDTs in general.

Namely, their proposed solution:

    1. For each document, there is a single authority that holds the source of truth: the document, applied steps, and the current version.
    2. A client submits some transactional steps and the lastSeenVersion.
    3. If the lastSeenVersion does not match the server’s version, the client must fetch recent changes(lastSeenVersion), rebase its own changes on top, and re-submit.
    (3a) If the extra round-trip for rebasing changes is not good enough for you, prosemirror-collab-commit does pretty much the same thing, but it rebases the changes on the authority itself.

This is 80% to a CRDT all by itself! Step 3 there, "rebase its own changes on top" is doing a lot of work and is essentially the core merge function of a CRDT. Also, the steps needed to get the rest of the way to a full CRDT is the solution to their logging woes: tracking every change and its causal history, which is exactly what is needed to exactly re-run any failing trace and debug it.

Here's a modified version of the steps of their proposed solution:

    1. For each document, every participating member holds the document, applied steps, and the current version.
    2. A client submits (to the "server" or p2p) some transactional steps and the lastSeenVersion.
    3. If the lastSeenVersion does not match the "server"/peer’s version, the client must fetch recent changes(lastSeenVersion). The server still accepts the changes. Both the client and the "server" rebase the changes of one on top of the other. Which one gets rebased on top of the other can be determined by change depth, author id, real-world timestamp, "server" timestamp, whatever. If it's by server timestamp, you get the exact behavior from the article's solution.

If you store the casual history of each change, you can also replay the history of the document and how every client sees the document change, exactly as it happened. This is the perfect debugging tool!

CRDTs can store this casual history very efficiently using run-length encoding: diamond-types has done really good work here, with an explanation of their internals here: https://github.com/josephg/diamond-types/blob/master/INTERNA...

In conclusion, the article seems to be really down on CRDTs in general, whereas I would argue that they're really down on Yjs and have written 80+% of a CRDT without meaning to, and would be happier if they finished to 100%. You can still have the exact behavior they have now by using server timestamps when available and falling back to local timestamps that always sort after server timestamps when offline. A 100% casual-history CRDT would also give them much better debugging, since they could replay whatever view of history they want over and over. The only downside is extra storage, which I think diamond-types has shown can be very reasonable.

antics · 20 hours ago

I know it seems that way, but it's actually not 80% of the way to a CRDT because rich text CRDTs are an open research problem. Yjs instead models the document as an XML tree and then attempts to recreate the underlying rich text transaction. This is much, much harder than it looks, and it's inherently lossy, and this fundamental impedance mismatch is one of the core complaints of this article. Some progress is being made on rich text CRDTS, e.g., Peritext[1]. But that only happened a few years ago.

Another important thing is that CRDTs by themselves cannot give you a causal ordering (by which I mean this[2]), because definitionally causal ordering requires a central authority. Funnily enough, the `prosemirror-collab` and `prosemirror-collab-commit` do give you this, because they depend on an authority with a monotonically increasing clock. They also also are MUCH better at representing the user intent, because they express the entirety of the rich text transaction model. This is very emphatically NOT the case with CRDTs, which have to pipe your transaction model through something vastly weaker and less expressive (XML transforms), and force you to completely reconstruct the `Transaction` from scratch.

Lastly for the algorithm you propose... that is, sort of what `prosemirror-collab-commit` is doing.

[1]: https://www.inkandswitch.com/peritext/

[2]: https://www.scattered-thoughts.net/writing/causal-ordering/

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

voctor · a day ago

I think Y.js 14 and the new y-prosemirror binding fix a lot of the encountered issues

antics · 20 hours ago

It might fix the replace-everything bug. It definitely does not fix any of the other issues I mentioned. Even just taking the permissions problem: Yjs is built for a truly p2p topology you as a baseline will have a very hard time establishign which peers are and aren't allowed to make which edits. You can adopt a central server, but then the machinery that makes Yjs amenable to p2p is uselessly complicated. And if you cross that bridge, you'll still have to figure out how to let some clients do mark-only edits to the document for things like comments, while other can edit the whole text. That can be done but it's not at all straightforward, because position mapping is very complicated in the Yjs world.

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

mweidner · a day ago

The PowerSync folks and I worked on a different approach to ProseMirror collaboration here: https://www.powersync.com/blog/collaborative-text-editing-ov... It is neither CRDT nor OT, but does use per-character IDs (like CRDTs) and an authoritative server order of changes (like OT).

The current implementation does suffer from the same issue noted for the Yjs-ProseMirror binding: collaborative changes cause the entire document to be replaced, which messes with some ProseMirror plugins. Specifically, when the client receives a remote change, it rolls back to the previous server state (without any pending local updates), applies the incoming change, and then re-applies its pending local updates; instead of sending a minimal representation of this overall change to ProseMirror, we merely calculate the final state and replace with that.

This is not an inherent limitation of the collaboration algorithm, just an implementation shortcut (as with the Yjs binding). It could be solved by diffing ProseMirror states to find the minimal representation of the overall change, or perhaps by using ProseMirror's built-in undo/redo features to "map" the remote change through the rollback & re-apply steps.

antics · 20 hours ago

Hi Matt! Good to see you here. For those who don't know, Matt also wrote a blog about how to do ProseMirror sync without CRDTs or OT here: https://mattweidner.com/2025/05/21/text-without-crdts.html and I will say I mostly cosign everything here. Our solution is not 100% overlap with theirs, but if it had existed when we started we might not have gone down this road at all.

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

cowboy_henk · a day ago

> sorry if this was not clear

It's easy to make that mistake reading your post because of sentences like

> I want to convince you that all of these things (except true master-less p2p architecture) are easily doable without CRDTs

> But what if you’re using CRDTs? Well, all these problems are 100x harder, and none of these mitigations are available to you.

It sure sounds a lot like you're calling CRDTs in general needlessly complex, not just the yjs-prosemirror integration.

antics · 20 hours ago

To be clear, we ARE arguing CRDTs needlessly complex for the centralized server use case. What I am describing in the "delete and replace all on every keystroke" problem is the point at which it became clear to me that the project did not understand what modern text editors need to perform well in any circumstance, let alone a collab one.

I think this is still reasonable to say because the final paragraph in that section is 100% about how they might fix the delete-all problem, and I hope they do, so that I can write about that, too. But also, that the rest of the article is going to be about how you have to swim upstream against their architecture to accomplish things that are either table stakes or trivial in other solutions.

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

gritzko · a day ago

When I was starting my research into collaborative editing as a PhD student 20+ years ago, rebase-and-resubmit was well known. It was used in one Microsoft team collab product (I forgot the name). It is 100% legit algo except intermittently-connected clients may face challenges (screw them then).

Unless you have to support some complicated scenarios, it will work. I believe Google Docs initially used something of the sort (diff-match-patch based). It annoyed users with alerts "lets rebase your changes", esp on bad WiFi. So they borrowed proper OT from Google Wave and lived happily since (not really).

One way to think about it: how many users will your product have and how strange your data races / corner cases can get. At Google's scale, 0.1% users complaining is a huge shit storm. For others, that is one crazy guy in the channel, no biggie. It all depends.

TLDR: people invented OT/CRDT for a reason.

antics · 20 hours ago

First of all, thanks for chiming in! I wish someone would collect stuff like this and write it down in some sort of "oral history of collab editing."

Second of all, I actually think we're more aligned than it seems here. What we're really advocating for is being super clear about what your end-user goals are, and deriving technology decisions from them, instead of the reverse. Our goals for this technology are (1) users should be able to predict what happens to their data, (2) the editor always run at 60fps, and (3) we are reasonably tolerant of transient periods of disconnection (up to, say, 30s-1m).

Because of (1) in particular, a lot of our evaluation was focused on understanding which situations users would be unable to predict what was going to happen to their data. This is only our own experience, but what we found (and the impetus for part 1 of this series) is that almost 100% of the time, when there is a direct editing conflict, users interpret the results of the dominant CRDT and OT implementations as silently corrupting their data. So, the name of the game is to decrease the likelihood of direct editing conflicts, e.g. presence carets in the live-collab case. In particular, we did not notice a meaningful difference between how users view reconciliations of OT and CRDT implementations.

Since our users could not tell the difference, and in fact viewed all options as equally bad ("disastrous" as one user said), this freed us up to consider a much broader class of algorithms, including prosemirror-collab and prosemirror-collab-commit.

I know there is a rich history of why OT is OT, but our final determination was made pretty simple by the fact that the source of the majority of race conditions in our view come from the difficulty of integrating CRDTs and OT directly into modern editing stacks, like ProseMirror. As far as I am aware, prosemirror-collab-commit behaves as good or better on every dimension than, say, an OTTypes implementation would... and mostly that is because it is native to the expressive `Transaction` model of the modern editor. If we had to do interop I think we would have shipped something noticably worse, and much slower.

If you have a different experience I would love to hear about it, as we are perennially in the market for new ideas here.

Loading parent story...

Loading comment...

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

ianhorn · a day ago

I always mentally slotted prosemirror-collab/your recommended solution in the OT category. What’s the difference between the “rebase” step and the “transformation” step you’re saying it doesn’t need?

antics · 20 hours ago

Great question. Matt has a comment about this here, and he has an actual PhD on the subject! So rather than doing a worse job explaining I will leave it to him to explain: https://news.ycombinator.com/user?id=mweidner

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

lostmsu · a day ago

From the "40 line CRDT replacement":

    const result = step.apply(this.doc);
    if (result.failed) return false;

I suspect this doesn't work.

antics · a day ago

Author here. I'll actually defend this. Most of the subtlety of this part is actually in document schema version mismatches, and you'd handle that at client connect, generally, since we want the server to dictate the schema version you're using.

In general, the client implementation of collab is pretty simple. Nearly all of the subtlety lies in the server. But it, too, is generally not a lot of code, see for example the author's implementation: https://github.com/ProseMirror/website/tree/master/src/colla...

antics commented on Lies I was told about collaborative editing, Part 2: Why we don't use Yjs moment.dev/blog/lies-i-wa... · Posted by u/antics

crashabr · a day ago

You have an amazing tagline. This is the first time I read a tagline and thought: this is exactly what I was looking for.

But the product seems much more narrow than an actual tool run the whole business in markdown. I was hoping to see Logseq on steroids, and it feels like a tool builder primarily. I love the tool building aspect, but the fundamentals of simply organizing docs (docs, presentations, assets etc, the basics of a business) are either not part of the core offering or not presented well at all.

I love the idea of building custom tools on top of MD and it's part of my wishlist, but I feel little deceived by your tagline so I wanted to share that :)

antics · a day ago

This is great feedback, thank you. I will say that IS our goal... but we only really launched last week and are still figuring out what resonates with people and what they really want! It sounds like you're saying that the organization aspects are not there, which is very helpful to know... I am not quite sure I understand if you also think the toolbuilding is lacking?

If you are open to it, I'd love the opportunity to hear more. Here or email (alex@moment.dev) or our Discord (bottom right of our website) or Twitter/X... or whatever you prefer.