Learn Yjs Interactively
learn.yjs.dev294 points by paulgb a day ago
294 points by paulgb a day ago
Hey HN! I'm the developer at Jamsocket who made this. In case you're not familiar with Yjs, it's a CRDT library for building collaborative and local-first apps.
The thing is, if you're not used to working with distributed state there's definitely a learning curve. Even simple things like incrementing a counter — the "hello world" of JavaScript framework demos — get tricky when dealing with multiple clients. Worse, a lot of tutorials are just like "install this library and text editor integration and boom you have an app", which doesn't give you a good mental model for what's actually happening.
So we made Learn Yjs! It's an interactive Yjs tutorial. I wanted it to be really intuitive for people just getting their feet wet with local-first development, so there are lots of explorable demos and coding exercises. The idea is to use interactive examples to build an understanding from the ground up.
Hope you like it :)
> "install this library and text editor integration and boom you have an app", which doesn't give you a good mental model for what's actually happening.
I was interested in working with CRDTs and collaboration in general for a while now and this has been the biggest issue whenever I tried to get into it. Websockets also seem a bit harder to find (free) hosting for.
Btw, your other tutorials on CRDTs were also a great help.
Thank you!
As far as websocket hosting, there are a few services out there. We offer a pretty generous free tier for Y-Sweet [1] with 10GB of storage. There's also PartyKit and Liveblocks, though I'm not sure what their pricing is like.
I am just looking at this but quick question. Is there an example for building a local-first app? Also, I working with plain vanilla HTML & JS and Alpine. Can this be used in my apps?
Y-sweet (which this is built with) allows you to make a Yjs app local-first by passing “offlineSupport” to the provider when constructing.
There’s a demo of this here: https://demos.y-sweet.dev/color-grid
The source for that demo is here: https://github.com/jamsocket/y-sweet/blob/5fa6941cf5568f4a3f...
That demo uses react, but there is a vanilla js version of the same demo here: https://github.com/jamsocket/y-sweet/tree/main/examples/vani...
(The vanilla js version is not local first, but it would just be a matter of passing the same offlineSupport flag to make it so.)
I just noticed that if you install the example service after setting up an account, it installs a version of @y-sweet/react which doesn't offer the offlineSupport prop.
The version specified is ^0.5.1. After switching to 0.7.1 as the repo you linked uses, all is well. Probably not a big deal for most newcomers, but it might be worth updating dependencies in the example service.
Yjs is nice. Easy to get p2p results.
Yjs backend, its persistence, eventual conflict resolution (one of editors stayed too long offline), history rewind seems like a tough engineering challenge to crack.
Always wanted to have a nice experience with block editor like Platejs https://platejs.org/
There are nice attempts simplify dev experience with solutions like Liveblocks https://liveblocks.io/text-editor However most options come at the expense of controlling your data.
CRDT keyword search on HN bring consistently good results and interest in the topic, but no good options on open source backend side. Checked it many times in the past. Please correct me if I missed something.
https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
Most people expectations it should work at least as in Google Docs or Notion, and on the dev side it should be store privately in Postgres JSONB like format without going deep into details.
I agree. In theory you need to just distribute serialized patches but in a real world backend scenario you may need to integrate with knowledge of current document state, user identity, and possibly even some level of access control.
I’ve wanted to use Y.js with a Go backend multiple times but gave up each time due to time constraints as it’s hard to find simple reference implementations.
I’ve been checking back over the years but it still seems hard to do this outside of Node.js.
Same experience. Go is my default choice for backend too
It can easily be a full time job
You need to mirror all logic and encoding/decoding part
https://github.com/yjs/yjs/blob/main/src/utils/encoding.js
https://github.com/yjs/yjs/commits/main/src/utils/encoding.j...
I was thinking about spining off nodejs instance just for persistence and syncing with postgres
Author here — I think our own almost server is almost exactly what you’re looking for? Y-Sweet [1] is open source, written in Rust and persists documents to S3. We offer a hosted version if you want to quickly check it out, which you can use with your own S3 bucket if you want to self host eventually.
This is the first I’ve heard of Platejs, but we do have a tutorial on integrating a block editor using BlockNote [2]
Fyi, building a collaborative editor right now and it's still hard to do outside of Node
Still better time spend than doing leetcode.
It's my go-to argument against leetcode style interviews.
I would rather ask a candidate to spend 30 mins and do research together on collaborative editing, or visualizing distances used in pgvector or similar vector database.
Imagine how far the whole colabarative editing space moved forward if 1% of leetcode grinding were rerouted
The thing I never got past with yjs was that the generalization of the todo app item-moving thing in this tutorial gets messy so quickly. If you have multiple objects that relate to each other, then you want them in the same Document, and keeping the relationships in sync has lots of gotchas (like needing the fractional indexing in the tutorial).
But then you accumulate a database worth of items and the simple toy model of putting everything into one document amounts to having to ship the whole DB (and it's history) to anything that wants to read any items. So you need to start breaking the items into separate objects and persisting them yourself, which has no documentation except for this tutorial [0] (spoiler alert: it's just an empty "Work in Progress - come back later" and has said that for years and years and years).
So you try to build a Provider to manage things, and the documentation story is the same [1].
Then you find out about Subdocuments and get happy that there's a solution to this common problem [2], but then learn that some Providers can't handle any relationships between subdocuments, and some Providers don't even support subdocuments, which goes back to issue of DIY providers not being documented.
So then you try to read some code to learn about it (after all, it's open source, we can't expect the dev to do all the work for us for free; let's do some work too!) and realize that what yjs really is at its core is a protocol. And that protocol -- the very essence of yjs -- is also totally undocumented and spread across a bunch of relatively idiosyncratic code. Unless you count a paper that documents how a yjs precursor worked a decade ago.
I didn't intend this to turn into a rant, but I've spent so much time trying to get it working over the years, and the gap between the toy 'Look it's magic!' demos and anything real is just so wide and the documentation for that part has been empty except "Work in Progress" for a decade.
I'm hopeful that some of the companies trying to monetize yjs, like the author of this post, are willing to step into that gap (which is who I'm writing this for, btw).
[0] https://docs.yjs.dev/tutorials/persisting-the-document-to-a-...
[1] https://docs.yjs.dev/tutorials/creating-a-custom-provider
[2] https://docs.yjs.dev/api/subdocuments
[3] I was really hoping that as I wrote this, the situation has changed, but it looks like all the documentation is just as WIP as five years ago and same for subdocument support.
I used Yjs in a small side project a while ago. The client side was fairly easy to learn and use. On the server side though, I wanted to deploy a single binary, but examples in languages other than Node (e.g. for the Rust port Yrs) seemed close to nonexistent, so I ended up slightly adapting https://github.com/yjs/y-websocket (Node) with persistence based on LevelDB. That's suboptimal for me. I wonder if there are good resources for implementing the server-side with persistence in, say, Rust.
Btw, IIRC existing resources for the client side are heavily focused on text editor integrations, so this is really helpful. I had to fumble a bit myself because I wasn't using a text editor integration.
I'm currently using Yjs to build an app that needs to work offline. It's not actually a real-time or collaborative app. However, if you think of the server as being one collaborator and the app being another, then it's easy to imagine use cases for it.
> The trick is a technique called fractional indexing, in which indices are fractions rather than integers.
How many times you can use fractional indexing before it stops working?
Sorry, just did that test - it is 1075, not massive, I would risk saying that may be major issue of this lib.
Here is the code:
let left = 0, right = 1
let rightPrevious = 2
let counter = 0;
while (right > left && right < rightPrevious) {
rightPrevious = right;
right /= 2;
counter++
console.log(counter)
Did look very briefly into repo, but didn't find a place where it is handled and don't have time to dig it, but if that issue is true I may help with resolving it. LMK
Good point. Usually fractional indexing is done with arbitrary-precision numbers to avoid exactly this issue. We landed on using JS floats to teach the technique because it makes it more intuitive to understand what’s going on, but you make a good point and I’ll make sure we add that caveat and link out to an arbitrary-precision fractional index library.
So perplexing - in the interactive demos, the latency slider acts not as a network latency but more a debounce buffer? Why? Why? I just don't get it
We wanted to go with the simplest model of latency that would suffice to demonstrate the problems that come up when dealing with state mutated in multiple locations. The advantage of this approach is that we can do it without hiding any state from the reader — the pie chart exposes the only internal state that exists. It’s not meant to be an accurate network simulation, just to build intuition.
The interactive demos are beautiful! Is there a library you used to build it?
Thank you! The site as a whole is built with Astro and the demos are “islands of interactivity” built with React.
The demos are really running Yjs under the hood — each “client” has its own document, and when the user clicks I set a timeout to simulate the latency and then manually merge the documents into each other. (There’s a third document too for the timeline in the center.)
Other than that, I’m using dnd kit [1] for the drag and drop functionality in the todos demo, Motion [2] for the animations and CodeMirror [3] for the text editor.
Great work Jake! I love the Astro and the “islands of interactivity”. Is it possible to see the repo, if it's public? I couldn't find it anywhere.
Thank you Jake! I just played with the motion website and loved it. I am going to try to use it sometime
I finally got a four leaf clover and somebody ruined it after like 2m.
Edit: Wait, where did everybody go? I think we broke it, or maybe I got banned. Kudos to the creator just the same. Neat stuff.
You definitely aren’t banned! Maybe people just tapered off for a bit — I see a bunch of cursors now.
There are two more secret emojis, by the way :)
The banner image game is silly but fun
Reminds me of the fun reddit used to have with places and such:
Team purple ftw: https://imgur.com/a/Oxc6jUA
another banger from the jamsocket team
is Yjs a platform like Convex? are they competing? would love a comparison page potentially! :pray:
You can think of Yjs as a protocol for data synchronization. It gives you a way to describe a JSON-like data structure (i.e. nested lists and maps), and keep them in sync across multiple devices.
Yjs itself doesn't provide a platform, but it's an open protocol so there there are service providers (like ourselves[1]) that offer Yjs backends as a service (other notable providers are TipTap/Hocusocus and Liveblocks).
what's the main differences in your opinion between y-sweet and Hocuspocus?
The biggest difference is that Y-Sweet is built around object storage (i.e. S3 and friends) rather than a database.
This allows writes to scale horizontally without a central database being the bottleneck. It’s also a much cheaper way to store lots of documents accumulating over time on S3 than in a database. This is because compute scales with how much data you actually access, not how much data you store.
I wrote about why object storage is a good fit for this here: https://digest.browsertech.com/archive/browsertech-digest-fi....