The smallest quantized version (2bit) needs 20GB of RAM (which can be offloaded onto the VRAM of a decent 4090 GPU). The 4bit quantized versions are the largest models that can just about fit onto a 32GB system (29GB-31B). The 6bit (41GB) and 8bit (52GB) models need a 64GB system. You would need multiple GPUs with shared memory if you wanted to offload the higher precision models to VRAM.

I've experimented with the 7B and 13B models, but haven't experimented with these models yet, nor other larger models.

Papers and references (page maintained by central academic in the world of CRDTs): https://crdt.tech

Group doing research into how they can be used to build interesting collaborative (and async) applications: https://www.inkandswitch.com

A few of the major open source implementations - mostly for rich text editing or JSON like data structures:

- Yjs: https://github.com/yjs/yjs

- Automerge: https://github.com/automerge/automerge

- Peritext: https://www.inkandswitch.com/peritext/

- Dimond types: https://github.com/josephg/diamond-types

People building eventually consistent database syncing with them:

- https://electric-sql.com (Postgres <-> SQLite)

- https://vlcn.io (SQLite <-> SQLite)

Open source colaborative servers (coordination, persistance, presence):

- https://github.com/ueberdosis/hocuspocus

- https://github.com/partykit/partykit

- https://github.com/firesync-org/firesync