SYSTEM DESIGN #17 · INTERVIEW GUIDE
Design a Real-Time Collaborative Document Editor
Google Docs makes it look easy — two people edit the same sentence simultaneously and neither sees a conflict. Behind that seamless experience is one of the most mathematically sophisticated algorithms in distributed systems: Conflict-free Replicated Data Types (CRDTs). This system implements a production-grade collaborative editor using Yjs on the client (a battle-tested CRDT library), WebSocket gateways that route users to shared document rooms, a Kafka operation stream for durability, and Redis Pub/Sub for broadcasting updates to all connected cursors in real time. The architecture handles the hard edge cases: network partitions where two users edit offline and merge on reconnect, large documents with 10,000+ concurrent cursors, and S3 snapshot checkpointing so the server doesn’t have to replay 6 months of operations on every restart.
Yjs CRDTWebSocketKafkaRedis Pub/SubS3
💡
The Gist — What Problem Are We Solving?
A whiteboard multiple people can write on simultaneously from different rooms
When Alice types at position 3 and Bob deletes position 3 at the same moment from different cities, what should happen? Without a careful algorithm, one of them loses their change. Operational Transformation (OT) and CRDTs are the mathematical frameworks that guarantee both changes survive correctly, every collaborator converges to the same document, and even offline edits merge cleanly when the user reconnects.
💬Think of it as Google Docs magic: no matter what everyone types simultaneously, the document always ends up consistent — and nobody’s work is ever lost.
These are the capabilities the system must deliver — what users and operators can actually do with it.
✏️
Collaborative Editing
✏️Multiple users edit same document simultaneously; all see each other’s changes
⚙️
Conflict Resolution
⚙️Concurrent edits always converge to the same result for all users
👁️
Presence
👁️Show cursor positions and selections of all active collaborators
📖
Version History
📖Full operation log; undo/redo; restore any previous version
💾
Offline Support
💾Edit while offline; changes merge correctly on reconnect
⚡
Non-Functional Requirements
These define how well the system must perform — the quality attributes that separate a toy from a production system.
⚡ Op Propagation
⚡<50ms from user keystroke to all collaborators seeing it
🔒 Consistency
🔒All clients always converge to identical document state
👥 Scale
👥1,000+ simultaneous collaborators per document
📶 Offline
📶Full offline editing; conflict-free merge on reconnect
📅 History
📅30 days of full operation log; 1 year of snapshots
📊
Key Metrics — The Numbers That Define This System
The headline numbers to know cold — and be ready to explain how each one is achieved.
🏗️
System Architecture Diagram
Full data flow from source to serving. Each layer scales independently.
Ingestion
Collaborative Editor Flow
Client
Yjs CRDT
→
→
→
↓
Processing
Document Store
Postgres + Redis op buffer + S3 snapshots); Op broadcast
→
all collaborators in room via Redis pub/sub; Presence
→
🗺️
End-to-End User Journey
Trace a single request end-to-end — the story interviewers want you to tell fluently.
1
User opens document
— Client loads latest snapshot from S3 + ops since snapshot from Redis ring buffer; initialises local Yjs CRDT state
2
User types character
— Yjs assigns unique ID (clientID + logical clock) to insert operation; sends op to WebSocket gateway
3
Server applies op
— Op Processor applies to authoritative Yjs state; assigns server sequence number; broadcasts to all room members via Redis pub/sub
4
All clients receive op
— Each client’s Yjs CRDT applies op deterministically — same result regardless of arrival order
5
Concurrent edits
— Alice inserts at pos 3, Bob deletes pos 3 simultaneously. Both ops arrive at server. Yjs CRDT resolves: Alice’s char gets unique ID, Bob’s delete targets specific char ID — no conflict.
6
Snapshot triggered
— After 1,000 ops: Yjs state serialised and stored in S3; Redis ring buffer for recent ops cleared to last 1,000
🔭
High-Level Design — Component Breakdown
Core components — each with a single, well-defined responsibility. The key architectural insight: each layer scales independently, and failure in one component is isolated from the rest.
1 — Client
Handles responsibilities for the Client layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
2 — WS Gateway
Terminates WebSocket connections. Maintains connection registry in Redis. Routes incoming messages to target connection via gRPC. Scales horizontally with sticky routing via consistent hash on user_id.
3 — Op Processor
Handles responsibilities for the Op Processor layer. Designed for independent horizontal scaling — additional instances added without architectural changes. Communicates asynchronously with adjacent components to maximise throughput and fault isolation.
4 — Kafka
Distributed event bus with RF=3 for durability. Partitioned by user_id_hash for per-user ordering. LZ4 compression reduces storage cost by 60%. Exactly-once semantics via idempotent producers and transactional consumers.
5 — Postgres
Relational source of truth for transactional data. ACID guarantees for order records, account state, and financial ledgers. Read replicas serve analytics queries; connection pooling via PgBouncer.
6 — S3
Object storage for raw events (Iceberg/Parquet), media assets, ML model artefacts, and snapshots. Lifecycle rules tier data to Glacier after 90 days. Versioning disabled on ephemeral buckets (snaps, stories) to ensure hard deletes.
🔬
Low-Level Design — Deep Dives
Deep dives worth explaining in detail in any senior engineering interview. For each: know the data structure, the algorithm, the why, and the trade-off you made.
1 — Yjs CRDT Client
Document State · Sync Protocol
Yjs Y.Doc maintains document state as a CRDT. Every local edit generates a Uint8Array update (binary delta). Client syncs with server via Y.WebsocketProvider: on connect, exchanges sync messages to reconcile diverged state. Offline edits buffered in IndexedDB. On reconnect: pending updates sent to server; server returns any missed updates since last seen clock. Merge is automatic and conflict-free by CRDT design.
import * as Y from ‘yjs’
import { WebsocketProvider } from ‘y-websocket’
const doc = new Y.Doc()
const provider = new WebsocketProvider(WS_URL, docId, doc)
const text = doc.getText(‘content’)
// Edit: automatically generates CRDT update
text.insert(0, ‘Hello world’)
2 — WebSocket Room Gateway
Room Routing · Consistent Hash
Consistent hash ring (using Rendezvous hashing) maps document_id → gateway instance. All users editing the same document connect to the same gateway — ensuring all updates flow through a single coordinator per document. On gateway scale-out, only ~1/N documents re-routed. Gateway maintains in-memory list of connected users per document for presence broadcasting. Redis Pub/Sub used for cross-gateway awareness when a document spans multiple shards.
def get_gateway(doc_id):
return consistent_hash_ring.get_node(doc_id)
# Client connects to assigned gateway
gateway = route_table.lookup(get_gateway(doc_id))
ws = WebSocket(f’wss://{gateway}/doc/{doc_id}’)
3 — Operation Stream (Kafka)
Durability · Ordered per Doc
Every Yjs update (binary delta) published to Kafka topic yjs_updates, partitioned by document_id_hash. Partition key ensures all updates for the same document are ordered and consumed sequentially. Consumer group: snapshot_service reads updates and applies to in-memory Yjs doc; writes snapshot to S3 every 1000 updates. Consumer group: audit_log writes update metadata (user_id, doc_id, ts, update_size) to ClickHouse for activity analytics.
producer.produce(
topic=’yjs_updates’,
key=doc_id.encode(), # ensures ordering per doc
value=yjs_update_bytes,
headers={‘user_id’: user_id, ‘doc_id’: doc_id}
)
4 — Snapshot & Recovery
S3 · 1000-op Checkpoint
Snapshot policy: full Yjs document state serialised (Y.encodeStateAsUpdate(doc)) and written to S3 every 1000 operations or 24h. Snapshot file: {doc_id}/{snapshot_id}.bin. On document load: (1) Fetch latest snapshot from S3. (2) Fetch Kafka updates after snapshot’s offset. (3) Apply updates to in-memory doc. Typical load time: <200ms. Snapshot retention: 30 days (30 versions). Rollback: apply snapshot + selective Kafka offset replay.
def load_document(doc_id):
snapshot_bytes, offset = s3.get_latest_snapshot(doc_id)
doc = Y.Doc()
Y.applyUpdate(doc, snapshot_bytes)
pending = kafka.read_from(doc_id, offset)
for update in pending:
Y.applyUpdate(doc, update)
return doc
⚖️
Trade-offs & Decision Log
Every senior interview comes down to these decisions. Know the exact trade-off, the reasoning, and the specific numbers that justify each choice.
⚖️ CRDT vs Operational Transform (OT)
✓
CRDT (Yjs) ✅ Chosen
- Convergence guaranteed mathematically — no coordination needed
- Merges offline edits on reconnection without server round-trip
- Yjs is battle-tested — used by Notion, Linear, Coda
- Higher memory overhead per document (tombstoned deletions)
→
Operational Transform (Google OT)
- Proven at Google Docs scale since 2006
- Requires central server to transform operations in order
- Network partition causes divergence — must re-sync on reconnect
- Transform functions for all operation pairs are hard to implement correctly
💡Decision: Yjs CRDT for client-side state; server stores compressed Yjs updates; OT considered but CRDT wins on offline-first requirement
⚖️ WebSocket Per-Document vs Shared Room Architecture
✓
Room-Based Gateway ✅ Chosen
- All users in same document routed to same gateway instance
- Redis Pub/Sub broadcasts updates to all room members
- O(1) fanout — one publish, N subscribers receive
- Requires sticky routing or consistent hash ring for room affinity
→
Direct peer-to-peer (WebRTC)
- Zero server bandwidth for document sync
- Latency lower for 2-user sessions
- NAT traversal fails in ~15% of network environments
- Does not work for >8 simultaneous editors
💡Decision: Room-based WebSocket gateway for all collaborative sessions; WebRTC data channel as experimental P2P mode for <3 users
🎯Interview Questions — Answered
The exact questions interviewers ask — with production-grade answers
Q1
What happens when two users simultaneously delete and edit the same character?
This is the classic CRDT concurrent deletion conflict. Yjs handles it via tombstoning: deleted characters are marked as deleted (tombstone) but not removed from the internal document array. A concurrent edit to a tombstoned character is silently discarded — the edit has no visible effect since the character no longer exists in the logical document. The tombstone is retained until all clients have acknowledged the deletion (tracked via version vector), after which it is garbage collected from the Yjs state. This ensures that the document state converges to the same result on all clients regardless of network partitioning or operation reordering.
Q2
How does the system handle a document with 10,000 concurrent editors?
Above ~500 concurrent editors per WebSocket gateway instance, the room is sharded: (1) Document is partitioned into sections (paragraphs). (2) Each section is assigned to a WebSocket gateway shard. (3) Editors working in different sections are routed to different gateways. (4) Inter-shard operation synchronisation happens via Kafka: each gateway publishes its section’s Yjs update stream, and all gateways subscribe to all sections to maintain a full document view. Cursor and presence data (who is editing where) uses a separate presence channel via Redis Pub/Sub — decoupled from document state for lower latency.
Q3
How are documents snapshotted to avoid replaying years of operations on load?
S3 snapshot policy: full document state snapshot every 1,000 operations or every 24 hours, whichever comes first. Snapshot is a binary-encoded Yjs document state (compact representation of all current content). On document load: (1) Latest snapshot is fetched from S3 (typically <500KB for a normal document). (2) Yjs update log since the snapshot is fetched from Kafka (typically <100 operations). (3) Updates are applied to the snapshot in order. (4) Document is fully initialised in <200ms. Without snapshots, loading a heavily-edited document would require replaying millions of operations — unacceptable latency. Snapshots are stored with version tags — rollback to any snapshot is possible within 30 days.
System Design Series · Every Tuesday & Thursday
Level up your system design interviews
Each post covers Gist, Functional & Non-Functional Requirements, Key Metrics, System Diagram, User Journey, HLD, LLD, and Trade-offs & FAQs.
Subscribe to never miss a post →
Previous Articles
Categories: System Design
Tags: collaborative editor, crdt, interview prep, operational transform, system design, websocket, yjs
Leave a Reply