Syncing

How it works

LiveStore is based on the idea of event-sourcing which means it syncs events across clients (via a central sync backend) and then materializes the events in the local SQLite database. This means LiveStore isn’t syncing the SQLite database itself directly but only the events that are used to materialize the database making sure it’s kept in sync across clients.

The syncing mechanism is similar to how Git works in that regard that it’s based on a “push/pull” model. Upstream events always need to be pulled before a client can push its own events to preserve a global total order of events. Local pending events which haven’t been pushed yet need to be rebased on top of the latest upstream events before they can be pushed.

Events

A LiveStore event consists of the following data:

• seqNum: event sequence number
• parentSeqNum: parent event sequence number
• name: event name (refers to a event definition in the schema)
• args: event arguments (encoded using the event’s schema definition, usually JSON)

Event sequence numbers

• Event sequence numbers: monotonically increasing integers
  • client event sequence number to sync across client sessions (never exposed to the sync backend)

Sync heads

• The latest event in a eventlog is referred to as the “head” (similar to how Git refers to the latest commit as the “head”).
• Given that LiveStore does hierarchical syncing between the client session, the client leader and the sync backend, there are three heads (i.e. the client session head, the client leader head, and the sync backend head).

Sync backend

The sync backend acts as the global authority and determines the total order of events (“causality”). It’s responsible for storing and querying events and for notifying clients when new events are available.

Requirements for sync backend

• Needs to provide an efficient way to query an ordered list of events given a starting event ID (often referred to as cursor).
• Ideally provides a “reactivity” mechanism to notify clients when new events are available (e.g. via WebSocket, HTTP long-polling, etc).
  • Alternatively, the client can periodically query for new events which is less efficient.

Clients

• Each client initially chooses a random clientId as its globally unique ID
  • LiveStore uses a 6-char nanoid
    • In the unlikely event of a collision which is detected by the sync backend the first time a client tries to push, the client chooses a new random clientId, patches the local events with the new clientId, and tries again.

Client sessions

• Each client has at least one client session
• Client sessions within the same client share local data
• In web adapters: multiple tabs/windows can be different sessions within the same client
• Sessions are identified by a sessionId which can persist (e.g., across tab reloads in web)
• For adapters which support multiple client sessions (e.g. web), LiveStore also supports local syncing across client sessions (e.g. across browser tabs or worker threads)
• Client session events are not synced to the sync backend

Auth (authentication & authorization)

• TODO
  • Provide basic example
  • Encryption

Advanced

Sequence diagrams

Pulling events (without unpushed events)

Pushing events

Rebasing

Merge conflicts

• Merge conflict handling isn’t implemented yet (see this issue).
• Merge conflict detection and resolution will be based on the upcoming facts system functionality.

Compaction

• Compaction isn’t implemented yet (see this issue)
• Compaction will be based on the upcoming facts system functionality.

Partitioning

• Currently LiveStore assumes a 1:1 mapping between an eventlog and a SQLite database.
• In the future, LiveStore aims to support multiple eventlogs (see this issue).

Backend Reset Detection

When a sync backend is reset (e.g., deleting .wrangler/state for Cloudflare, or resetting Postgres), clients that have cached data locally may not know about this reset. LiveStore detects backend resets using a unique backendId that is generated when the backend is first created.

How it works

1. When a sync backend is initialized, it generates a unique backendId
2. Clients store this backendId locally alongside their eventlog
3. On subsequent sync operations, the client sends its stored backendId to verify the backend identity
4. If the backend has been reset, it will have a new backendId, causing a mismatch

Configuring the behavior

You can configure how LiveStore handles backend identity changes using the onBackendIdMismatch option:

const store = await makeStore({
  // ... other options
  sync: {
    backend: yourSyncBackend,
    onBackendIdMismatch: 'reset', // 'reset' | 'shutdown' | 'ignore'
  }
})

Options:

• 'reset' (default): Clear local storage (eventlog and state databases) and shutdown. The app will need to restart and will sync fresh data from the backend. This is the recommended option for development.
• 'shutdown': Shutdown without clearing local storage. On restart, the client will still have stale data and encounter the same error.
• 'ignore': Log the error and continue running. The client will show stale data but keep running (effectively offline mode).

Common scenarios

This feature is particularly useful during development when:

• The sync backend state is deleted (e.g., .wrangler/state for Cloudflare)
• Running with a --reset flag
• Schema changes require re-backfilling data
• Running multiple services (CLI and web UI) that need to stay in sync after a reset

Design decisions / trade-offs

• Require a central sync backend to enforce a global total order of events.
  • This means LiveStore can’t be used in a fully decentralized/P2P manner.
• Do rebasing on the client side (instead of on the sync backend). This allows the user to have more control over the rebase process.

Notes

• Rich text data is best handled via CRDTs (see #263)

Further reading

• Distributed Systems lecture series by Martin Kleppmann: YouTube playlist / lecture notes