1# The data layer and sync
2
3OneNote 2010 got multi-user editing without a server. A notebook is a folder
4on a file share. Every client edits the section files in place, and the
5format's append-only revisions plus some careful file locking let clients
6merge each other's work. Snowbound joins that arrangement as one more peer. It
7has no service, no account and no sync protocol of its own. The share is the
8source of truth, and whatever OneNote can do to a file while Snowbound is
9using it, Snowbound has to handle.
10
11This essay follows an edit from the keyboard to the share, then covers what
12happens when the share is busy, changed or gone.
13
14## Three threads
15
16```text
17UI thread (winit, UIKit) section thread sync thread
18──────────────────────── ────────────────────────────── ──────────────────────────────
19editor emits Edit (ops) ───► apply to the parsed Section read the stamp (header + length)
20 and returns at once write each burst to SQLite unchanged: ask for a seal,
21page reads ◄──────────────── answered from the Section publish it
22events ◄──────────────────── Changed, Rejected, published changed: read once, ask
23 seal ◄────────────────────────── for a rebase
24 rebase onto the new image ◄───── acknowledge: base += transaction
25```
26
27- **The UI thread only emits ops.** The editor records the ops each change
28 lowers to as it makes the change. Submitting them to the session returns
29 immediately. Parsing, revision building and SQLite never run on a frame.
30- **The section thread** (`notebook::working`) is the only owner of the parsed
31 `onestore::Section`. It applies edits, writes each burst of them to the
32 replica in one durable commit, and answers page reads. A rebase runs on a
33 fresh thread that builds its own section while the old one keeps answering
34 reads, so opening a page never waits on the network.
35- **The sync thread** (`notebook::worker`, stepping `notebook::sync`) does all
36 network I/O. It reads the stamp when a local edit waits or the notebook's watch
37 reports the file (every two seconds where nothing watches), then publishes or
38 rereads. Idle, it reads nothing.
39
40`notebook::session` wraps this up for apps. `Notebook` covers discovery and
41structure, `Section` covers one section's pages, edits, events and conflict
42pages. Both desktop and iOS use exactly this surface.
43
44## The replica
45
46Each open section has a replica: a SQLite database in the app's cache, named by
47where the notebook is and the section's logical identity, so the same file
48reopens the same queue after a relaunch. It holds:
49
50- **the base**: the last remote image the queue applies to, stored in chunks so
51 that acknowledging a publication rewrites only the chunks the transaction
52 touched;
53- **batches of edits**: each edit's ops, serialized, grouped into the batch
54 that will publish together;
55- **payloads**: picture and attachment bytes, stored once each, keyed by hash.
56
57The working state is never stored. It is the base with each sealed batch
58replayed and the open batch applied, rebuilt on open.
59
60A password-protected section's replica holds nothing in the clear. Its base and sealed
61transactions are the file's own ciphertext, and its edits and payloads are sealed with
62AES-256-GCM under a key derived from the section's, so the queue opens, offline or after a
63relaunch, only once the section is unlocked. A locked section is never opened in the
64background; its edits wait until it is unlocked again. A replica of a section another device
65then protects is deleted the next time the notebook is read, unless edits wait in it; those
66wait for the key, then come back as copies of the pages they changed. The database runs in WAL
67mode with full synchronous commits, and every setting is read back to check it
68took.
69
70The location matters because OneNote lets a notebook folder be copied whole, a
71Finder duplicate or an iCloud `Name 2` conflict copy, and every section in the
72copy keeps its identity. Keyed by identity alone, the copy would open the
73original's queue and publish its edits into the wrong file. So replicas live in
74`cache/replicas/<hash of location>/<identity>.sqlite` (`notebook::location`),
75where a location is a canonical local path or `smb://server/share/root`, and a
76`location` file in each folder names it. A section renamed or moved inside its
77notebook keeps its replica; a copy of a section inside one notebook (`Cross
782.one` beside `Cross.one`, which OneNote opens as a section of its own) is keyed
79by its own path instead, the original being the one the folder's TOC lists.
80
81When a notebook moves, its queue follows by one of three roads:
82
83- **the app moved it** (iOS Rename, or Notebook Properties' "Rename the folder
84 too"): `location::moved` moves the folder's replicas to the new location, the
85 latter through `Notebook::rename_folder` once no other writer holds a file;
86- **something else moved it** (Finder): on open, a section with no replica takes
87 one from the folder of a local location that no longer exists, if the file
88 stands as that replica's base or has moved on from it (a higher header
89 generation). A replica whose base is newer than the file, or a different state
90 of the same generation, belongs to another copy and stays;
91- **an older install** named replicas by identity alone in the cache's top or
92 its `smb` folder: the first location to open the section takes it.
93
94An attached file is a payload like a picture: its bytes ride in the edit that
95inserts it and publish inside that edit's revision. No other file is written,
96so nothing needs to reach the share before the revision that names it. That
97ordering would matter only for `_onefiles` payloads, which OneNote 2010 never
98writes ([file format](file-format.md)). A large file makes a publication as
99large as itself.
100
101A keystroke that publishes immediately costs three commits: the edit, the seal
102(which also records the publication attempt), and the receipt. Each one is
103ordered against something outside the cache. The edit must be durable before
104`apply` answers. The attempt must be durable before any byte reaches the
105share, because an attempt that might have landed is never replayed blindly.
106The receipt follows the file's own commit. Merging any two of them would open a
107window where a crash forgets or duplicates work.
108
109## A sync step
110
111```text
112stamp = remote.stamp() one small, uncoordinated read
113if stamp == base.stamp:
114 publish the sealed batch (if any) Ok ─► acknowledge
115 NotCommitted ─► retry later
116 Unknown ─► attempted; confirm before anything else
117else:
118 image = remote.read() coordinated snapshot, only now
119 rebase the queue onto image merge, maybe conflict pages
120 base := image
121```
122
123A batch stays open while the sync thread is busy, so keystrokes that arrive
124during one round trip publish together in the next. A check costs a
125kilobyte-sized read. A publication costs one appended revision, not the
126section.
127
128Uncertainty is handled explicitly. A publication attempt is recorded before
129the first network byte. If the reply is lost, the attempt confirms only when
130the remote holds its revisions (or every page it changed, as it changed them).
131Otherwise it waits as `AwaitingConfirmation`. A Live Share host lands a batch at
132most once, as it looks for the batch's revisions before applying it, and commits a
133protected section's sealed transaction only on the image it was made on. OneNote's
134Optimize drops a revision a later one supersedes, so the host also keeps, with the
135share, each device's last commit to each section, the only one it may send again.
136The next step therefore sends it the same batch again. Refused because the host no
137longer holds that image, the batch has landed if the section holds its revisions,
138and else never will: the queue replays on the section as it is. Any other remote, where a late write could land
139a second copy, waits for `release`, which exports a recovery archive first, then
140republishes or abandons.
141
142## Merging
143
144When the stamp has moved, someone else committed. The queue replays on the new
145image one op at a time (`notebook::merge`):
146
147- an op whose objects the remote left alone applies as it is;
148- a text op on text the remote also changed shifts past the remote's changes,
149 provided its range stays clear of them;
150- a move, deletion or setting the remote already made is dropped as done;
151- anything else conflicts.
152
153Conflicts follow OneNote 2010 exactly, because the other clients are OneNote.
154The conflicting op is dropped, along with every later op that names what it
155named. The remote's version stays the page. The local version becomes a
156read-only **conflict page** under it, with the conflicting objects marked and
157the page labelled with its author, so OneNote and Snowbound both show the
158same bar, the same highlighted paragraphs and the same Delete Conflict Page. A
159conflict never blocks the queue: it becomes one more queued edit. Page-list
160edits merge the way OneNote merges page series (a page someone moved keeps
161their placement, and a page deleted remotely but edited locally comes back as a
162copy). Each rule was checked against a capture of two real OneNote clients
163doing it first (`corpus/conflict-page`).
164
165## Why an embedded SMB client
166
167OneNote doesn't use lock files. It coordinates through the section file
168itself, with share modes on open and one-byte locks far past the end of the
169data:
170
171| Role | Open | Byte locks |
172| --- | --- | --- |
173| reader | read, sharing read/write/delete | shared lock on the reader byte, held for the read |
174| writer | read/write, denying other writers | the reader byte shared, plus an exclusive writer byte |
175| maintenance (Optimize) | as a writer | an exclusive range that covers the reader byte |
176
177Maintenance rewrites the file under a new name and renames it into place, so a
178handle to the old file goes stale even though its contents still parse.
179
180Snowbound has to take exactly these locks, or OneNote will either trample it
181or be locked out. macOS's `smbfs` can't express them. POSIX byte-range locks
182are unsupported. `flock` of either kind becomes an exclusive lock over the
183whole file, which fails OneNote's reads. The only exclusive-byte primitive is
184private and has no shared form. On top of that, `smbfs` serves reads from its
185own lease-backed cache, defers closes, writes whole cached pages back on
186`fsync`, and leaves AppleDouble `._` files beside sections it writes. On iOS
187an app sees a share only through Files, with no control over locks.
188
189So `notebook::smb` speaks SMB2 itself (on the `smb2` crate's message layer)
190and takes OneNote's opens and bytes exactly, without leases. It never denies
191OneNote a read. The leases matter because a Windows client holding a lease
192keeps its byte locks local and only reveals them when another open breaks the
193lease. So a peer must open before it locks, and never hold handles between
194operations. A commit is a handful of compound requests: open; take the locks,
195check identity and stamp; write appended bytes and patches, then flush; write
196the header, the counter and the version, each flushed in order; close.
197
198A notebook on a share that macOS has mounted is opened through the embedded
199client with the account the system keeps for that mount. It falls back to the
200mount only when it can't sign in that way.
201
202A notebook can also be opened from its server's address, with no mount at all (File ▸
203Open Notebook from Server), which is how Mac OS X 10.6 reaches a server that no longer
204speaks SMB1, the only version its Finder has. The notebook is kept by that address
205(`smb://[domain;]user@server/share/folder`), never with a password; the password lives in
206the keychain (the Secret Service on Linux) when the user asks to remember it, and
207otherwise the sign-in asks again at the next launch.
208
209A file's identity is its root object space, not its server file ID, which
210changes when maintenance replaces the file. A check opens by path each time
211for the same reason.
212
213## Offline
214
215Offline is just a sync step that fails. Edits keep landing in the replica, and
216the sync thread retries with backoff and wakes on new local edits. A notebook's
217last listing lets it open while the server is unreachable, and a section opens
218from its replica. On reconnect the queue publishes, or rebases and publishes,
219through the same path as always. A failed directory read keeps the previous
220catalog, so an unreachable share never looks like an emptied notebook.
221Working offline on purpose, as OneNote's Work Offline does, is the same state
222chosen: the sync thread stops stepping until Sync Now or until the user works
223online again.
224
225## Sections that aren't open
226
227OneNote keeps every section of an open notebook in sync and on this computer, not just
228the one on screen. Watched in the lab with a notebook of 200 sections on Samba, OneNote
2292010:
230
231- reads every section file once when it first opens the notebook (all 200 within 15
232 seconds), so a section never shown before opens with the share gone; opening it again
233 with its cache warm, it only lists the folders and reads the files listed otherwise;
234- arms one CHANGE_NOTIFY on the notebook's folder, with WATCH_TREE and a filter of names,
235 attributes, size and last write, and otherwise sends nothing: 22 idle minutes put no SMB
236 request on the wire, not even an echo, and the section on screen waits on the same watch;
237- lists the folder once when the server reports a change, and reads only the section that
238 changed, some seconds later;
239- tries a section it could not reach again about every 31 seconds.
240
241Snowbound does the same. Opening a notebook lists its folders and reads only the files whose
242listed size or last write time changed since it last read them, keeping what it took from
243each, with the file's stamp, in the cache (`discover::Cache`). On a share, a changed section
244whose replica already holds it as it stands, as after its own edits published, costs just a
245stamp read. Each section it did read is handed on with its image, which makes the offline copy
246or rebases the replica while the file's stamp is still the image's, so a launch reads each file
247once, as OneNote does. Dot files (macOS's `.DS_Store` and AppleDouble `._` companions, the
248`.snowbound` folder) and Office's `~$` files are not the notebook's and are skipped.
249`session::Background` then keeps the sections in sync, one thread per notebook:
250
251```text
252connect: arm the watch, list every folder
253 listed as before: compare the known stamp with the replica, locally
254 listed otherwise: check, one section every 100 ms
255share ─ CHANGE_NOTIFY (tree) ─► the file it names is due in 1 s ─► stamp ─► moved: rebase
256 └► a folder it names is listed in 1 s ─► the files listed otherwise
257failing: again in 31 s otherwise: again in an hour, against a lost notification
258```
259
260A check costs one stamp read. Only a section whose replica has edits waiting, or whose file
261moved past the replica's base, has its replica opened for the usual sync steps, and it is
262closed again afterwards; a section without one gets its offline copy. The section open on
263screen is left to its session, whose worker no longer polls: the watch wakes it, and once the
264session lets go of the replica the background checks the file at once. A connection that drops
265takes its watch with it, so the next one lists every folder again, and so does working online
266again.
267
268A notebook in a folder on this computer has no copies and learns of changes from FSEvents or
269inotify. One on a network volume the system mounted keeps copies. A Mac's SMB mount reports
270another client's change to FSEvents only for a folder watched in its own right, and names just
271that folder, so every folder of the notebook is watched and a report lists it. A Linux SMB
272mount reports none of another client's changes to inotify, so there each section's stamp is
273read every 15 seconds. On iOS, a folder on the device (Snowbound's own, or On My iPhone) has no
274copies and learns of other apps' writes through file coordination (`NSFilePresenter`); one a
275file provider keeps elsewhere, as iCloud Drive does, keeps copies and is checked every 15
276seconds. Closing a notebook deletes its copies, except any with edits still waiting or images
277of uploads it keeps (`notebook::published`).
278
279## iCloud Drive
280
281A cloud drive syncs whole files with no lock and no compare-and-swap. The stamp check still runs
282before each append, but only against this device's copy; when two devices append to the same
283base, iCloud keeps one as the file and the other beside it as a **conflict version**
284(`NSFileVersion`), on every device. A conflict version is a stamp check that failed after the
285fact, and each sync step merges the versions it finds before anything else:
286
287```text
288for each version V the remote lists (Remote::versions)
289 C := the file, read coordinated
290 per space of V: a := its newest revision C holds, or holds merged,
291 else C's newest that V holds or that merged one of V's (V's chain stopped
292 at a checkpoint), else none
293 ancestor := V opened at those revisions (Section::open_at)
294 ops := lower_page(ancestor[S], V[S]) per page V changed, V's new pages imported under their
295 identities, its deletions, moves and conflict pages; a page with no a stays C's,
296 V's beside it as a conflict page
297 replay ops on C through merge.rs (conflict pages where they clash), seal_as the named
298 revisions, publish on C's stamp; then retire V (resolved, removed)
299```
300
301A merged page revision is named after the incoming version's revision and the file's revision it
302depends on (a one-way hash of both), and conflict page identities after the version's revision
303alone, so "holds merged" is a lookup by revision identity; the names older builds gave merges
304are recognised too, as devices run mixed builds. A version merged once, by this device or
305another, merges as nothing the next time. Two devices merging the same version onto the same
306current page state name corresponding outcomes alike; onto different states, differently, as
307their contents differ. This deduplicates retries; it does not make integrations from opposite
308primary bases order-independent or produce byte-identical files. Any device merges any version
309at once; nothing waits for the device that wrote it. A device lists none while its own file is
310not yet the newest iCloud holds: a version found held in that stale copy would be retired
311everywhere, then lost to the download that replaces the copy. A version identical to the file is
312such a copy whatever iCloud reports, and waits too. A version of another section, or one that
313cannot be read, is kept beside the file as `Name (Device).one` instead.
314
315iCloud Drive can also lose an upload outright: another device's lands over it and no conflict
316version reaches this one. So on the Mac, the iPhone and Android each image a section publishes
317to a file a provider keeps stays in the cache (`notebook::published`) until the file is seen
318otherwise, then lists as one more version: one the file holds merges as nothing and goes, one
319it lost merges back as a conflict version does, its conflict pages named for this device. One
320that cannot merge is kept beside the file as `Name (Device).one` on the Mac and the iPhone, and
321in the cache on Android, where no provider takes a new document beside it.
322
323On a cloud drive every read and write of a section file runs under `NSFileCoordinator`, so the
324iCloud daemon never swaps a file between the stamp check and the append. A file iCloud evicted
325(a `.Name.icloud` placeholder, or on macOS 14 and later a dataless file under its own name) lists
326as last read, or as downloading; the host asks for it, and reads the notebook again as files
327arrive. The sync thread never waits for one: the replica serves it meanwhile. Opening a section
328with no replica yet waits for its file. macOS and iOS offer no public way to keep a file
329downloaded, as the Finder's Keep Downloaded does; evicted files are asked for again whenever
330seen. Each publication is an upload and a
331chance to conflict, so edits wait for a 3 second pause in typing (at most 30 seconds,
332`Section::set_pause`), and publish at once when the app leaves the foreground, the screen locks,
333the Mac's window loses focus or the app quits. A folder presenter reports other devices'
334changes and conflict versions, which change no file, and every section is checked every 15
335seconds besides. Signing out of iCloud closes its notebooks; replicas holding edits stay.
336
337`crates/notebook/tests/cloud.rs` runs devices against a fake iCloud (uploads without
338compare-and-swap, either side of a race winning, late deliveries, offline spans) and checks that
339every device ends on the same file with each typed string once. OneNote 2010 cold-opens a
340merged section with its conflict page as its own (`corpus/icloud-merge`).
341
342## Live Share
343
344Live Share (`notebook::live::share`, beta) opens a notebook one Snowbound holds on others,
345with no service holding the notebook. The host serves its notebook's storage verbs to the peers
346in the share's room; a guest runs the replica, queue and merge it runs on an SMB share against
347them (`Notebook::open_hosted`, `Section::resume_hosted`, `Background::hosted`), so offline
348queueing, rebases and conflict pages behave as on a share, and the host's files are only ever
349written by its own storage, OneNote's locks included. A guest meets the host first in the
350room of a short code (`7KQ-4MZ-9XR`: Crockford base32, its room's number, 30 bits of
351secret and a check symbol; SPAKE2 on its secret and any password) through
352Snowbound's relay or by mDNS, and is welcomed with the share's room and its random secret;
353stopping or restarting a share retires the secret. A guest's commit is checked on the host's
354image before it is committed, and a guest can name nothing outside the notebook. Large reads
355and uploads travel 128 KiB at a time, at most 512 KiB unanswered, so a relay never holds much
356for a slow peer. A host that goes away leaves its guests working offline: the notebook opens
357from its last listing and edits publish when it is back. The relay is
358`crates/relay`.
359
360Live presence, on by default, shows who else has a notebook others reach too (on a server, in
361iCloud Drive, in another computer's folder, or by Live Share): their avatars, page and caret.
362Its room's secret is a random one in `.snowbound/live.json`, which Live Share never serves.
363
364## Notebook structure
365
366Sections, groups and the notebook's own colour live in the `.onetoc2` files,
367which are revision stores edited through the same transaction path. Moving or
368renaming a file also rewrites two header fields OneNote checks on open (the
369parent TOC's identity and a CRC of the file's name). Without them OneNote
370treats the file as a stranger and re-identifies it. Deleting sends sections and
371pages to `OneNote_RecycleBin`, as OneNote does, and opening a notebook empties
372what the bin has held unchanged for 60 days, as OneNote prunes it: judged by the
373content's last change, the pages in one revision, binned section files deleted
374with their TOC entries left (`corpus/recycle-purge`).
375Move or Copy puts a section in another notebook as OneNote 2010 does, written anew under new
376identities, taking the edits its replica still queues. Where OneNote deletes the original, a
377move sends it to the recycle bin, and only once the copy reads back whole and the original is
378as it was copied; otherwise the section stays in both notebooks and the move says so.
379
380What only Snowbound reads lives in the notebook's `.snowbound` folder, which carries the
381Windows hidden attribute so that OneNote never makes a section group of it. Its files are
382plain files, not revision stores: pictures named by their content, and small JSON
383mappings that every writer merges before replacing. The first is tag art: a tag keeps
384OneNote's definition and a fallback symbol on the page, and `tags.json` maps its name and
385symbol to a picture that Snowbound draws in the symbol's place. `live.json` holds the
386notebook's presence room secret.
387
388## Recovery and migration
389
390`export_recovery` snapshots a replica into a single read-only archive: base and
391remote images, queue, attempts, receipts and media. It is the evidence behind
392any decision that could lose work. Cache schema conversions check their own
393output: every converted page must equal the page the old cache held, or the
394conversion rolls back and names the archive it made first.
395
396The [notebook README](../crates/notebook/README.md) documents the API in detail.
397
398A separate [draft multi-file coordination protocol](multi-file-sync.md) explores
399generic cloud mounts; it is independent of the Android port and is not enabled.