1# notebook
2
3The application-facing crate: notebook discovery, a durable local replica with
4reconnect reconciliation, external-asset caching, recovery export and optional
5embedded SMB access. A replica keeps one section's queue in a local SQLite database:
6the image the queued edits apply to (the base, in 16 KiB chunks), the edits as
7`onestore::op` ops in publication batches, and picture and attachment bytes once
8each by hash. Local success does not acknowledge publication to a shared notebook.
9
10```text
11editor ── Edit ──► section thread ─ Section::apply, INSERT edit ─┐
12 (the parsed section: base + sealed + open) │ one fsync per burst
13sync thread: remote.stamp() == base ? publish(sealed batch) ──────┤ base chunks += transaction
14 otherwise read once, replay the queue on it ─────────┘ base := remote, conflict pages queued
15```
16
17## Edits
18
19`Replica::apply(author, edit)` applies an `onestore::op::Edit` to the section the
20queue leaves and returns its id once written; a refused edit returns `Rejected`
21and changes nothing. `page(space)` and `pages()` read what the queue leaves, for
22opening and reloading. Edits collect in an open batch until the sync thread seals
23it, so edits arriving while a publication is in flight publish together.
24
25```no_run
26use onestore::{ExGuid, op::{Edit, Op, PageOp}};
27use notebook::Replica;
28# fn example(path: &std::path::Path, source: &[u8], space: ExGuid, text: ExGuid)
29# -> Result<(), Box<dyn std::error::Error>> {
30let cache = Replica::create(path, source)?;
31let typed = PageOp::Text { text, range: 0..0, with: "Hello ".into() };
32let id = cache.apply("Author", Edit { at: 133_000_000_000_000_000, ops: vec![Op::Page { space, op: typed }] })?;
33drop(cache);
34
35let reopened = Replica::open(path)?;
36assert_eq!(reopened.pending()?.last().map(|edit| edit.id), Some(id));
37# Ok(())
38# }
39```
40
41A move across sections reads the page with `moving(space)`, together with its conflict pages
42and versions, which no copy carries; copies it there; then retires it here with
43`retire(author, edit, moving)`, which applies the edit only while all of them are as read,
44checked on the section thread in order with every other edit, and refuses it as `Rejected`
45otherwise, so an edit made since the copy keeps the page.
46
47Page reads never wait for the sync thread: a rebase or reread of the queue runs on a
48new thread while the current one keeps answering `page` and `pages` from the section as
49it was, then hands its requests over; edits sent meanwhile apply to the rebuilt section.
50
51Share one `Replica` between threads. Keep the cache on a local filesystem: the
52connection holds exclusive ownership between transactions, and a second open fails
53busy. No network wait occurs in a local edit. An edit a session submitted that the cache
54cannot write for now (a full disk, an I/O error) stays applied in memory and is written
55again, in order, each second; meanwhile `written` returns `WouldBlock`. A replica dropped
56meanwhile keeps its cache owned and its edits retrying until they are written or the cache is
57deleted, and only then releases it; `notebook::unwritten()` counts the edits waiting in the
58process, for a host to ask before it quits. After a commit fails, a fresh connection
59decides whether it landed. Until the cache reopens (too many open files, a busy cache), the
60edits wait the same way, reads of the cache answer `WouldBlock`, and the cache stays owned:
61the replica locks `<cache>-lock` and lists the cache for the process, and every open fails
62busy while it does. A cache gone or no longer a database refuses every edit waiting, as an
63error, and stays closed. An edit a caller blocks on, or one the cache could never write, is
64refused at once, and the section thread rereads the queue; while the cache is closed, that
65reread waits for it, page reads answer from the section as it was, and later edits wait to
66apply to the reread section; the session then reports the section's pages as `Changed`.
67
68## Sessions
69
70`session::Notebook::open(root, cache_dir)` discovers a notebook directory, reading only
71the files its folders list otherwise than when last opened (`discover::Cache`, kept under
72`cache_dir/listings`), and
73`session::Section::open(file, cache_dir, offline, notify)` opens one section file through
74a replica named by the file's location and the section's document identity, so the same
75file reopens the same queue after a relaunch and a copy of it elsewhere has its own
76(`notebook::location`: `folder`, `local`, `smb`, `moved` for a notebook the app moves,
77and `forget` for one it deletes).
78`Notebook::section(path, offline, notify)` opens a mounted notebook's section; a replica already
79named by the identity its discovery read resumes without reading the file, which the
80worker checks next, as on a share. `offline` seeds Work Offline before the worker starts;
81explicit `wake()` still synchronizes while offline.
82A section is `Send + Sync`: `apply(author, edit)` returns at once and reports a refusal as `Event::Rejected`; `events()` drains
83remote changes (`Changed(spaces)`), publication outcomes, unreachable files and
84failures; `notify` runs on a background thread whenever an event waits.
85`conflicts()` lists each page's conflict pages, which `page` reads and `delete_pages`
86removes; `release(id, archive, Resolution)` ends an uncertain attempt. The author an
87edit names is the host's: the app passes the account's full name.
88`import_page` creates a page holding a copy of another; `delete_pages` removes pages.
89`set_pause(pause)` has edits publish once `pause` passes without another (at most 30 s after the
90first waiting), as a notebook on a cloud drive does; `wake()` publishes them at once.
91`sync_status()` gives when the section file was last reached, why it could not be since
92and how many edits wait for it. `set_offline(true)` works offline as OneNote does: the
93worker stops connecting and edits queue until `wake()` (Sync Now) or `set_offline(false)`.
94
95`session::Background` keeps the sections no session holds in sync, as OneNote 2010 keeps
96every section of an open notebook: `Background::smb(root, limit, connect, notify)` on a
97share, `Notebook::background(watched, copies, notify)` for a mounted notebook, then
98`watch(notebook.replicas())`. On a share, one CHANGE_NOTIFY on the notebook's folder
99(`smb::Client::watch`) reports what changed; a host watching a mounted folder passes the
100changed paths to `touched`, and says whether it reports every change as `watched`. A
101reported section is checked a second later; a reported folder is listed a second later and
102only its sections listed otherwise are checked. A failing section is tried every 31 seconds,
103and any other every `Background::BACKSTOP` while a watch reports, `UNWATCHED` otherwise. On
104connecting, and on working online again, every folder is listed: a section whose file lists as
105discovery or the last check found it needs no reading, the others are checked one at a time
106100 ms apart. A check reads the file's stamp; a section whose replica has edits waiting, or
107whose file moved past the replica's base, has the replica opened for the synchronization steps
108that publish or rebase it and closed again, and a section without a replica gets one from the
109file, its offline copy, on a share or with `copies`. `hold(path, section)` leaves an open
110section to its session: its worker no longer polls while a watch reports, the watch wakes it
111instead, and once its replica is released the background checks the file at once. A replica
112a session holds is otherwise skipped (`Error::busy`), so opening a section may wait out one
113step. `status()` gives each section's `SyncStatus`, `changed()` the sections another client
114changed, `wake` and `set_offline` follow Sync Now and Work Offline, and `discard` stops the
115thread and deletes the replicas holding nothing unpublished, as when the notebook closes.
116`Notebook::replica_path(path)` names a section's replica for either kind of notebook.
117
118`Section::resume(file, replica, offline, notify)` starts from an owned `Replica` without
119consulting the remote file; with the `smb` feature, `Section::resume_smb(path,
120replica, limit, connect, offline, notify)` binds a share-relative path, `connect` running on
121the worker again after transport failure.
122
123## Reconciliation
124
125`sync_once(&mut remote)` returns what one step did as `Synced { edit, changed }`:
126the state the step's batch reached, named by its newest edit, and the pages a remote
127change replaced. While the remote's stamp (header and length) equals the base's, the
128oldest sealed batch publishes as its `Transaction`, and the base's chunks take its
129writes; nothing is read. When the stamp moved, the remote image is read once and the
130queue replays on it (`merge.rs`): an op whose objects the remote left alone applies
131as it is; a text op shifts past the remote's changes to the same text when its
132range stays clear of them; a move, deletion or property the remote already made is
133done; a page the remote already holds as the local edits leave it drops their ops.
134Anything else conflicts, as in OneNote 2010: the op is dropped with every later op naming
135what it named, the remote's version stays the page, and the local version becomes a
136conflict page under it (`SectionOp::Conflict`, queued as one more edit and published with
137the rest), its conflicting objects marked, named for the author of the first edit that did
138not replay. Page lists merge as OneNote 2010 merges page series: a page the remote moved
139keeps the remote's place, a page moved here goes before the next page in the local order
140the remote left in place, or last, and a page the remote removed comes back as a new copy
141of the local one, placed the same way (`corpus/conflict-page/native-pages`,
142`native-restore`). Nothing blocks the queue.
143
144A remote a file provider keeps may list versions of the file beside it (`Remote::versions`,
145`version`, `retire`), as iCloud Drive keeps a commit that lost to another device's. Each step
146merges them first (`resolve.rs`): per object space, the newest revision of the version's that
147the file holds, or holds merged, is where the two last agreed, or failing that (a chain that
148stops at a checkpoint) the newest of the file's that the version holds or that merged one of the
149version's; the section opened at those revisions (`onestore::Section::open_at`) is the base the
150version's changes lower from (`lower_page`), and they replay on the file through the rebase
151rules above, conflict pages where they clash. A page whose two copies share no revision stays
152the file's, the version's kept beside it as a conflict page
153(`corpus/icloud-merge/first-edit-checkpoints`), and a page list with none counts as unchanged.
154The revisions the merge writes are named after the version's it merged and the file's they
155depend on (`Section::seal_as`), so a version merged once, here or on another device or by an
156older build, merges as nothing again, and merges of one version onto different states never
157share a name. It is published on the file's stamp, then retired; one of another section, or one
158that cannot be read, is retired with `keep` for the remote to keep beside the file.
159A provider can also lose an upload outright, another device's landing over it with no version
160kept. A remote keeps what it publishes with `published::Published` (`keep` after each
161publication) and lists each image the file no longer stands as among its versions
162(`Published::versions`): one the file holds merges as nothing and is dropped, one it lost
163merges back.
164
165Publication attempts are recorded before network I/O. An attempt confirms only
166when the remote holds its revisions, or every page it changed as it changed them,
167followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation`. A remote that
168accepts edits (a Live Share host) lands a batch at most once, a protected section's
169transaction included, so the next step sends it the same batch again; one it refuses as
170`StaleNetworkFileHandle` is acknowledged where the image holds its revisions, else rebased.
171Any other remote never gets it again unreviewed.
172`release(id, archive, Resolution)` exports the queue first, then publishes the batch again
173(`Mine`) or abandons every unpublished edit (`Theirs`).
174A durable `Published` receipt survives reopening.
175
176A schema-14 cache is converted when opened: it is exported to
177`<cache>.v14-recovery`, each queued page is lowered against the page the conversion
178has so far, and every converted page must equal the page in the old working image,
179or the conversion rolls back and the open fails naming the archive; a page it held in
180conflict becomes a conflict page. A schema-15 cache's batches lose the review conflict
181they recorded when opened.
182
183With the optional `smb` feature, `SmbRemote::new(client, path, limit)` binds an
184`notebook::smb::Client` to one share-relative file and snapshot limit. Remote identity uses the logical root
185object space, which survives the tested native compaction that replaces the file ID.
186
187An `Arc<Replica>` can own one background worker. Supply a connection factory, poll
188interval and observer; successful publications drain immediately, durable local
189edits wake the worker, and `wake()` requests an immediate reachability retry.
190While nothing is queued, or the queue waits on a remote that has not changed since,
191a remote whose `Remote::stamp` holds is not read again; a worker a `Background` holds reads
192the stamp only when the notebook's watch wakes it.
193Transport failures discard the old connection and retry through the factory;
194Read contention and `NotCommitted` operations with `WouldBlock` or `ResourceBusy`
195reuse the connection. Contended `NotCommitted` operations use randomized backoff,
196capped at one second, to separate competing retry cycles. Cancellation interrupts this delay; local wake notifications
197remain coalesced until its end.
198`RemoteIo` distinguishes connection/read failures from
199local `Io` errors. Cache/document errors stop the worker.
200
201```no_run
202# #[cfg(feature = "smb")]
203# fn example(cache: std::sync::Arc<notebook::Replica>, username: String, password: String)
204# -> Result<(), Box<dyn std::error::Error>> {
205use notebook::SmbRemote;
206use notebook::smb::{Client, Credentials};
207use std::time::Duration;
208
209let worker = cache.start_sync(
210 Duration::from_secs(2),
211 move || {
212 Client::connect(
213 "server:445", "notes",
214 Credentials { username: &username, password: &password, domain: "" },
215 Duration::from_secs(5),
216 ).map(|client| SmbRemote::new(client, "Personal/Video.one", 64 * 1024 * 1024))
217 },
218 |result| {
219 if let Err(error) = result { eprintln!("{error}"); }
220 },
221)?;
222// Retain `worker` while synchronization should run; local edits wake it automatically.
223worker.stop()?;
224# Ok(())
225# }
226```
227
228`stop()` cancels future steps and joins the worker, returning a fatal cache error
229or worker panic. Dropping it requests cancellation without waiting. Remote
230operations and callbacks must have bounded execution times if shutdown latency
231matters. Credentials belong to the factory, not the cache database.
232
233`create` refuses an existing path; `open_or_create(path, key, source)` opens the cache at
234`path`, reading `source` only to make it where there is none, and opens one another thread
235made meanwhile. Opening validates database integrity and replays
236the queue on its base. SQLite runs in WAL mode under exclusive locking with FULL
237synchronization and fullfsync, each queried back: every commit is durable, and the only
238file beside the cache is `<cache>-wal`, which holds committed pages until a checkpoint
239and must travel with the cache when it is copied. Recovery archives are single files.
240The cache contains notebook content.
241
242The SMB-enabled `smb_offline_client` example is an owned-lab workload for
243`tools/native_collaboration.py --offline --embedded-smb`; its append-specific
244conflict policy lives in the client.
245
246## Recovery archives
247
248`export_recovery(new_path)` captures the base and remote images, the edit queue,
249uncertain attempts, receipts, downloaded media and the edit-ID sequence in one
250SQLite snapshot. It refuses existing destinations and leaves the live queue
251unchanged. Export to a local directory from a background thread: copying holds
252the cache mutex while capturing the database. Failure after the final rename can
253leave a complete archive at the requested path; it never acknowledges a remote
254edit. Archives contain notebook content and use a separate database identity, so
255`Replica::open` rejects them as writable caches.
256
257```no_run
258use notebook::{Recovery, Replica};
259# fn example(cache: &Replica) -> Result<(), Box<dyn std::error::Error>> {
260cache.export_recovery("review.sqlite")?;
261let review = Recovery::open("review.sqlite")?;
262let counts = review.summary()?;
263let local = review.snapshot()?;
264let remote = review.remote_snapshot()?;
265let pending = review.pending()?;
266let receipts = review.receipts()?;
267# Ok(())
268# }
269```
270
271`Recovery` provides read-only inspection and no synchronization or restore method.
272`status(id)` preserves the same state interpretation as the live replica.
273`recovery_summary()` on the live replica and `summary()` on an archive return
274counts and byte sizes without notebook text, paths, authors or credentials.
275Opening an archive validates its schema and images without migration. A recovery
276archive is evidence for a reviewed recovery decision, not a second active queue.
277
278## Downloaded media
279
280`fetch_asset(source, section, filename, limit)` resolves a declared external
281payload through `notebook::discover::Source` and durably caches its exact bytes.
282The section path is relative to the source root; the filename comes from a
283`FileDataReference::External` in the retained working or remote image. Local and
284SMB sources use the same API. Downloads release the cache mutex during network
285I/O and recheck the reference before committing; edits can continue meanwhile.
286
287`cached_asset(filename, limit)` reads previously downloaded bytes without network
288access. `None` means never downloaded; `Some(Vec::new())` is a downloaded empty
289payload. Each read checks the stored SHA-256 and enforces the byte limit before
290loading the payload. Cache contents describe the prior download, not current
291server reachability or presence. A failed fetch returns its error without
292silently substituting cached bytes. Different bytes for an already cached file
293identity return `AssetChanged` and preserve the previous download.
294
295Downloads do not change pending edits, publication attempts or receipts. Recovery
296archives include cached media and expose the same bounded `cached_asset` lookup.
297
298## Discovery
299
300`notebook::discover` provides read-only notebook discovery over a caller-supplied
301root, keeping directory traversal out of the single-file storage parser.
302
303Discovery returns ordered sections and nested groups with file identities and
304share-relative paths. Section display-name overrides remain distinct from file
305names. TOC references whose identities are absent from the directory remain
306inspectable; a cached filename never substitutes for an identity match.
307Reserved `_onefiles` directories are excluded from section-group traversal;
308OneNote's `OneNote_RecycleBin` is listed as the section group OneNote shows.
309Dot files and folders (`.DS_Store`, AppleDouble `._` companions, `.snowbound`) and Office's
310`~$` owner files are skipped wherever they are, never read as sections.
311Encrypted sections and valid storage with an unreadable document graph retain
312their identity as `Locked` or `Unreadable`, without being presented as empty pages.
313A child folder or section file that is denied or gone while listing (a folder
314another client holds delete-pending reads as denied) is kept as `unavailable` and
315tried again on the next discovery; the root itself, malformed storage, other failed
316reads and ambiguous identities reject the discovery.
317
318`Cache::discover` keeps each file's listing (size and last write time), stamp and what
319discovery took from it; the next discovery reads only the files listed otherwise, taking one
320from a copy where `Source::copy` has it as it stands (on a share, a replica whose base has the
321file's stamp), and `found(path)` gives a section's listing and stamp for `Background::watch`.
322`take` hands on, once, the sections it read from the source, up to a memory bound, which
323`Notebook::replicas` gives `Background::watch` so that its first check need not read them
324again. A file written during discovery is read again next time; a failed discovery leaves the
325cache as it was.
326
327Each file read must be a consistent, bounded snapshot. The result is an
328observation across multiple files, not an atomic notebook transaction or
329authorization to publish an edit. Refresh rejects observed topology changes. Of
330several section files with one identity (a `Name 2.one` copy), each lists, as OneNote opens
331each; all but the one its folder's TOC names (else the one with the shortest path) are
332marked `copy`, have replicas of their own, and are never placed or listed anew by structure
333operations, which touch only a TOC entry OneNote gave the copy's name. Of several groups
334with one TOC identity, the one its parent's TOC names, else the newest, lists; the others
335list in `unavailable` as `Reason::Copy`. iOS's
336`.Name.one.icloud` placeholders, and macOS's dataless files, list there under their real names
337as `Reason::Evicted`, unless listed unchanged since discovery last read them.
338A file listed but not reachable, as a cloud provider offline lists from its cache, stands as
339it was last read, and is read again next time; one never read fails the discovery.
340Retain the last accepted catalog if discovery fails; a connection failure does not mean
341files were deleted.
342
343`read_external_asset` resolves a validated UUID `.onebin` filename beneath the
344selected section's sibling `_onefiles` folder and returns exact bounded bytes.
345Missing files, permissions and size failures retain their I/O error kinds; an
346empty payload is a successful empty buffer. Embedded payloads remain available
347directly from the core document model.
348
349## Notebook structure
350
351`Notebook::create`, `create_section`, `create_group`, `rename`, `move_entry`,
352`set_section_color`, `set_color`, `reorder` and `delete` change a notebook the way OneNote does: the
353table of contents (`Open Notebook.onetoc2`, created when a folder has none) gains,
354renames, reorders or loses entries; a new notebook holds "New Section 1", and a new
355section an empty section's file plus the page its `PageCreation` makes, in OneNote's
356new-section colour order; a section's colour lives in its own metadata, the notebook's
357in its root table of contents (`corpus/section-color`); a deleted
358section moves into `OneNote_RecycleBin`, a group with its own TOC, and a deleted group's
359sections move there too before its folders go. `recycle_pages` keeps copies of pages a
360section is about to delete in `OneNote_RecycleBin/OneNote_DeletedPages.one`, with their
361identities, titles, dates and creation times, as OneNote does
362(`corpus/notebook-management`); `unrecycle_pages` takes them out again by identity, as
363OneNote's Undo of a page delete does. A bin or deleted-pages file its TOC does not list is
364placed and listed again, a bin whose TOC OneNote named otherwise keeps it, and an
365unavailable bin refuses the delete (`corpus/recycle-bin-repair`). `empty_recycle_bin` is OneNote's
366Empty Recycle Bin: Deleted Pages loses every page in one revision and each binned section's
367file goes, the bin's TOC left as it was (`corpus/recycle-bin-view`). An entry a TOC still
368lists for a file gone from its folder gives way to the file an edit gives its name.
369`rename_folder` renames the notebook's own folder, which OneNote 2010 opens again from its
370new name with no file inside changed (it drops the old one from its list): on a share it
371first takes and lets go of OneNote's writer locks on every section and TOC, refusing with
372`WouldBlock` while another writer holds one, then the replicas and listing follow.
373Edits keep entries in the order their ordering numbers give; a rename or colour keeps
374the numbers. Every created, renamed or moved file is placed with
375`onestore::place_file`, which sets the header's ancestor to the parent TOC's
376identity and the name CRC OneNote checks on open; a file without them is
377re-identified and listed anew. They run over `Storage`: a mounted directory
378(`Notebook::open`) or an SMB share (`Notebook::open_smb` with a
379`smb::Client`, which gained create, directory creation, rename, delete and
380header placement under native writer coordination); `tools/test_smb_structure.py
381VM OUTPUT` drives them against a disposable Samba lab VM.
382
383`copy_section(path, replica, target, folder, after)` and `move_section(path, section, target,
384folder, after)` are OneNote 2010's Move or Copy into another notebook (or this one opened
385again): the section is written anew under new identities (`onestore::protected::rekey`), named
386as it is or with the first free number after ("Beta 2"), with its colour, pages, versions and
387conflict pages and the edits its replica holds, and listed in `target`'s folder after `after`
388or last once it reads back whole, written until then under a name discovery passes over. A
389copy that cannot be listed is deleted again. A move stops `section` first, then sends the
390original to its notebook's recycle bin with `Storage::rename_unchanged`, under the writers'
391coordination provided its stamp is still the copy's, and deletes its replica; where the copy
392landed but the original changed since or cannot reach the bin, it stays and `Moved::kept` says
393why. Queued edits waiting to rebase onto a file another client changed, or a file that can't
394be reached, refuse both.
395
396`Notebook::find_page(url)` resolves a stored internal link to a section path
397and page space by identity: the linked section first, then every readable
398section, so a link follows its page when the section is renamed or moved and
399when the page itself was moved to another section. Other URLs and unknown
400targets are `None`.
401
402`Notebook::read_section(path)` reads a section file as stored, without opening a
403replica, and `session::stored_pages(image)` builds its pages with each page's
404`LastModifiedTime` and the author of its latest change: the app's search indexes the
405sections it has not opened this way, and unread changes leave out the reader's own.
406
407A password-protected section lists as `Locked`. `Notebook::unlock(path, password)` opens
408its key (a wrong password is `Error::Protected(PasswordMismatch)`), and
409`section_unlocked(path, &key, offline, notify)` (`section_unlocked_with`, or
410`Replica::open_or_create(path, Some(&key), source)` for a host that opens replicas itself)
411opens it as any section opens: edits queue, publish, merge and become conflict pages as an ordinary section's do,
412each revision sealed under the section's key. The replica keeps none of it in the clear:
413its base and transactions are the file's own ciphertext, and its queued edits and payloads
414are sealed with AES-256-GCM (a random nonce each, the row's kind as associated data) under
415a key HMAC-SHA256 derives from the section's, payloads named by HMAC rather than SHA-256.
416Only the author's name, payload names and revision identities stay readable, and the
417queue opens only under the key (a recovery archive with `Recovery::open_unlocked`).
418`stored_pages_unlocked(image, &key)` reads a protected section's pages for search. The
419background never opens a locked section.
420
421`Notebook::set_password(path, key, password)` sets, changes or removes a password as
422OneNote 2010 does: the section is written anew under new identities
423(`onestore::protected::rekey`), next to the file as a dot file that `Storage::supersede`
424puts in its place under the writers' coordination while the file's stamp holds (on a share as
425OneNote's maintenance does it: the old file aside, the new one in, the old one deleted), and
426its TOC entry takes the new identity (`TocEdit::Reidentify`). The old replica, a cache of the
427old file, is deleted, so the section's session must be closed and its edits published first;
428a section whose edits wait refuses.
429
430Another device learns of the password when it reads the notebook again (`open`, `refresh`):
431a replica of the file the protected section superseded, found by the placement the rewritten
432header keeps, is deleted when nothing waits in it. Edits that wait stay until `unlock` on that
433device queues them under the key, each page they changed coming back as a copy, as a page
434another client removed does. A session open meanwhile stops as `SyncState::Protected`.
435
436`Section::import_page(page, author)` copies a page, usually read from another
437section, to the end of this one as one `SectionOp::Import` queued like the
438user's own edits, under fresh identities (`Page::copy`); payloads travel with
439the model, so the copy publishes offline later like any edit. Content outside
440the model refuses to copy. `session::copied(page, author)` is that op, for a replica held
441elsewhere. A move is an import here followed by
442`Section::delete_pages` there.
443
444`Notebook::refresh` rereads the directory on the caller's schedule and reports
445what another client changed as `Change`s keyed by file identity: a renamed or
446moved section is `Moved`, not removed and added, and a deleted section moves
447into `OneNote_RecycleBin`; a folder whose surviving entries changed sequence
448is `Reordered`. A failed read returns the error and keeps
449the previous catalog, so an unreachable share never reads as an emptied
450notebook.
451
452## Packages
453
454`package` reads and writes OneNote packages (`.onepkg`), cabinet files of a notebook's tables
455of contents and sections at their paths in its folder. `notebook_files(notebook, image)` gathers
456what OneNote 2010's Save As packs (each folder's TOC and sections, groups after, the recycle bin
457and copies left out, its TOC entry kept), taking a section the caller holds edits for from
458`image` (a replica's `snapshot`); `pack` writes them MSZIP-compressed, each file outside any
459notebook (no `guidAncestor` or `crcName`), as OneNote's are. `read` takes MSZIP or OneNote's LZX,
460refusing a path leaving the folder or more than a byte limit unpacked, and `unpack(files, root)`
461writes a new folder as Unpack Notebook does: every file takes an identity of its own
462(`onestore::reidentify`) and is placed under its folder's TOC, whose entries follow it where
463OneNote lists each anew beside the stale one. `section_copy` and `page_section` are Save As's
464section and page: a copy under a new identity, and a new section holding the page with its
465identity, both outside any notebook (`corpus/notebook-package`).
466
467## Snowbound's folder
468
469`sidecar` keeps Snowbound-only data in the notebook's `.snowbound` folder, which
470OneNote 2010 skips because it has the Windows hidden attribute: `Storage::hide` sets it
471on a share with SET_INFO FileBasicInformation (always, since Samba reports a dot name hidden
472without storing it), `SetFileAttributesW` on Windows, or macOS's
473`UF_HIDDEN`, which an smbfs mount passes to the share; Linux keeps no such attribute. Discovery skips dot names, and a notebook
474without the folder is whole.
475
476```text
477.snowbound/
478├ tags.json [{ name, shape, art, mapped }]: a tag's name and symbol → its art
479├ tags/ <SHA-256>.png | .svg, written through a temporary name, never rewritten
480└ themes.json { themes: [{ id, name, styles, modified, deleted }],
481 assignments: [{ scope, theme, assigned }] }: style themes and who wears them
482```
483
484`Notebook::map_tag_art(name, shape, bytes, extension)` makes the folder and hides it,
485keeps the picture, then rereads `tags.json`, merges its mapping and replaces the file,
486reading it back and merging again until its own holds. `sidecar::merge` keeps every tag
487either side mapped; one tag mapped by both keeps the later `mapped`, then the greater art
488name. A mapping two writers replace within one round trip of each other can still lose
489one side's new tag until that side maps it again. `tag_art` reads the mappings, passing
490over entries that name no picture, and `tag_art_file` returns a picture once its bytes
491match its name.
492
493`sidecar::themes` keeps style themes: each gives OneNote 2010's eleven gallery styles
494(`STYLES`: `h1`…`h6`, `PageTitle`, `cite`, `blockquote`, `code`, `p`) a font, size,
495weight, slant, colour and spacing, and `Theme::sheet` turns them into the paragraph style
496definitions a page stores. `built_in()` lists the shipped themes, which never change once
497shipped. A scope is the notebook, a section by file identity or a page by its
498notebook-management identity; `Themes::effective(section, page)` takes the page's, else the
499section's, else the notebook's. `Notebook::themes` reads the file and `save_themes(change)`
500merges a change in as `map_tag_art` does: themes by id and assignments by scope, the later
501timestamp winning, then the greater entry. `onestore::op::restyle(page, sheet)` gives the
502`Restyle` ops that put a page in a sheet.
503
504## SMB (feature `smb`)
505
506`notebook::smb` provides blocking SMB access for OneNote sections and
507table-of-contents files. The core `onestore` crate remains independent of network runtimes. This is an
508experimental Rust API with native interoperability evidence in the repository's
509[Milestone 9](../../evidence/MILESTONE9.md).
510
511```no_run
512# #[cfg(feature = "smb")] {
513use notebook::smb::{Client, Credentials};
514use std::time::Duration;
515
516let client = Client::connect(
517 "server:445",
518 "notes",
519 Credentials { username: "user", password: "password", domain: "" },
520 Duration::from_secs(5),
521)?;
522let snapshot = client.read("Personal/Video.one", 64 * 1024 * 1024)?;
523let store = onestore::Store::parse(&snapshot)?;
524let revisions = onestore::RevisionIndex::parse(&store)?;
525let document = onestore::document::Document::parse(&revisions)?;
526# }
527# Ok::<(), Box<dyn std::error::Error>>(())
528```
529
530Paths are relative to the share. The read limit bounds the complete physical
531snapshot. Call from a background thread outside a Tokio runtime.
532`Client::commit_transaction` publishes a `Transaction` from `Section::seal`; its errors
533retain `onestore::CommitState`. `Client::stamp` reads a file's header and length in one
534round trip without coordination, for polling. `Client::confirm` checks and flushes a
535stamp, then refreshes its header version metadata without adding a revision. The caller
536must first establish which intents that image contains and reread before another commit.
537
538A commit is eight round trips while its appended bytes and patches fit fifteen 64 KiB
539writes: open; the coordination locks, identity queries and the stamp check's reads as one
540compound request; the path's identity; the appended bytes and patches, the header, the
541counter (twice at a carry) and the version, each as its writes and a flush in one
542compound; close. The server
543performs a compound in order and answers its flush once the writes are durable, so the
544writes carry no write-through flag.
545
546Readers use shared native guards while writers publish under native write-open
547and byte-lock exclusion. Maintenance is excluded during each operation; pathname
548identity is checked after acquiring the guards. Connection loss retires the
549client. Reconnect for subsequent operations, and reconcile an `Unknown` edit
550before retrying it. The transport does not automatically replay requests.
551
552`Client::read_dir(path, entry_limit)` enumerates a directory, including the share
553root with an empty path. It follows every response page and returns no partial
554list on interruption, entry-limit overflow or close failure. Entries retain exact
555Unicode names, observed sizes, last write times and MS-FSCC attributes, including
556directory/reparse flags. Concurrent directory changes are not an atomic snapshot; repeated names
557are rejected with `ResourceBusy`. Notebook identities come from the files, not
558directory names or sizes. Missing paths, denied access and non-directory paths
559have distinct I/O error kinds.
560
561`smb::shares(address, credentials, timeout)` lists the disk shares a server offers the
562account, over `IPC$` and srvsvc. `smb::Refusal::of(&error)` names why `Client::connect`,
563`shares` or `read_dir` failed as a person remedies it: unreachable, SMB1 only, sign-in
564refused, no such share, denied, no such folder. A server that turns SMB2's negotiation
565away is asked for SMB1's `NT LM 0.12` once, which an SMB1-only server agrees to.
566
567`Client::watch(path, changed)` arms one CHANGE_NOTIFY with WATCH_TREE on a directory, as
568OneNote 2010 watches a notebook's folder, and keeps it armed on the client's runtime without
569probing the connection, so an idle watch sends nothing. `changed` hears each batch of changed
570paths relative to the directory, `""` when the server lost count, and an error when the watch
571ends with its connection.
572
573An external payload on a share (`discover::Smb`) is read under a read-only share handle that excludes writes and deletion. Empty files succeed; limits,
574interrupted reads and failed close never return partial bytes. This payload read
575does not parse a OneStore header or acquire its reader-coordination bytes.
576
577`python3 tools/test_smb_directory.py VM OUTPUT` checks a caller-owned disposable
578Linux lab VM against its filesystem listing and interrupts directory requests,
579responses and close. It creates synthetic files in that VM; the caller retains
580responsibility for VM teardown. The parser also has bounded-record/truncation
581tests independent of the server.
582
583Device and simulator builds link for iOS. Native acceptance uses disposable
584OneNote 2010 clients and Samba; it does not establish on-device execution or
585physical power-loss durability.
586
587## Document providers
588
589`provider` opens notebooks a document provider keeps, as Android's Storage Access Framework
590reaches them. The host implements `provider::Provider` (a document's description, a folder's
591children, `open` in a `ContentResolver` mode as an owned `File`, `close` with the provider's
592verdict, `rename` returning the new id) and reports an unreachable provider as `NotConnected`,
593a revoked grant as `PermissionDenied` and a gone document as `NotFound`.
594`provider::Documents::new(provider, root, cache, device)` holds one root, the app's
595cache and the device's name: a picked folder opens with `Notebook::open_provider(documents)`, its sections with `Section::resume_provider`
596and the rest with `Background::provider`, as a Live Share guest's do; a picked document opens
597as a lone section with `Section::open_provider`. Catalog paths resolve to ids by listing their
598folders, again once when an id has gone, as a rename leaves it; of documents sharing a name the
599first by id lists and the rest are `unavailable`, named with their ids.
600
601A publication resolves the document's id once, from the last listing, and writes nothing to
602a document that took its name since. Without a write grant a document is only read, and
603publication is refused as `ReadOnlyFilesystem`; where its `r` descriptor seeks, `rw` opens it
604and a commit checks the stamp and appends in place exactly as on a disk; a document that only
605streams is read whole for each stamp, and a commit reads it, checks its stamp, applies the
606transaction and writes the whole image with `wt`, or `w` where that is refused, images only
607growing. No provider offers a lock or compare-and-swap: the check sees only this device's view
608of the document, and another device's upload can replace this one's. So each image a section
609publishes is kept in the cache, beside its replica: once written in place, as the image it was
610published on (kept the same way) with the transaction applied, else read back; once a whole
611upload has opened, as uploaded. Once the document is seen otherwise it is listed as a
612`Remote::versions` version, until the document holds the image's revisions: a lost publication
613merges back as a conflict version does, conflict pages where it clashes, named for the device
614`Documents::new` was given. A document still standing as the image an upload was made on is
615given a minute for reads to catch up before the upload counts as lost. Only this device's
616publications are kept so, and one that cannot merge is set aside in the cache, as no new
617document can be made beside it. A document read torn as this device's own upload cut off on a
618provider that writes in place leaves it, the start of an image it kept, is written again from
619that image, once a step; any other image that does not parse is left as it is, and is never a
620base. An open
621section checks its file every `Background::UNWATCHED`, the host's change reports waking it
622between. While the provider can't be reached the notebook opens as its folders were last
623listed and sections from their replicas; a revoked grant keeps them, and the notebook does not
624open until access returns. Creating, deleting and moving documents and renaming the folder are
625`Unsupported`; renaming within a folder is not.
626
627## Live presence and Live Share (feature `live`)
628
629`live::Live::start(hello, room, reach, relay, events)` listens on a TCP port and, with a
630`Reach`, advertises `_snowbound._tcp` by mDNS on every network or on loopback alone,
631connecting to the peers in the same `Room` that it finds: a notebook's or a share's random
632secret (`Room::Notebook`), or a code typed on both (`Room::join("7KQ-4MZ-9XR", password)`).
633With a `relay` (`wss://live.example.net`, `crates/relay`) it also joins the room there and
634meets its peers through it; the end sharing a code (`Room::share`) has the relay number its
635secret, `code()` then has the whole code, and it burns a code after too many wrong tries.
636Codes are Crockford base32 (`live::code`): two symbols numbering the room, six of secret
637(30 bits) and a check symbol that refuses a typo before it spends one of the relay's tries.
638`connect(address)` meets a peer discovery did not find. Peers meet through SPAKE2 on the
639room's secret, then every frame is AES-256-GCM under the keys it agreed: its number, which is
640also its nonce, then a message kind and a CBOR map (`live::wire`). A frame lost, repeated,
641reordered or forged on the way fails where it lands; the connection is dropped as broken,
642nothing from it after the fault is applied, and the ends meet again from scratch. A reader
643skips kinds and map keys it doesn't know, so later versions add both freely.
644`set_presence` says which section, page and caret this end has (text object and UTF-16
645offset, as ops address text); a connection sends only the newest. `peers()` lists each
646connected peer's `Hello` (name, picture) and presence; `events` hears when they change, who
647was met and left, and every other frame with the `Line` to answer on. `leave(reason)` says
648`Bye` first; dropping the `Live` leaves at once. `Notebook::presence_room()` is the secret of
649a notebook's presence room, kept in `.snowbound/live.json` and made where it has none.
650
651`live::share` is Live Share. `Host::start(storage, hello, sharing, name, reach, relay, events)`
652serves `Notebook::into_storage()` to the peers in the share's room and welcomes whoever knows
653`Sharing::code` (and its password) from the code's room; `code()` replaces a burned code with
654a new secret, `guests()` lists who is connected, `touched(paths)` passes the host's own changes
655on, and `stop()` lets every guest go. `join(hello, code, password, reach, relay)` returns the
656`Welcome` (the share and its secret), or a `Refusal` saying why not: a wrong code, no one
657sharing it, an expired code, too many wrong tries, or no relay. `Guest::start` joins the share;
658`Notebook::open_hosted(guest, cache)`, `Background::hosted(guest, notify)` and
659`Section::resume_hosted(path, replica, guest, offline, notify)` then work as on a share, with
660`Guest::host()` and `stopped()` for whether the host is there and still sharing.
661
662## Queue measurement
663
664`cargo run -p notebook --release --example queue_scale -- NEW_DIRECTORY 1000`
665measures alternating-page edits, cache reopen, recovery export and one local-file
666publication of the queue. `keystroke_probe PARAGRAPHS KEYSTROKES` types into a large
667page and reports the bytes the cache and the section file write per keystroke.