authoravatarclover caruso<git@paperclover.net>2026-10-19 11:52:09-07:00
committeravatarclover caruso<git@paperclover.net>2026-10-19 23:57:03-07:00
log18a89c7d5115a98335626904ed081efcded373bb
tree95effb331b6d64071530c8cbccf65105f3bcf606
parent5ef7269b5da51f5c9173f07bc7a7587fe5f46c83
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

fix(live): a guest replays a batch its host may not have heard

A guest's batch whose request was in flight when its connection dropped ends uncertain, as the host may or may not have committed it. The queue then held that batch until someone reviewed it, but nothing in the app offers the review, so unless the host had in fact committed it, the section published nothing more and took in no one else's changes for good. A Live Share host lands a batch at most once: it applies none whose revisions the section holds, or which it landed itself since it started, as OneNote's Optimize drops a revision a later one supersedes (the page-versions corpus shows it), and it seals any merge under the guest's names. So the guest now sends an uncertain batch again on its next step, whether the section is open or kept in sync from the background, and the host applies it or confirms the copy it already holds. The batch stays uncertain until the host takes it; a refusal while OneNote holds the section only waits for the next step. Other remotes, whose commits a late write could repeat, still never replay one. Tests lose a batch's request together with the guest's relay connection and check that it lands once the guest is back, after the host refused it once while the section was held, and from the background for a closed section; a unit test drops a landed batch's revision as Optimize does, after the host deleted what it typed, and checks that sending it again brings nothing back. The cold-restart test now expects a batch sealed before its process died to publish without review. - A Live Share guest's edits sent just as its connection dropped now publish once it reconnects, instead of holding up every later edit in that section. Assisted-by: claude-opus-5.5

11 files changed, 410 insertions(+), 70 deletions(-)

arc/sync.md+5-2
...@@ -128,8 +128,11 @@ section....@@ -128,8 +128,11 @@ section.
128Uncertainty is handled explicitly. A publication attempt is recorded before128Uncertainty is handled explicitly. A publication attempt is recorded before
129the first network byte. If the reply is lost, the attempt confirms only when129the first network byte. If the reply is lost, the attempt confirms only when
130the remote holds its revisions (or every page it changed, as it changed them).130the remote holds its revisions (or every page it changed, as it changed them).
131Otherwise it waits as `AwaitingConfirmation`, and `release` resolves it131Otherwise it waits as `AwaitingConfirmation`. A Live Share host lands a batch at
132explicitly: it exports a recovery archive first, then republishes or abandons.132most once, as it looks for the batch's revisions before applying it, so the next
133step sends it the same batch again. Any other remote, where a late write could land
134a second copy, waits for `release`, which exports a recovery archive first, then
135republishes or abandons.
133136
134## Merging137## Merging
135138
crates/notebook/README.md+5-3
...@@ -164,9 +164,11 @@ merges back....@@ -164,9 +164,11 @@ merges back.
164164
165Publication attempts are recorded before network I/O. An attempt confirms only165Publication attempts are recorded before network I/O. An attempt confirms only
166when the remote holds its revisions, or every page it changed as it changed them,166when the remote holds its revisions, or every page it changed as it changed them,
167followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation` and is never167followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation`. A remote that
168replayed. `release(id, archive, Resolution)` exports the queue first, then168accepts edits (a Live Share host) lands a batch at most once, so the next step sends it the
169publishes the batch again (`Mine`) or abandons every unpublished edit (`Theirs`).169same batch again; any other remote never gets it again unreviewed.
170`release(id, archive, Resolution)` exports the queue first, then publishes the batch again
171(`Mine`) or abandons every unpublished edit (`Theirs`).
170A durable `Published` receipt survives reopening.172A durable `Published` receipt survives reopening.
171173
172A schema-14 cache is converted when opened: it is exported to174A schema-14 cache is converted when opened: it is exported to
crates/notebook/src/background.rs+13
...@@ -1105,6 +1105,19 @@ impl<R: Remote> Remote for Discovered<'_, R> {...@@ -1105,6 +1105,19 @@ impl<R: Remote> Remote for Discovered<'_, R> {
1105 self.remote.confirm(base)1105 self.remote.confirm(base)
1106 }1106 }
11071107
1108 fn accepts_edits(&self) -> bool {
1109 self.remote.accepts_edits()
1110 }
1111
1112 fn publish_edits(
1113 &mut self,
1114 transaction: &onestore::Transaction,
1115 edits: &[crate::PendingEdit],
1116 revisions: &std::collections::BTreeMap<onestore::ExGuid, onestore::ExGuid>,
1117 ) -> std::result::Result<(), onestore::CommitError> {
1118 self.remote.publish_edits(transaction, edits, revisions)
1119 }
1120
1108 fn versions(&mut self) -> io::Result<Vec<crate::Version>> {1121 fn versions(&mut self) -> io::Result<Vec<crate::Version>> {
1109 self.remote.versions()1122 self.remote.versions()
1110 }1123 }
crates/notebook/src/live/share.rs+8-1
...@@ -14,7 +14,7 @@ use crate::{Error, Result, background::Reports, discover, session::Storage};...@@ -14,7 +14,7 @@ use crate::{Error, Result, background::Reports, discover, session::Storage};
14use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction};14use onestore::{CommitError, CommitState, RevisionIndex, Stamp, Store, Transaction};
15use sha2::{Digest, Sha256};15use sha2::{Digest, Sha256};
16use std::{16use std::{
17 collections::{BTreeMap, HashMap},17 collections::{BTreeMap, HashMap, HashSet},
18 io,18 io,
19 path::PathBuf,19 path::PathBuf,
20 sync::{20 sync::{
...@@ -294,6 +294,9 @@ struct Served {...@@ -294,6 +294,9 @@ struct Served {
294 puts: Mutex<ByGuest<Vec<u8>>>,294 puts: Mutex<ByGuest<Vec<u8>>>,
295 guests: Mutex<BTreeMap<[u8; 16], Admitted>>,295 guests: Mutex<BTreeMap<[u8; 16], Admitted>>,
296 writers: Mutex<HashMap<String, mpsc::SyncSender<batch::Waiting>>>,296 writers: Mutex<HashMap<String, mpsc::SyncSender<batch::Waiting>>>,
297 /// The revisions guests' batches landed as, by page: OneNote's Optimize drops one a
298 /// later revision supersedes, and a batch sent again must still find it.
299 landed: Mutex<HashSet<(onestore::ExGuid, onestore::ExGuid)>>,
297 /// Hears the paths guests changed, as the host's own notebook should.300 /// Hears the paths guests changed, as the host's own notebook should.
298 host: Mutex<Option<crate::session::Listener>>,301 host: Mutex<Option<crate::session::Listener>>,
299 /// The share's room, to tell guests what changed.302 /// The share's room, to tell guests what changed.
...@@ -1848,6 +1851,8 @@ pub struct HostedRemote {...@@ -1848,6 +1851,8 @@ pub struct HostedRemote {
1848 path: String,1851 path: String,
1849 /// The stamp last asked for, which an image the guest holds may already have.1852 /// The stamp last asked for, which an image the guest holds may already have.
1850 seen: Option<Stamp>,1853 seen: Option<Stamp>,
1854 /// The host refused the last batch, so the next step reads its image before sending
1855 /// edits. A stamp clears it, as an uncertain batch goes again regardless.
1851 rejected: bool,1856 rejected: bool,
1852 #[cfg(target_arch = "wasm32")]1857 #[cfg(target_arch = "wasm32")]
1853 pending: Option<web::Pending>,1858 pending: Option<web::Pending>,
...@@ -1914,6 +1919,7 @@ impl crate::Remote for HostedRemote {...@@ -1914,6 +1919,7 @@ impl crate::Remote for HostedRemote {
1914 }1919 }
19151920
1916 fn stamp(&mut self) -> io::Result<Stamp> {1921 fn stamp(&mut self) -> io::Result<Stamp> {
1922 self.rejected = false;
1917 let stamp = self.guest.stamp(&self.path)?;1923 let stamp = self.guest.stamp(&self.path)?;
1918 self.seen = Some(stamp.clone());1924 self.seen = Some(stamp.clone());
1919 Ok(stamp)1925 Ok(stamp)
...@@ -2159,6 +2165,7 @@ mod tests {...@@ -2159,6 +2165,7 @@ mod tests {
2159 puts: Mutex::default(),2165 puts: Mutex::default(),
2160 guests: Mutex::default(),2166 guests: Mutex::default(),
2161 writers: Mutex::default(),2167 writers: Mutex::default(),
2168 landed: Mutex::default(),
2162 host: Mutex::default(),2169 host: Mutex::default(),
2163 room: Mutex::default(),2170 room: Mutex::default(),
2164 });2171 });
crates/notebook/src/live/share/batch.rs+140-9
...@@ -114,9 +114,10 @@ impl Served {...@@ -114,9 +114,10 @@ impl Served {
114 let mut image = Arc::clone(&before);114 let mut image = Arc::clone(&before);
115 let mut combined: Option<Transaction> = None;115 let mut combined: Option<Transaction> = None;
116 let mut accepted = Vec::new();116 let mut accepted = Vec::new();
117 let mut landing = Vec::new();
117 for batch in waiting {118 for batch in waiting {
118 match self.prepare_batch(path, &image, &batch) {119 match self.prepare_batch(path, &image, &batch) {
119 Ok(Some(transaction)) => {120 Ok((Some(transaction), revisions)) => {
120 let mut next = (*image).clone();121 let mut next = (*image).clone();
121 let extended =122 let extended =
122 transaction123 transaction
...@@ -131,6 +132,7 @@ impl Served {...@@ -131,6 +132,7 @@ impl Served {
131 match extended {132 match extended {
132 Ok(()) => {133 Ok(()) => {
133 image = Arc::new(next);134 image = Arc::new(next);
135 landing.extend(revisions);
134 accepted.push(batch);136 accepted.push(batch);
135 }137 }
136 Err(error) => {138 Err(error) => {
...@@ -138,7 +140,7 @@ impl Served {...@@ -138,7 +140,7 @@ impl Served {
138 }140 }
139 }141 }
140 }142 }
141 Ok(None) => accepted.push(batch),143 Ok((None, _)) => accepted.push(batch),
142 Err(error) => {144 Err(error) => {
143 let _ = batch.reply.send(failed(0, error));145 let _ = batch.reply.send(failed(0, error));
144 }146 }
...@@ -166,6 +168,7 @@ impl Served {...@@ -166,6 +168,7 @@ impl Served {
166 }),168 }),
167 };169 };
168 if result.is_ok() {170 if result.is_ok() {
171 self.landed.lock().unwrap().extend(landing);
169 self.tell_delta(path, &before, &image, None);172 self.tell_delta(path, &before, &image, None);
170 self.keep(path, Arc::clone(&image));173 self.keep(path, Arc::clone(&image));
171 self.changed(&[path.to_owned()]);174 self.changed(&[path.to_owned()]);
...@@ -194,7 +197,7 @@ impl Served {...@@ -194,7 +197,7 @@ impl Served {
194 path: &str,197 path: &str,
195 image: &[u8],198 image: &[u8],
196 batch: &Waiting,199 batch: &Waiting,
197 ) -> Result<Option<Transaction>> {200 ) -> Result<(Option<Transaction>, BTreeMap<ExGuid, ExGuid>)> {
198 if !self.guests.lock().unwrap().contains_key(&batch.peer) {201 if !self.guests.lock().unwrap().contains_key(&batch.peer) {
199 return Err(refused(202 return Err(refused(
200 io::ErrorKind::PermissionDenied,203 io::ErrorKind::PermissionDenied,
...@@ -219,14 +222,15 @@ impl Served {...@@ -219,14 +222,15 @@ impl Served {
219 }222 }
220 let store = Store::parse(image)?;223 let store = Store::parse(image)?;
221 let index = RevisionIndex::parse(&store)?;224 let index = RevisionIndex::parse(&store)?;
225 let landed = self.landed.lock().unwrap();
222 if edits.revisions.iter().all(|(space, revision)| {226 if edits.revisions.iter().all(|(space, revision)| {
223 index227 landed.contains(&(*space, *revision))
224 .spaces228 || (index.spaces.get(space))
225 .get(space)229 .is_some_and(|space| space.revisions.contains_key(revision))
226 .is_some_and(|space| space.revisions.contains_key(revision))
227 }) {230 }) {
228 return Ok(None);231 return Ok((None, edits.revisions));
229 }232 }
233 drop(landed);
230 let names = edits234 let names = edits
231 .revisions235 .revisions
232 .iter()236 .iter()
...@@ -306,7 +310,7 @@ impl Served {...@@ -306,7 +310,7 @@ impl Served {
306 "The batch needs a guest's commit",310 "The batch needs a guest's commit",
307 ));311 ));
308 }312 }
309 Ok(transaction)313 Ok((transaction, edits.revisions))
310 }314 }
311}315}
312316
...@@ -333,6 +337,7 @@ mod tests {...@@ -333,6 +337,7 @@ mod tests {
333 puts: Mutex::default(),337 puts: Mutex::default(),
334 guests: Mutex::default(),338 guests: Mutex::default(),
335 writers: Mutex::default(),339 writers: Mutex::default(),
340 landed: Mutex::default(),
336 host: Mutex::default(),341 host: Mutex::default(),
337 room: Mutex::default(),342 room: Mutex::default(),
338 });343 });
...@@ -461,6 +466,132 @@ mod tests {...@@ -461,6 +466,132 @@ mod tests {
461 assert_eq!(served.writers.lock().unwrap().len(), 1);466 assert_eq!(served.writers.lock().unwrap().len(), 1);
462 }467 }
463468
469 /// OneNote's Optimize drops a revision a later one superseded, as the page-versions
470 /// corpus shows; a batch sent again after its revision went that way is not applied
471 /// again.
472 #[cfg(not(target_arch = "wasm32"))]
473 #[test]
474 fn a_batch_whose_revision_optimize_dropped_lands_once() {
475 let directory = tempfile::tempdir().unwrap();
476 let folder = directory.path().join("notebook");
477 std::fs::create_dir(&folder).unwrap();
478 let path = "Garden.one";
479 let base = onestore::create_section(path, "Original text", "Fixture").unwrap();
480 std::fs::write(folder.join(path), &base).unwrap();
481 let served = Arc::new(Served {
482 storage: crate::session::Notebook::open(&folder, directory.path().join("cache"))
483 .unwrap()
484 .into_storage(),
485 images: Mutex::default(),
486 snapshots: Mutex::default(),
487 puts: Mutex::default(),
488 guests: Mutex::default(),
489 writers: Mutex::default(),
490 landed: Mutex::default(),
491 host: Mutex::default(),
492 room: Mutex::default(),
493 });
494 let peer = [1; 16];
495 let (line, _outbound) = mpsc::channel();
496 let (queue, _requests) = mpsc::sync_channel(1);
497 served.guests.lock().unwrap().insert(
498 peer,
499 Admitted {
500 hello: Arc::new(Hello {
501 peer,
502 ..Hello::new("Guest".into(), None).unwrap()
503 }),
504 line: crate::live::Line(line),
505 queue,
506 starts: BURST,
507 counted: Instant::now(),
508 held: Vec::new(),
509 },
510 );
511 let store = Store::parse(&base).unwrap();
512 let index = RevisionIndex::parse(&store).unwrap();
513 let document = onestore::document::Document::parse(&index).unwrap();
514 let (space, text) = document
515 .spaces
516 .iter()
517 .find_map(|(space, object)| {
518 object.revisions[&object.contexts[&ExGuid::default()]]
519 .nodes
520 .iter()
521 .find_map(|(id, node)| {
522 matches!(node.kind, onestore::document::Kind::RichText { .. })
523 .then_some((*space, *id))
524 })
525 })
526 .unwrap();
527 let typed = |range: std::ops::Range<u32>, with: &str| Edit {
528 at: 134_000_000_000_000_000,
529 ops: vec![Op::Page {
530 space,
531 op: PageOp::Text {
532 text,
533 range,
534 with: with.into(),
535 },
536 }],
537 };
538 let arena = Arena::default();
539 let mut section = Section::open(&arena, base.clone()).unwrap();
540 section.apply("Guest", &typed(13..13, "X")).unwrap();
541 let transaction = section.seal().unwrap().unwrap();
542 let request = Request {
543 path: path.into(),
544 stamp: Some(transaction.base().into()),
545 bytes: Some(
546 encode(
547 &Edits {
548 edits: vec![("Guest".into(), typed(13..13, "X"))],
549 revisions: section.newest().filter(|(at, _)| *at == space).collect(),
550 },
551 LIMIT,
552 )
553 .unwrap(),
554 ),
555 ..Request::default()
556 };
557 assert!(
558 served
559 .batch(&peer, request.clone())
560 .unwrap()
561 .failure
562 .is_none()
563 );
564
565 // The host deletes what the batch typed, and Optimize keeps the page's newest
566 // revision alone: the file holds neither the text nor the batch's revision.
567 let arena = Arena::default();
568 let mut optimized = Section::open(&arena, base.clone()).unwrap();
569 optimized.apply("Guest", &typed(13..13, "X")).unwrap();
570 optimized.apply("Host", &typed(13..14, "")).unwrap();
571 let mut image = base;
572 optimized
573 .seal()
574 .unwrap()
575 .unwrap()
576 .apply(&mut image)
577 .unwrap();
578 std::fs::write(folder.join(path), &image).unwrap();
579 let replayed = served.batch(&peer, request).unwrap();
580 assert!(replayed.failure.is_none(), "{:?}", replayed.failure);
581 let stored = std::fs::read(folder.join(path)).unwrap();
582 let arena = Arena::default();
583 let section = Section::open(&arena, stored).unwrap();
584 let texts: Vec<String> = (section.page(space).unwrap().objects.iter())
585 .filter_map(|object| match object {
586 PageObject::Outline(outline) => Some(outline.paragraphs.iter()),
587 _ => None,
588 })
589 .flatten()
590 .filter_map(|paragraph| Some(paragraph.text()?.text.text().to_owned()))
591 .collect();
592 assert_eq!(texts, ["Original text"]);
593 }
594
464 #[test]595 #[test]
465 fn attachment_encoding_stops_at_the_upload_budget() {596 fn attachment_encoding_stops_at_the_upload_budget() {
466 let id = ExGuid {597 let id = ExGuid {
crates/notebook/src/live/share/handover_tests.rs+1
...@@ -438,6 +438,7 @@ fn served() -> (tempfile::TempDir, Arc<Served>) {...@@ -438,6 +438,7 @@ fn served() -> (tempfile::TempDir, Arc<Served>) {
438 puts: Mutex::default(),438 puts: Mutex::default(),
439 guests: Mutex::default(),439 guests: Mutex::default(),
440 writers: Mutex::default(),440 writers: Mutex::default(),
441 landed: Mutex::default(),
441 host: Mutex::default(),442 host: Mutex::default(),
442 room: Mutex::default(),443 room: Mutex::default(),
443 });444 });
crates/notebook/src/live/share/membership.rs+1
...@@ -52,6 +52,7 @@ impl Host {...@@ -52,6 +52,7 @@ impl Host {
52 puts: Mutex::default(),52 puts: Mutex::default(),
53 guests: Mutex::default(),53 guests: Mutex::default(),
54 writers: Mutex::default(),54 writers: Mutex::default(),
55 landed: Mutex::default(),
55 host: Mutex::default(),56 host: Mutex::default(),
56 room: Mutex::default(),57 room: Mutex::default(),
57 }),58 }),
crates/notebook/src/live/share/web.rs+1
...@@ -90,6 +90,7 @@ impl crate::Remote for HostedRemote {...@@ -90,6 +90,7 @@ impl crate::Remote for HostedRemote {
90 }90 }
9191
92 fn stamp(&mut self) -> io::Result<Stamp> {92 fn stamp(&mut self) -> io::Result<Stamp> {
93 self.rejected = false;
93 let (guest, path) = (self.guest.clone(), self.path.clone());94 let (guest, path) = (self.guest.clone(), self.path.clone());
94 let result = self95 let result = self
95 .operation(async move {96 .operation(async move {
crates/notebook/src/sync.rs+36-3
...@@ -27,6 +27,8 @@ pub trait Remote {...@@ -27,6 +27,8 @@ pub trait Remote {
27 fn accepts_edits(&self) -> bool {27 fn accepts_edits(&self) -> bool {
28 false28 false
29 }29 }
30 /// Where the remote accepts edits, a batch published again with the same revisions,
31 /// however its first attempt ended, lands at most once.
30 fn publish_edits(32 fn publish_edits(
31 &mut self,33 &mut self,
32 transaction: &Transaction,34 transaction: &Transaction,
...@@ -223,7 +225,7 @@ impl Replica {...@@ -223,7 +225,7 @@ impl Replica {
223 /// Publishes the oldest unpublished batch, rebasing the queue first when the remote225 /// Publishes the oldest unpublished batch, rebasing the queue first when the remote
224 /// changed. Reads the remote image only when its stamp moved; network I/O holds226 /// changed. Reads the remote image only when its stamp moved; network I/O holds
225 /// synchronization ownership without holding the cache mutex. Uncertain attempts are227 /// synchronization ownership without holding the cache mutex. Uncertain attempts are
226 /// never replayed.228 /// replayed only to a remote that accepts edits.
227 /// Versions the remote keeps beside the file merge into it first, each published as one229 /// Versions the remote keeps beside the file merge into it first, each published as one
228 /// more revision and then retired (`resolve.rs`).230 /// more revision and then retired (`resolve.rs`).
229 pub fn sync_once(&self, remote: &mut impl Remote) -> Result<Synced> {231 pub fn sync_once(&self, remote: &mut impl Remote) -> Result<Synced> {
...@@ -281,6 +283,19 @@ impl Replica {...@@ -281,6 +283,19 @@ impl Replica {
281 let observed = awaited(remote, Remote::stamp)283 let observed = awaited(remote, Remote::stamp)
282 .await284 .await
283 .map_err(Error::RemoteIo)?;285 .map_err(Error::RemoteIo)?;
286 let replay = match &state.blocked {
287 Some(blocked) if self.section.key.is_none() && remote.accepts_edits() => {
288 working::sealed(&*self.lock()?)?
289 .filter(|sealed| sealed.batch == blocked.batch)
290 .and_then(|sealed| sealed.transaction)
291 }
292 _ => None,
293 };
294 if let (Some(blocked), Some(transaction)) = (&state.blocked, replay) {
295 return self
296 .publish_sealed(remote, blocked.batch, transaction, true, true, Vec::new())
297 .await;
298 }
284 if let Some(blocked) = &state.blocked299 if let Some(blocked) = &state.blocked
285 && observed == *state.remote.as_ref().unwrap_or(&state.base)300 && observed == *state.remote.as_ref().unwrap_or(&state.base)
286 {301 {
...@@ -419,8 +434,8 @@ impl Replica {...@@ -419,8 +434,8 @@ impl Replica {
419 changed,434 changed,
420 });435 });
421 };436 };
422 let id = newest(&*self.lock()?, batch)?;
423 let Some(transaction) = transaction else {437 let Some(transaction) = transaction else {
438 let id = newest(&*self.lock()?, batch)?;
424 // Edits that changed nothing are published once the remote's image is durable.439 // Edits that changed nothing are published once the remote's image is durable.
425 let base = base::base_stamp(&*self.lock()?)?;440 let base = base::base_stamp(&*self.lock()?)?;
426 if let Err(error) = awaited(remote, |remote| remote.confirm(&base)).await {441 if let Err(error) = awaited(remote, |remote| remote.confirm(&base)).await {
...@@ -441,6 +456,24 @@ impl Replica {...@@ -441,6 +456,24 @@ impl Replica {
441 .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?;456 .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?;
442 return Err(error.into());457 return Err(error.into());
443 }458 }
459 self.publish_sealed(remote, batch, transaction, batched, false, changed)
460 .await
461 }
462
463 /// Publishes the attempted `batch` sealed as `transaction`, rebasing on the remote's
464 /// image after where `batched`; `changed` holds the pages a rebase already changed. An
465 /// attempt made `again` stays uncertain until the remote takes it, as the first may
466 /// still land.
467 async fn publish_sealed(
468 &self,
469 remote: &mut impl Remote,
470 batch: i64,
471 transaction: Transaction,
472 batched: bool,
473 again: bool,
474 mut changed: Vec<ExGuid>,
475 ) -> Result<Synced> {
476 let id = newest(&*self.lock()?, batch)?;
444 let published = if self.section.key.is_none() && remote.accepts_edits() {477 let published = if self.section.key.is_none() && remote.accepts_edits() {
445 let (edits, revisions) = {478 let (edits, revisions) = {
446 let connection = self.lock()?;479 let connection = self.lock()?;
...@@ -461,7 +494,7 @@ impl Replica {...@@ -461,7 +494,7 @@ impl Replica {
461 };494 };
462 match published {495 match published {
463 Ok(()) => {}496 Ok(()) => {}
464 Err(error) if error.state == CommitState::NotCommitted => {497 Err(error) if error.state == CommitState::NotCommitted && !again => {
465 self.lock()?498 self.lock()?
466 .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?;499 .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?;
467 return Err(error.into());500 return Err(error.into());
crates/notebook/tests/live_batch.rs+192-51
...@@ -207,34 +207,6 @@ fn edit_receipts_survive_guest_and_host_process_death() {...@@ -207,34 +207,6 @@ fn edit_receipts_survive_guest_and_host_process_death() {
207 stopped_at_boundary(&root);207 stopped_at_boundary(&root);
208 }208 }
209 let mut remote = HostedRemote::new(&guest, "Garden.one");209 let mut remote = HostedRemote::new(&guest, "Garden.one");
210 if boundary == "unsent" {
211 let attempted = replica.status(id).unwrap();
212 assert!(matches!(
213 attempted,
214 Some(EditStatus::AwaitingConfirmation { .. })
215 ));
216 let pending = replica.pending().unwrap();
217 let snapshot = replica.snapshot().unwrap();
218 let original = std::fs::read(root.join("Garden/Garden.one")).unwrap();
219 for _ in 0..3 {
220 replica.sync_once(&mut remote).unwrap();
221 assert_eq!(replica.status(id).unwrap(), attempted);
222 assert_eq!(replica.pending().unwrap(), pending);
223 assert_eq!(replica.snapshot().unwrap(), snapshot);
224 assert_eq!(
225 std::fs::read(root.join("Garden/Garden.one")).unwrap(),
226 original
227 );
228 }
229 let archive = root.join("before-review.sqlite");
230 replica
231 .release(id, &archive, notebook::Resolution::Mine)
232 .unwrap();
233 let archive = notebook::Recovery::open(archive).unwrap();
234 assert_eq!(archive.status(id).unwrap(), attempted);
235 assert_eq!(archive.pending().unwrap(), pending);
236 assert_eq!(archive.snapshot().unwrap(), snapshot);
237 }
238 for _ in 0..5 {210 for _ in 0..5 {
239 replica.sync_once(&mut remote).unwrap();211 replica.sync_once(&mut remote).unwrap();
240 if matches!(212 if matches!(
...@@ -421,29 +393,6 @@ fn edit_receipts_survive_guest_and_host_process_death() {...@@ -421,29 +393,6 @@ fn edit_receipts_survive_guest_and_host_process_death() {
421 assert!(replica.pending().unwrap().is_empty());393 assert!(replica.pending().unwrap().is_empty());
422 assert_eq!(replica.snapshot().unwrap(), result);394 assert_eq!(replica.snapshot().unwrap(), result);
423 drop(replica);395 drop(replica);
424 if boundary == "unsent" {
425 let archive = notebook::Recovery::open(root.join("before-review.sqlite")).unwrap();
426 assert!(matches!(
427 archive.status(id).unwrap(),
428 Some(EditStatus::AwaitingConfirmation { .. })
429 ));
430 assert_eq!(archive.pending().unwrap()[0].id, id);
431 assert_eq!(archive.pending().unwrap().len(), 1);
432 assert_eq!(
433 body(&archive.snapshot().unwrap()).2,
434 format!("{content}{INSERTION}")
435 );
436 let database = rusqlite::Connection::open(root.join("before-review.sqlite")).unwrap();
437 assert_eq!(
438 database
439 .query_row("PRAGMA integrity_check", [], |row| row.get::<_, String>(0))
440 .unwrap(),
441 "ok"
442 );
443 println!(
444 "unsent: cold uncertainty retained; explicit Mine review archived the original queue before publication"
445 );
446 }
447 let database = rusqlite::Connection::open(&database).unwrap();396 let database = rusqlite::Connection::open(&database).unwrap();
448 assert_eq!(397 assert_eq!(
449 database398 database
...@@ -535,3 +484,195 @@ fn concurrent_batches_keep_receipts_and_rebase_later_keystrokes() {...@@ -535,3 +484,195 @@ fn concurrent_batches_keep_receipts_and_rebase_later_keystrokes() {
535 let mut section = onestore::Section::open(&arena, result).unwrap();484 let mut section = onestore::Section::open(&arena, result).unwrap();
536 assert!(section.conflicts().unwrap().is_empty());485 assert!(section.conflicts().unwrap().is_empty());
537}486}
487
488/// A batch whose request the relay loses with the guest's connection is uncertain; once the
489/// guest is back, the host hears it again and it lands once.
490#[test]
491fn a_batch_lost_on_the_way_lands_once_the_guest_is_back() {
492 use notebook::{EditStatus, Error, Remote, live::share::Guest};
493 use onestore::{CommitError, CommitState};
494
495 let directory = tempfile::tempdir().unwrap();
496 let folder = notebook(directory.path());
497 let url = relay(Default::default());
498 let sharing = Sharing::new("").unwrap();
499 let host = host(&folder, &directory.path().join("host"), &sharing, &url);
500 let welcome =
501 notebook::live::share::join(hello("Grace"), &code(&host), "", None, Some(&url)).unwrap();
502 let relay = PresenceRelay::new(&url, &welcome.room);
503 let guest = Guest::start(
504 hello("Grace"),
505 welcome.share,
506 welcome.secret,
507 None,
508 Some(&relay.url),
509 || {},
510 )
511 .unwrap();
512 until("the host was never met", || guest.host().is_some());
513 let file = folder.join("Garden.one");
514 let original = std::fs::read(&file).unwrap();
515 let (space, text, _) = server::text(&original);
516 let typed = |with: &str| Edit {
517 at: 134_000_000_000_000_000,
518 ops: vec![Op::Page {
519 space,
520 op: PageOp::Text {
521 text,
522 range: 0..0,
523 with: with.into(),
524 },
525 }],
526 };
527 let replica = Replica::open_or_create(directory.path().join("grace.sqlite"), None, || {
528 Ok(original.clone())
529 })
530 .unwrap();
531 let remote = std::cell::RefCell::new(HostedRemote::new(&guest, "Garden.one"));
532 let first = replica.apply("Grace", typed("A")).unwrap();
533 until("the first edit was never published", || {
534 let _ = replica.sync_once(&mut *remote.borrow_mut());
535 matches!(
536 replica.status(first).unwrap(),
537 Some(EditStatus::Published { .. })
538 )
539 });
540 until("the host's stamp was never known without asking", || {
541 let before = relay.state.lock().unwrap().access_bytes;
542 remote.borrow_mut().stamp().unwrap();
543 relay.state.lock().unwrap().access_bytes == before
544 });
545
546 relay.state.lock().unwrap().swallow = true;
547 let second = replica.apply("Grace", typed("B")).unwrap();
548 assert!(matches!(
549 replica.sync_once(&mut *remote.borrow_mut()),
550 Err(Error::Remote(CommitError {
551 state: CommitState::Unknown,
552 ..
553 }))
554 ));
555 assert!(matches!(
556 replica.status(second).unwrap(),
557 Some(EditStatus::AwaitingConfirmation { .. })
558 ));
559 // OneNote on the host holds the section as the batch comes again, and lets it go. The
560 // app's sync worker keeps its remote after such a refusal.
561 let held = std::fs::File::open(&file).unwrap();
562 held.lock_shared().unwrap();
563 let remote = std::cell::RefCell::new(HostedRemote::new(&guest, "Garden.one"));
564 until(
565 "the host never refused the batch while OneNote held it",
566 || {
567 matches!(
568 replica.sync_once(&mut *remote.borrow_mut()),
569 Err(Error::Remote(CommitError {
570 state: CommitState::NotCommitted,
571 ..
572 }))
573 )
574 },
575 );
576 drop(held);
577 until("the uncertain batch never landed", || {
578 let _ = replica.sync_once(&mut *remote.borrow_mut());
579 matches!(
580 replica.status(second).unwrap(),
581 Some(EditStatus::Published { .. })
582 )
583 });
584 assert_eq!(
585 server::text(&std::fs::read(&file).unwrap()).2,
586 "BAOriginal text"
587 );
588}
589
590/// A section closed while its batch is uncertain sends the batch again from the
591/// background, which keeps a notebook's closed sections in sync.
592#[test]
593fn a_closed_section_s_uncertain_batch_lands_from_the_background() {
594 use notebook::{
595 EditStatus, Error, Remote,
596 live::share::Guest,
597 session::{Background, Notebook},
598 };
599 use onestore::{CommitError, CommitState};
600
601 let directory = tempfile::tempdir().unwrap();
602 let folder = notebook(directory.path());
603 let url = relay(Default::default());
604 let sharing = Sharing::new("").unwrap();
605 let host = host(&folder, &directory.path().join("host"), &sharing, &url);
606 let welcome =
607 notebook::live::share::join(hello("Grace"), &code(&host), "", None, Some(&url)).unwrap();
608 let relay = PresenceRelay::new(&url, &welcome.room);
609 let guest = Guest::start(
610 hello("Grace"),
611 welcome.share,
612 welcome.secret,
613 None,
614 Some(&relay.url),
615 || {},
616 )
617 .unwrap();
618 until("the host was never met", || guest.host().is_some());
619 let mut notebook =
620 Notebook::open_hosted(Arc::clone(&guest), directory.path().join("grace")).unwrap();
621 let file = folder.join("Garden.one");
622 let original = std::fs::read(&file).unwrap();
623 let (space, text, _) = server::text(&original);
624 let path = notebook.replica_path("Garden.one").unwrap();
625 std::fs::create_dir_all(path.parent().unwrap()).unwrap();
626 let replica = Replica::open_or_create(&path, None, || Ok(original.clone())).unwrap();
627 let typed = |with: &str| Edit {
628 at: 134_000_000_000_000_000,
629 ops: vec![Op::Page {
630 space,
631 op: PageOp::Text {
632 text,
633 range: 0..0,
634 with: with.into(),
635 },
636 }],
637 };
638 let first = replica.apply("Grace", typed("A")).unwrap();
639 until("the first edit was never published", || {
640 let _ = replica.sync_once(&mut HostedRemote::new(&guest, "Garden.one"));
641 matches!(
642 replica.status(first).unwrap(),
643 Some(EditStatus::Published { .. })
644 )
645 });
646 until("the host's stamp was never known without asking", || {
647 let before = relay.state.lock().unwrap().access_bytes;
648 HostedRemote::new(&guest, "Garden.one").stamp().unwrap();
649 relay.state.lock().unwrap().access_bytes == before
650 });
651 relay.state.lock().unwrap().swallow = true;
652 let id = replica.apply("Grace", typed("B")).unwrap();
653 assert!(matches!(
654 replica.sync_once(&mut HostedRemote::new(&guest, "Garden.one")),
655 Err(Error::Remote(CommitError {
656 state: CommitState::Unknown,
657 ..
658 }))
659 ));
660 drop(replica);
661 let background = Background::hosted(Arc::clone(&guest), || {}).unwrap();
662 background.watch(notebook.replicas());
663 // As the app does, the background is woken whenever the guest meets its host again.
664 let reached = std::cell::Cell::new(guest.host().is_some());
665 until("the closed section's batch never landed", || {
666 let now = guest.host().is_some();
667 if now && !reached.replace(now) {
668 background.wake();
669 }
670 server::text(&std::fs::read(&file).unwrap()).2 == "BAOriginal text"
671 });
672 background.stop();
673 let replica = Replica::open(&path).unwrap();
674 assert!(matches!(
675 replica.status(id).unwrap(),
676 Some(EditStatus::Published { .. })
677 ));
678}
crates/notebook/tests/support/live.rs+8-1
...@@ -168,6 +168,8 @@ pub struct RelayState {...@@ -168,6 +168,8 @@ pub struct RelayState {
168 pub blocked: bool,168 pub blocked: bool,
169 pub access_connections: usize,169 pub access_connections: usize,
170 pub access_bytes: usize,170 pub access_bytes: usize,
171 /// The next bytes a guest sends for access are lost, and their connection with them.
172 pub swallow: bool,
171 sockets: Vec<(bool, std::net::TcpStream)>,173 sockets: Vec<(bool, std::net::TcpStream)>,
172 stopped: bool,174 stopped: bool,
173}175}
...@@ -237,7 +239,12 @@ impl PresenceRelay {...@@ -237,7 +239,12 @@ impl PresenceRelay {
237 break;239 break;
238 }240 }
239 if !presence {241 if !presence {
240 counted.lock().unwrap().access_bytes += length;242 let mut counted = counted.lock().unwrap();
243 if std::mem::take(&mut counted.swallow) {
244 let _ = from.shutdown(Shutdown::Both);
245 break;
246 }
247 counted.access_bytes += length;
241 }248 }
242 if to.write_all(&bytes[..length]).is_err() {249 if to.write_all(&bytes[..length]).is_err() {
243 break;250 break;