| author | |
| committer | |
| log | 650ea9ccbc132590b96c6c067d62f2dc753117b8 |
| tree | 705b5bf6816694e8fb0a371c4dd35e83912c5649 |
| parent | aa378b005697f7ce0e295aa865a7d46afa0e3f78 |
| signature | Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU |
A protected section publishes its sealed transaction whole, as the host cannot read its edits, so an attempt made uncertain by a dropped connection held the section until a review nothing in the app offers. A Live Share host commits such a transaction only on the image it was made on, so the same transaction sent again lands at most once, and the guest now sends it again like a batch. The host now names a commit made on an image it has moved past a stale handle, as it does a batch whose base it dropped; the guest then reads the section, takes the commit as landed where the section holds its revisions, and otherwise replays its queue on the section, as it can never land. Without that check a commit whose answer was lost after it landed would be replayed on a section where it had since been undone, bringing it back. As the guest has rebased by the time it reports such a refusal, its sync worker keeps the remote and tries again after the short pause it takes when another writer holds the section, rather than waiting for the next check. A test loses a protected commit's request and checks it lands once the guest is back, then loses the answer to one that landed, has the host delete what it typed, and checks that it stays deleted.
- Edits to a password-protected section of a Live Share notebook, sent just as the connection dropped, now publish once the guest reconnects.
Assisted-by: claude-opus-5.56 files changed, 205 insertions(+), 17 deletions(-)
arc/sync.md+5-2| ... | ... | @@ -129,8 +129,11 @@ Uncertainty is handled explicitly. A publication attempt is recorded before |
| 129 | 129 | the first network byte. If the reply is lost, the attempt confirms only when |
| 130 | 130 | the remote holds its revisions (or every page it changed, as it changed them). |
| 131 | 131 | Otherwise it waits as `AwaitingConfirmation`. A Live Share host lands a batch at |
| 132 | most once, as it looks for the batch's revisions before applying it, so the next | |
| 133 | step sends it the same batch again. Any other remote, where a late write could land | |
| 132 | most once, as it looks for the batch's revisions before applying it, and commits a | |
| 133 | protected section's sealed transaction only on the image it was made on, so the next | |
| 134 | step sends it the same batch again. Refused because the host no longer holds that | |
| 135 | image, the batch has landed if the section holds its revisions, and else never will: | |
| 136 | the queue replays on the section as it is. Any other remote, where a late write could land | |
| 134 | 137 | a second copy, waits for `release`, which exports a recovery archive first, then |
| 135 | 138 | republishes or abandons. |
| 136 | 139 |
crates/notebook/README.md+4-2| ... | ... | @@ -165,8 +165,10 @@ merges back. |
| 165 | 165 | Publication attempts are recorded before network I/O. An attempt confirms only |
| 166 | 166 | when the remote holds its revisions, or every page it changed as it changed them, |
| 167 | 167 | followed by `Remote::confirm`; otherwise it stays `AwaitingConfirmation`. A remote that |
| 168 | accepts edits (a Live Share host) lands a batch at most once, so the next step sends it the | |
| 169 | same batch again; any other remote never gets it again unreviewed. | |
| 168 | accepts edits (a Live Share host) lands a batch at most once, a protected section's | |
| 169 | transaction included, so the next step sends it the same batch again; one it refuses as | |
| 170 | `StaleNetworkFileHandle` is acknowledged where the image holds its revisions, else rebased. | |
| 171 | Any other remote never gets it again unreviewed. | |
| 170 | 172 | `release(id, archive, Resolution)` exports the queue first, then publishes the batch again |
| 171 | 173 | (`Mine`) or abandons every unpublished edit (`Theirs`). |
| 172 | 174 | A durable `Published` receipt survives reopening. |
crates/notebook/src/live/share.rs+1-1| ... | ... | @@ -777,7 +777,7 @@ impl Served { |
| 777 | 777 | })?; |
| 778 | 778 | if Stamp::of(&image).ok().as_ref() != Some(transaction.base()) { |
| 779 | 779 | return Err(not_committed(io::Error::new( |
| 780 | io::ErrorKind::ResourceBusy, | |
| 780 | io::ErrorKind::StaleNetworkFileHandle, | |
| 781 | 781 | "The section changed since", |
| 782 | 782 | ))); |
| 783 | 783 | } |
crates/notebook/src/sync.rs+30-11| ... | ... | @@ -27,9 +27,10 @@ pub trait Remote { |
| 27 | 27 | fn accepts_edits(&self) -> bool { |
| 28 | 28 | false |
| 29 | 29 | } |
| 30 | /// Where the remote accepts edits, a batch published again with the same revisions, | |
| 31 | /// however its first attempt ended, lands at most once; one refused as | |
| 32 | /// `StaleNetworkFileHandle` never can, as the remote no longer holds its base. | |
| 30 | /// Where the remote accepts edits, a batch published again with the same revisions, or | |
| 31 | /// a protected section's transaction published again, however its first attempt ended, | |
| 32 | /// lands at most once; one refused as `StaleNetworkFileHandle` has landed if the image | |
| 33 | /// holds its revisions, and else never can, as the remote no longer holds its base. | |
| 33 | 34 | fn publish_edits( |
| 34 | 35 | &mut self, |
| 35 | 36 | transaction: &Transaction, |
| ... | ... | @@ -285,11 +286,9 @@ impl Replica { |
| 285 | 286 | .await |
| 286 | 287 | .map_err(Error::RemoteIo)?; |
| 287 | 288 | let replay = match &state.blocked { |
| 288 | Some(blocked) if self.section.key.is_none() && remote.accepts_edits() => { | |
| 289 | working::sealed(&*self.lock()?)? | |
| 290 | .filter(|sealed| sealed.batch == blocked.batch) | |
| 291 | .and_then(|sealed| sealed.transaction) | |
| 292 | } | |
| 289 | Some(blocked) if remote.accepts_edits() => working::sealed(&*self.lock()?)? | |
| 290 | .filter(|sealed| sealed.batch == blocked.batch) | |
| 291 | .and_then(|sealed| sealed.transaction), | |
| 293 | 292 | _ => None, |
| 294 | 293 | }; |
| 295 | 294 | if let (Some(blocked), Some(transaction)) = (&state.blocked, replay) { |
| ... | ... | @@ -464,7 +463,8 @@ impl Replica { |
| 464 | 463 | /// Publishes the attempted `batch` sealed as `transaction`, rebasing on the remote's |
| 465 | 464 | /// image after where `batched`; `changed` holds the pages a rebase already changed. An |
| 466 | 465 | /// attempt made `again` stays uncertain until the remote takes it, as the first may |
| 467 | /// still land. A batch whose base the remote refuses for good is rebased onto its image. | |
| 466 | /// still land. A batch whose base the remote refuses for good is acknowledged where the | |
| 467 | /// remote's image holds it, else rebased onto that image. | |
| 468 | 468 | async fn publish_sealed( |
| 469 | 469 | &self, |
| 470 | 470 | remote: &mut impl Remote, |
| ... | ... | @@ -499,11 +499,30 @@ impl Replica { |
| 499 | 499 | if error.state == CommitState::NotCommitted |
| 500 | 500 | && error.error.kind() == io::ErrorKind::StaleNetworkFileHandle => |
| 501 | 501 | { |
| 502 | self.lock()? | |
| 503 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; | |
| 504 | 502 | let image = awaited(remote, Remote::read) |
| 505 | 503 | .await |
| 506 | 504 | .map_err(Error::RemoteIo)?; |
| 505 | // A whole commit made uncertain may have landed before the one refused. | |
| 506 | let revisions = decode_revisions(&self.lock()?.query_row( | |
| 507 | "SELECT revisions FROM batches WHERE id=?1", | |
| 508 | [batch], | |
| 509 | |row| row.get::<_, String>(0), | |
| 510 | )?)?; | |
| 511 | let store = Store::parse(&image)?; | |
| 512 | let index = RevisionIndex::parse(&store)?; | |
| 513 | if revisions.iter().all(|(space, revision)| { | |
| 514 | (index.spaces.get(space)) | |
| 515 | .is_some_and(|space| space.revisions.contains_key(revision)) | |
| 516 | }) { | |
| 517 | self.acknowledge(batch, Some(&transaction), None)?; | |
| 518 | let revision = self.receipt(id)?; | |
| 519 | return Ok(Synced { | |
| 520 | edit: Some((id, EditStatus::Published { revision })), | |
| 521 | changed: self.rebase(Some(image))?, | |
| 522 | }); | |
| 523 | } | |
| 524 | self.lock()? | |
| 525 | .execute("UPDATE batches SET attempted=0 WHERE id=?1", [batch])?; | |
| 507 | 526 | self.rebase(Some(image))?; |
| 508 | 527 | return Err(error.into()); |
| 509 | 528 | } |
crates/notebook/src/worker.rs+4-1| ... | ... | @@ -296,12 +296,15 @@ impl Replica { |
| 296 | 296 | } |
| 297 | 297 | Ok(Synced { edit: None, .. }) => contention = 0, |
| 298 | 298 | Ok(_) => {} |
| 299 | // A commit made on an image the remote moved past was rebased already. | |
| 299 | 300 | Err(Error::Remote(onestore::CommitError { |
| 300 | 301 | state: onestore::CommitState::NotCommitted, |
| 301 | 302 | ref error, |
| 302 | 303 | })) if matches!( |
| 303 | 304 | error.kind(), |
| 304 | io::ErrorKind::WouldBlock | io::ErrorKind::ResourceBusy | |
| 305 | io::ErrorKind::WouldBlock | |
| 306 | | io::ErrorKind::ResourceBusy | |
| 307 | | io::ErrorKind::StaleNetworkFileHandle | |
| 305 | 308 | ) => |
| 306 | 309 | { |
| 307 | 310 | contention = contention.saturating_add(1); |
crates/notebook/tests/live_batch.rs+161| ... | ... | @@ -760,3 +760,164 @@ fn a_lost_batch_whose_base_the_host_dropped_is_rebased() { |
| 760 | 760 | "BAOriginal text!" |
| 761 | 761 | ); |
| 762 | 762 | } |
| 763 | ||
| 764 | /// A protected section's commit is sent again as it was, whole: one the relay lost lands | |
| 765 | /// once the guest is back, and one whose answer was lost after it landed is found in the | |
| 766 | /// host's section rather than landing twice. | |
| 767 | #[test] | |
| 768 | fn a_protected_section_s_uncertain_commit_lands_once() { | |
| 769 | use notebook::{EditStatus, Error, PendingEdit, Remote, live::share::Guest, session::Notebook}; | |
| 770 | use onestore::{CommitError, CommitState, ExGuid, Stamp, Transaction}; | |
| 771 | use std::collections::BTreeMap; | |
| 772 | ||
| 773 | /// A remote whose commits land but whose answers are lost. | |
| 774 | struct Unanswered(HostedRemote); | |
| 775 | impl Remote for Unanswered { | |
| 776 | fn read(&mut self) -> std::io::Result<Vec<u8>> { | |
| 777 | self.0.read() | |
| 778 | } | |
| 779 | fn stamp(&mut self) -> std::io::Result<Stamp> { | |
| 780 | self.0.stamp() | |
| 781 | } | |
| 782 | fn publish(&mut self, transaction: &Transaction) -> Result<(), CommitError> { | |
| 783 | self.0.publish(transaction)?; | |
| 784 | Err(CommitError { | |
| 785 | state: CommitState::Unknown, | |
| 786 | error: std::io::ErrorKind::BrokenPipe.into(), | |
| 787 | }) | |
| 788 | } | |
| 789 | fn confirm(&mut self, stamp: &Stamp) -> Result<(), CommitError> { | |
| 790 | self.0.confirm(stamp) | |
| 791 | } | |
| 792 | fn accepts_edits(&self) -> bool { | |
| 793 | self.0.accepts_edits() | |
| 794 | } | |
| 795 | fn publish_edits( | |
| 796 | &mut self, | |
| 797 | transaction: &Transaction, | |
| 798 | _: &[PendingEdit], | |
| 799 | _: &BTreeMap<ExGuid, ExGuid>, | |
| 800 | ) -> Result<(), CommitError> { | |
| 801 | self.publish(transaction) | |
| 802 | } | |
| 803 | } | |
| 804 | ||
| 805 | let directory = tempfile::tempdir().unwrap(); | |
| 806 | let folder = notebook(directory.path()); | |
| 807 | let url = relay(Default::default()); | |
| 808 | let sharing = Sharing::new("").unwrap(); | |
| 809 | let host = host(&folder, &directory.path().join("host"), &sharing, &url); | |
| 810 | let welcome = | |
| 811 | notebook::live::share::join(hello("Grace"), &code(&host), "", None, Some(&url)).unwrap(); | |
| 812 | let relay = PresenceRelay::new(&url, &welcome.room); | |
| 813 | let guest = Guest::start( | |
| 814 | hello("Grace"), | |
| 815 | welcome.share, | |
| 816 | welcome.secret, | |
| 817 | None, | |
| 818 | Some(&relay.url), | |
| 819 | || {}, | |
| 820 | ) | |
| 821 | .unwrap(); | |
| 822 | until("the host was never met", || guest.host().is_some()); | |
| 823 | let notebook = | |
| 824 | Notebook::open_hosted(Arc::clone(&guest), directory.path().join("grace")).unwrap(); | |
| 825 | let key = notebook.unlock("Sealed.one", PASSWORD).unwrap(); | |
| 826 | let file = folder.join("Sealed.one"); | |
| 827 | let original = std::fs::read(&file).unwrap(); | |
| 828 | let replica = | |
| 829 | Replica::open_or_create(directory.path().join("sealed.sqlite"), Some(&key), || { | |
| 830 | Ok(original.clone()) | |
| 831 | }) | |
| 832 | .unwrap(); | |
| 833 | let arena = onestore::Arena::default(); | |
| 834 | let mut unlocked = onestore::Section::unlock(&arena, original.clone(), &key).unwrap(); | |
| 835 | let (space, ..) = unlocked.pages().unwrap()[0]; | |
| 836 | let text = server::page_texts(&unlocked.page(space).unwrap()); | |
| 837 | assert_eq!(text, ["Sealed text"]); | |
| 838 | let paragraph = unlocked | |
| 839 | .page(space) | |
| 840 | .unwrap() | |
| 841 | .objects | |
| 842 | .iter() | |
| 843 | .find_map(|object| match object { | |
| 844 | onestore::page::PageObject::Outline(outline) => outline | |
| 845 | .paragraphs | |
| 846 | .iter() | |
| 847 | .find_map(|p| p.text().map(|t| t.id)), | |
| 848 | _ => None, | |
| 849 | }) | |
| 850 | .unwrap(); | |
| 851 | let typed = |range: std::ops::Range<u32>, with: &str| Edit { | |
| 852 | at: 134_000_000_000_000_000, | |
| 853 | ops: vec![Op::Page { | |
| 854 | space, | |
| 855 | op: PageOp::Text { | |
| 856 | text: paragraph, | |
| 857 | range, | |
| 858 | with: with.into(), | |
| 859 | }, | |
| 860 | }], | |
| 861 | }; | |
| 862 | let stored = || { | |
| 863 | let arena = onestore::Arena::default(); | |
| 864 | let section = | |
| 865 | onestore::Section::unlock(&arena, std::fs::read(&file).unwrap(), &key).unwrap(); | |
| 866 | server::page_texts(&section.page(space).unwrap()) | |
| 867 | }; | |
| 868 | let published = |id| { | |
| 869 | matches!( | |
| 870 | replica.status(id).unwrap(), | |
| 871 | Some(EditStatus::Published { .. }) | |
| 872 | ) | |
| 873 | }; | |
| 874 | let first = replica.apply("Grace", typed(0..0, "A")).unwrap(); | |
| 875 | until("the first edit was never published", || { | |
| 876 | let _ = replica.sync_once(&mut HostedRemote::new(&guest, "Sealed.one")); | |
| 877 | published(first) | |
| 878 | }); | |
| 879 | HostedRemote::new(&guest, "Sealed.one").read().unwrap(); | |
| 880 | until("the host's stamp was never known without asking", || { | |
| 881 | host.touched(&["Sealed.one".into()]); | |
| 882 | let before = relay.state.lock().unwrap().access_bytes; | |
| 883 | HostedRemote::new(&guest, "Sealed.one").stamp().unwrap(); | |
| 884 | relay.state.lock().unwrap().access_bytes == before | |
| 885 | }); | |
| 886 | ||
| 887 | relay.state.lock().unwrap().swallow = true; | |
| 888 | let second = replica.apply("Grace", typed(0..0, "B")).unwrap(); | |
| 889 | assert!(matches!( | |
| 890 | replica.sync_once(&mut HostedRemote::new(&guest, "Sealed.one")), | |
| 891 | Err(Error::Remote(CommitError { | |
| 892 | state: CommitState::Unknown, | |
| 893 | .. | |
| 894 | })) | |
| 895 | )); | |
| 896 | until("the commit the relay lost never landed", || { | |
| 897 | let _ = replica.sync_once(&mut HostedRemote::new(&guest, "Sealed.one")); | |
| 898 | published(second) | |
| 899 | }); | |
| 900 | assert_eq!(stored(), ["BASealed text"]); | |
| 901 | ||
| 902 | let third = replica.apply("Grace", typed(0..0, "C")).unwrap(); | |
| 903 | until("the commit whose answer was lost never landed", || { | |
| 904 | replica | |
| 905 | .sync_once(&mut Unanswered(HostedRemote::new(&guest, "Sealed.one"))) | |
| 906 | .is_err() | |
| 907 | }); | |
| 908 | assert_eq!(stored(), ["CBASealed text"]); | |
| 909 | // The host deletes what the commit typed before the guest hears of it. | |
| 910 | let image = std::fs::read(&file).unwrap(); | |
| 911 | let arena = onestore::Arena::default(); | |
| 912 | let mut section = onestore::Section::unlock(&arena, image.clone(), &key).unwrap(); | |
| 913 | section.apply("Ada", &typed(0..1, "")).unwrap(); | |
| 914 | let mut image = image; | |
| 915 | section.seal().unwrap().unwrap().apply(&mut image).unwrap(); | |
| 916 | std::fs::write(&file, image).unwrap(); | |
| 917 | host.touched(&["Sealed.one".into()]); | |
| 918 | until("the commit that landed was never confirmed", || { | |
| 919 | let _ = replica.sync_once(&mut HostedRemote::new(&guest, "Sealed.one")); | |
| 920 | published(third) | |
| 921 | }); | |
| 922 | assert_eq!(stored(), ["BASealed text"]); | |
| 923 | } |