| author | |
| committer | |
| log | d1248829c53385b698b3081b8db058854239cc0c |
| tree | 635f072f8f9ee65f5c1a5ee5ac5a042dade4cb95 |
| parent | 126c7a891494ab1998b1e6de368edf62a80cc7ab |
| signature | Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU |
A merge named the revision it wrote after the merged version's revision alone.
Two devices merging one version onto different states of a page wrote
different contents under one revision identity, which MS-ONESTORE forbids, and
a later merge of their results took the file as already holding the other's
page and dropped a device's typing. The name now hashes the file revision
the merge depends on as well; onto no revision, as in a space the merge
creates, it stays the name older builds gave. Conflict page identities stay as
older builds wrote them, and the names older builds gave merges still count as
held, as devices run mixed builds. A seal refuses a name its space already
stores, so a mix-up sets the version aside rather than writing a duplicate
identity.
In the cloud-drive sweep (release, seeds 200 to 799) the strings lost this way
(seeds 342, 593 and 600) now arrive. Fourteen seeds still show a string twice,
where both devices merged one version and a third merge replays it, and seed
728 loses one to a version retired as held by a device whose copy was stale.
Assisted-by: claude-opus-5.59 files changed, 171 insertions(+), 43 deletions(-)
arc/sync.md+10-8| ... | ... | @@ -289,14 +289,16 @@ for each version V the remote lists (Remote::versions) |
| 289 | 289 | revisions, publish on C's stamp; then retire V (resolved, removed) |
| 290 | 290 | ``` |
| 291 | 291 | |
| 292 | Merged page revisions and conflict page identities are named after the incoming version's | |
| 293 | revision (a hash of it), so "holds merged" is a lookup by revision identity. A version merged | |
| 294 | once, by this device or another, merges as nothing the next time. Two devices merging the same | |
| 295 | version onto the same current page state name corresponding outcomes alike. This deduplicates | |
| 296 | retries; it does not make integrations from opposite primary bases order-independent or | |
| 297 | produce byte-identical files. Any device merges any version at once; nothing waits for the | |
| 298 | device that wrote it. A version of another section, or one that cannot be read, is kept beside | |
| 299 | the file as `Name (Device).one` instead. | |
| 292 | A merged page revision is named after the incoming version's revision and the file's revision it | |
| 293 | depends on (a one-way hash of both), and conflict page identities after the version's revision | |
| 294 | alone, so "holds merged" is a lookup by revision identity; the names older builds gave merges | |
| 295 | are recognised too, as devices run mixed builds. A version merged once, by this device or | |
| 296 | another, merges as nothing the next time. Two devices merging the same version onto the same | |
| 297 | current page state name corresponding outcomes alike; onto different states, differently, as | |
| 298 | their contents differ. This deduplicates retries; it does not make integrations from opposite | |
| 299 | primary bases order-independent or produce byte-identical files. Any device merges any version | |
| 300 | at once; nothing waits for the device that wrote it. A version of another section, or one that | |
| 301 | cannot be read, is kept beside the file as `Name (Device).one` instead. | |
| 300 | 302 | |
| 301 | 303 | iCloud Drive can also lose an upload outright: another device's lands over it and no conflict |
| 302 | 304 | version reaches this one. So on the Mac, the iPhone and Android each image a section publishes |
corpus/icloud-merge/older-build-merges/clash-merged.one created| Binary files /dev/null and b/corpus/icloud-merge/older-build-merges/clash-merged.one differ |
corpus/icloud-merge/older-build-merges/clash-version.one created| Binary files /dev/null and b/corpus/icloud-merge/older-build-merges/clash-version.one differ |
corpus/icloud-merge/older-build-merges/clean-merged.one created| Binary files /dev/null and b/corpus/icloud-merge/older-build-merges/clean-merged.one differ |
corpus/icloud-merge/older-build-merges/clean-version.one created| Binary files /dev/null and b/corpus/icloud-merge/older-build-merges/clean-version.one differ |
crates/notebook/README.md+6-4| ... | ... | @@ -149,10 +149,12 @@ stops at a checkpoint) the newest of the file's the version holds; the section o |
| 149 | 149 | revisions (`onestore::Section::open_at`) is the base the version's changes lower from |
| 150 | 150 | (`lower_page`), and they replay on the file through the rebase rules above, conflict pages where |
| 151 | 151 | they clash. A page whose two copies share no revision stays the file's, the version's kept |
| 152 | beside it as a conflict page (`corpus/icloud-merge/first-edit-checkpoints`). The revisions the merge writes are named after the version's it merged | |
| 153 | (`Section::seal_as`), so a version merged once, here or on another device, merges as nothing | |
| 154 | again. It is published on the file's stamp, then retired; one of another section, or one that | |
| 155 | cannot be read, is retired with `keep` for the remote to keep beside the file. | |
| 152 | beside it as a conflict page (`corpus/icloud-merge/first-edit-checkpoints`). The revisions the | |
| 153 | merge writes are named after the version's it merged and the file's they depend on | |
| 154 | (`Section::seal_as`), so a version merged once, here or on another device or by an older build, | |
| 155 | merges as nothing again, and merges of one version onto different states never share a name. It | |
| 156 | is published on the file's stamp, then retired; one of another section, or one that cannot be | |
| 157 | read, is retired with `keep` for the remote to keep beside the file. | |
| 156 | 158 | A provider can also lose an upload outright, another device's landing over it with no version |
| 157 | 159 | kept. A remote keeps what it publishes with `published::Published` (`keep` after each |
| 158 | 160 | publication) and lists each image the file no longer stands as among its versions |
crates/notebook/src/resolve.rs+42-20| ... | ... | @@ -4,11 +4,12 @@ |
| 4 | 4 | //! (`merge.rs`), with conflict pages where they clash, as a queue replays on a changed |
| 5 | 5 | //! remote. |
| 6 | 6 | //! |
| 7 | //! Each revision the merge writes is named after the version's revision it merged | |
| 8 | //! (`merged`). A space whose newest revision the file holds, or holds merged, needs nothing; | |
| 9 | //! older ones are where the version and the file last agreed. So a version merged once, by | |
| 10 | //! this device or another, merges as nothing the next time, and two devices merging it at | |
| 11 | //! once write revisions of the same names, which the next merge of the two finds held. | |
| 7 | //! Each revision the merge writes is named after the version's revision it merged and the | |
| 8 | //! file's revision it merged onto (`merged`), and older builds' names (`named`) still count. | |
| 9 | //! A space whose newest revision the file holds, or holds merged, needs nothing; older ones | |
| 10 | //! are where the version and the file last agreed. So a version merged once, by this device or | |
| 11 | //! another, merges as nothing the next time, and two devices merging it onto one state write | |
| 12 | //! revisions of the same names, which the next merge of the two finds held. | |
| 12 | 13 | |
| 13 | 14 | use crate::{PendingEdit, Result, merge}; |
| 14 | 15 | use onestore::{ |
| ... | ... | @@ -39,9 +40,9 @@ pub(crate) enum Merged { |
| 39 | 40 | Foreign, |
| 40 | 41 | } |
| 41 | 42 | |
| 42 | /// The name of the revision a merge writes in a space for the version's revision `rid`, a | |
| 43 | /// different one for each `kind` of space it writes. | |
| 44 | fn merged(rid: ExGuid, kind: &str) -> ExGuid { | |
| 43 | /// What older builds named a merge of the version's revision `rid`: the revisions it wrote | |
| 44 | /// (`"page"`), and the conflict pages it made (`"conflict"`), which keep the name. | |
| 45 | fn named(rid: ExGuid, kind: &str) -> ExGuid { | |
| 45 | 46 | let digest = Sha256::new() |
| 46 | 47 | .chain_update(b"snowbound merge ") |
| 47 | 48 | .chain_update(kind) |
| ... | ... | @@ -53,6 +54,25 @@ fn merged(rid: ExGuid, kind: &str) -> ExGuid { |
| 53 | 54 | ExGuid { guid, n: 1 } |
| 54 | 55 | } |
| 55 | 56 | |
| 57 | /// The name of the revision a merge writes for the version's revision `rid` onto the file's | |
| 58 | /// revision `onto` of that space, which it depends on: two merges of one version onto | |
| 59 | /// different states are different revisions. Onto no revision, as in a space the merge | |
| 60 | /// creates, it is the name older builds gave. | |
| 61 | fn merged(rid: ExGuid, onto: Option<ExGuid>) -> ExGuid { | |
| 62 | let mut digest = Sha256::new() | |
| 63 | .chain_update(b"snowbound merge page") | |
| 64 | .chain_update(rid.guid) | |
| 65 | .chain_update(rid.n.to_le_bytes()); | |
| 66 | if let Some(onto) = onto { | |
| 67 | digest = digest | |
| 68 | .chain_update(onto.guid) | |
| 69 | .chain_update(onto.n.to_le_bytes()); | |
| 70 | } | |
| 71 | let mut guid = [0; 16]; | |
| 72 | guid.copy_from_slice(&digest.finalize()[..16]); | |
| 73 | ExGuid { guid, n: 1 } | |
| 74 | } | |
| 75 | ||
| 56 | 76 | /// Merges `version` into `current`, the file as it stands, `device` naming who saved the |
| 57 | 77 | /// version. |
| 58 | 78 | pub(crate) fn merge(current: &[u8], version: &[u8], device: &str) -> Result<Merged> { |
| ... | ... | @@ -79,18 +99,16 @@ pub(crate) fn merge(current: &[u8], version: &[u8], device: &str) -> Result<Merg |
| 79 | 99 | let meets = |rid: ExGuid| { |
| 80 | 100 | let held = ours.is_some_and(|ours| { |
| 81 | 101 | ours.revisions.contains_key(&rid) |
| 82 | || ours.revisions.contains_key(&merged(rid, "page")) | |
| 83 | }) || file.spaces.contains_key(&ExGuid { | |
| 84 | guid: merged(rid, "conflict").guid, | |
| 85 | n: 1, | |
| 86 | }); | |
| 102 | || ours.revisions.contains_key(&named(rid, "page")) | |
| 103 | || (ours.revisions.iter()) | |
| 104 | .any(|(merge, revision)| *merge == merged(rid, revision.dependency)) | |
| 105 | }) || file.spaces.contains_key(&named(rid, "conflict")); | |
| 87 | 106 | if held { |
| 88 | 107 | return Some(rid); |
| 89 | 108 | } |
| 90 | ours? | |
| 91 | .revisions | |
| 92 | .keys() | |
| 93 | .find(|theirs| merged(**theirs, "page") == rid) | |
| 109 | let onto = revisions.revisions.get(&rid)?.dependency; | |
| 110 | (ours?.revisions.keys()) | |
| 111 | .find(|theirs| merged(**theirs, onto) == rid || named(**theirs, "page") == rid) | |
| 94 | 112 | .copied() |
| 95 | 113 | }; |
| 96 | 114 | // Failing that, as when the version's chain stops at a checkpoint, the newest state of |
| ... | ... | @@ -252,9 +270,13 @@ pub(crate) fn merge(current: &[u8], version: &[u8], device: &str) -> Result<Merg |
| 252 | 270 | for space in &unrelated { |
| 253 | 271 | clashes.entry(*space).or_default(); |
| 254 | 272 | } |
| 273 | let onto = |space: &ExGuid| { | |
| 274 | let labels = &file.spaces.get(space)?.labels; | |
| 275 | labels.get(&(ExGuid::default(), 1)).copied() | |
| 276 | }; | |
| 255 | 277 | let mut names: BTreeMap<ExGuid, ExGuid> = changed |
| 256 | 278 | .iter() |
| 257 | .filter_map(|space| Some((*space, merged(head(space)?, "page")))) | |
| 279 | .filter_map(|space| Some((*space, merged(head(space)?, onto(space))))) | |
| 258 | 280 | .collect(); |
| 259 | 281 | for (space, objects) in &clashes { |
| 260 | 282 | let Ok(page) = theirs.page(*space) else { |
| ... | ... | @@ -264,7 +286,7 @@ pub(crate) fn merge(current: &[u8], version: &[u8], device: &str) -> Result<Merg |
| 264 | 286 | continue; |
| 265 | 287 | }; |
| 266 | 288 | // Named after the version's page, so that every merge of it makes the same page. |
| 267 | let guid = merged(rid, "conflict").guid; | |
| 289 | let guid = named(rid, "conflict").guid; | |
| 268 | 290 | merge::conflict_page( |
| 269 | 291 | &mut new, |
| 270 | 292 | &mut theirs, |
| ... | ... | @@ -276,7 +298,7 @@ pub(crate) fn merge(current: &[u8], version: &[u8], device: &str) -> Result<Merg |
| 276 | 298 | crate::now(), |
| 277 | 299 | Some(guid), |
| 278 | 300 | )?; |
| 279 | names.insert(ExGuid { guid, n: 1 }, merged(rid, "page")); | |
| 301 | names.insert(ExGuid { guid, n: 1 }, merged(rid, onto(space))); | |
| 280 | 302 | } |
| 281 | 303 | return Ok(match new.seal_as(&names)? { |
| 282 | 304 | Some(transaction) => Merged::Publish(Box::new(transaction)), |
crates/notebook/tests/cloud.rs+99| ... | ... | @@ -1206,3 +1206,102 @@ fn a_page_past_the_chain_cap_on_one_device_merges_as_edits() { |
| 1206 | 1206 | ); |
| 1207 | 1207 | } |
| 1208 | 1208 | } |
| 1209 | ||
| 1210 | /// Two devices merge one version onto different states of a page, then their results meet: | |
| 1211 | /// each merge is its own revision, so neither device's typing is taken as held by the other | |
| 1212 | /// (sweep seed 600). The version's own typing, merged on both sides, may show twice: the limit | |
| 1213 | /// on integrations from opposite bases in `arc/sync.md`. | |
| 1214 | #[test] | |
| 1215 | fn merges_of_one_version_onto_different_states_keep_both() { | |
| 1216 | let source = fixture(); | |
| 1217 | let arena = onestore::Arena::default(); | |
| 1218 | let mut section = Section::open(&arena, source.clone()).unwrap(); | |
| 1219 | let space = section.pages().unwrap()[0].0; | |
| 1220 | let (text, body) = bodies(&section.page(space).unwrap())[0].clone(); | |
| 1221 | let end = body.encode_utf16().count() as u32; | |
| 1222 | let typed = |at: u32, with: &str| { | |
| 1223 | let arena = onestore::Arena::default(); | |
| 1224 | let mut section = Section::open(&arena, source.clone()).unwrap(); | |
| 1225 | let op = PageOp::Text { | |
| 1226 | text, | |
| 1227 | range: at..at, | |
| 1228 | with: with.into(), | |
| 1229 | }; | |
| 1230 | edit(&mut section, "Device", vec![Op::Page { space, op }]); | |
| 1231 | section.seal().unwrap(); | |
| 1232 | section.image() | |
| 1233 | }; | |
| 1234 | let (a, b, c) = (typed(0, "⟨A⟩"), typed(end, "⟨B⟩"), typed(4, "⟨C⟩")); | |
| 1235 | let onto_a = merged(&[a, c.clone()], 0); | |
| 1236 | let onto_b = merged(&[b, c], 0); | |
| 1237 | for winner in 0..2 { | |
| 1238 | let image = merged(&[onto_a.clone(), onto_b.clone()], winner); | |
| 1239 | let (listed, conflicts) = texts_of(&image); | |
| 1240 | for (token, shown) in [("⟨A⟩", 1..=1), ("⟨B⟩", 1..=1), ("⟨C⟩", 1..=2)] { | |
| 1241 | let count = listed.matches(token).count(); | |
| 1242 | assert!(shown.contains(&count), "{winner} {token}:\n{listed}"); | |
| 1243 | } | |
| 1244 | assert!(conflicts.is_empty(), "{winner}: {conflicts}"); | |
| 1245 | } | |
| 1246 | } | |
| 1247 | ||
| 1248 | /// A version an older build merged, under the names it gave merges, merges as nothing again: | |
| 1249 | /// neither its clean merge nor its conflict page comes back | |
| 1250 | /// (`corpus/icloud-merge/older-build-merges`). | |
| 1251 | #[test] | |
| 1252 | fn a_version_an_older_build_merged_merges_as_nothing() { | |
| 1253 | let row = format!( | |
| 1254 | "{}/../../corpus/icloud-merge/older-build-merges", | |
| 1255 | env!("CARGO_MANIFEST_DIR") | |
| 1256 | ); | |
| 1257 | for name in ["clean", "clash"] { | |
| 1258 | let read = |kind: &str| std::fs::read(format!("{row}/{name}-{kind}.one")).unwrap(); | |
| 1259 | let file = read("merged"); | |
| 1260 | assert!( | |
| 1261 | merged(&[file.clone(), read("version")], 0) == file, | |
| 1262 | "{name}" | |
| 1263 | ); | |
| 1264 | } | |
| 1265 | } | |
| 1266 | ||
| 1267 | /// A merge of a merge onto the same state is a revision of its own: Z's version merged into X's | |
| 1268 | /// file, that file merged into W's on X's state, and W's file merged back into Z's copy keeps | |
| 1269 | /// X's typing. Z's own, merged on both sides, may show twice, as in the sweep's duplicates. | |
| 1270 | #[test] | |
| 1271 | fn a_merge_merged_again_onto_the_same_state_is_its_own_revision() { | |
| 1272 | let source = fixture(); | |
| 1273 | let arena = onestore::Arena::default(); | |
| 1274 | let mut section = Section::open(&arena, source.clone()).unwrap(); | |
| 1275 | let pages = section.pages().unwrap(); | |
| 1276 | let space = pages[0].0; | |
| 1277 | let (text, words) = bodies(&section.page(space).unwrap())[0].clone(); | |
| 1278 | let end = words.encode_utf16().count() as u32; | |
| 1279 | let typed = |image: Vec<u8>, space, text, at: u32, with: &str| { | |
| 1280 | let op = PageOp::Text { | |
| 1281 | text, | |
| 1282 | range: at..at, | |
| 1283 | with: with.into(), | |
| 1284 | }; | |
| 1285 | let arena = onestore::Arena::default(); | |
| 1286 | let mut section = Section::open(&arena, image).unwrap(); | |
| 1287 | edit(&mut section, "Device", vec![Op::Page { space, op }]); | |
| 1288 | section.seal().unwrap(); | |
| 1289 | section.image() | |
| 1290 | }; | |
| 1291 | let x = typed(source.clone(), space, text, end, "⟨X⟩"); | |
| 1292 | let z = typed(source.clone(), space, text, 0, "⟨Z⟩"); | |
| 1293 | let onto_x = merged(&[x.clone(), z.clone()], 0); | |
| 1294 | let other = pages[1].0; | |
| 1295 | let (text, _) = bodies(&section.page(other).unwrap())[0].clone(); | |
| 1296 | let w = typed(x, other, text, 0, "⟨W⟩"); | |
| 1297 | let onto_w = merged(&[w, onto_x], 0); | |
| 1298 | let image = merged(&[z, onto_w], 0); | |
| 1299 | let (listed, conflicts) = texts_of(&image); | |
| 1300 | for (token, shown) in [("⟨X⟩", 1..=1), ("⟨W⟩", 1..=1), ("⟨Z⟩", 1..=2)] { | |
| 1301 | let count = listed.matches(token).count(); | |
| 1302 | assert!( | |
| 1303 | shown.contains(&count), | |
| 1304 | "{token}:\n{listed}\n--\n{conflicts}" | |
| 1305 | ); | |
| 1306 | } | |
| 1307 | } |
crates/onestore/src/section.rs+14-11| ... | ... | @@ -820,17 +820,20 @@ impl<'a> Section<'a> { |
| 820 | 820 | /// merged, so that the next merge knows it holds that. |
| 821 | 821 | pub fn seal_as(&mut self, names: &BTreeMap<ExGuid, ExGuid>) -> Result<Option<Transaction>> { |
| 822 | 822 | self.usable()?; |
| 823 | if names.iter().any(|(space, name)| { | |
| 824 | name.guid == [0; 16] | |
| 825 | || self | |
| 826 | .spaces | |
| 827 | .get(space) | |
| 828 | .is_some_and(|stored| stored.rid == Some(*name)) | |
| 829 | }) { | |
| 830 | return Err(Error { | |
| 831 | offset: 0, | |
| 832 | message: "Name each revision anew", | |
| 833 | }); | |
| 823 | if !names.is_empty() { | |
| 824 | let image = self.image(); | |
| 825 | let store = Store::parse(&image)?; | |
| 826 | let index = RevisionIndex::parse(&store)?; | |
| 827 | if names.iter().any(|(space, name)| { | |
| 828 | name.guid == [0; 16] | |
| 829 | || (index.spaces.get(space)) | |
| 830 | .is_some_and(|stored| stored.revisions.contains_key(name)) | |
| 831 | }) { | |
| 832 | return Err(Error { | |
| 833 | offset: 0, | |
| 834 | message: "Name each revision anew", | |
| 835 | }); | |
| 836 | } | |
| 834 | 837 | } |
| 835 | 838 | self.broken = true; |
| 836 | 839 | let arena = &self.arena.0; |