authoravatarclover caruso<git@paperclover.net>2026-10-13 19:57:38-07:00
committeravatarclover caruso<git@paperclover.net>2026-10-21 02:52:29-07:00
log796fc67fb8d5f978f496b727f34dc28347747940
tree60f7beb19fa7dd72e9c06a7d9c29688c21775c8f
parent8bfd74a99a223fc46af93692a4a5827d1fa98930
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU

Manage Copyparty shares and folder access from Snowglobe

Add admin-managed folder grants for Snow accounts and standalone API accounts, with scoped permissions, one-time credentials, and session revocation. Preserve native public links and selected-file shares through explicit retarget and offline move actions. Persist management state through deployment and staging, and add a bounded, reversible cutover for the six requested shared folders. Prefer Zenith staging previews for service verification. Assisted-by: gpt-6.1-sol

26 files changed, 2321 insertions(+), 50 deletions(-)

AGENTS.md+13-8
...@@ -7,14 +7,15 @@ availability and existing data as part of correctness....@@ -7,14 +7,15 @@ availability and existing data as part of correctness.
77
8## Access and authorization8## Access and authorization
99
10- Local edits and rehearsal VM changes are always implied by a task. Carry10- Local edits and staging previews on Zenith are always implied by a task.
11 them through staging and verification without asking for permission again.11 Carry them through verification without asking for permission again. Use
12 disposable preview data for writes and keep active production services intact.
12- Production changes require a request to take the work to prod. A request to13- Production changes require a request to take the work to prod. A request to
13 fix, stage, test, or commit something does not authorize production deployment.14 fix, stage, test, or commit something does not authorize production deployment.
14 Once prod is requested, complete the normal deployment workflow.15 Once prod is requested, complete the normal deployment workflow.
15- Read-only infrastructure inspection over SSH is allowed for the task. Confirm16- Read-only infrastructure inspection over SSH is allowed for the task. Confirm
16 the target host before a remote mutation; deployment defaults point at the17 the target host before a remote mutation. Set both deployment host and port
17 rehearsal VM, but inherited environment variables can select production.18 explicitly; the CLI's legacy defaults still point at the rehearsal VM.
18- Private file contents require authorization per source. Infrastructure access,19- Private file contents require authorization per source. Infrastructure access,
19 a mounted share, or permission to list a tree does not authorize opening its20 a mounted share, or permission to list a tree does not authorize opening its
20 contents. Keep private material and derived model output inside its store.21 contents. Keep private material and derived model output inside its store.
...@@ -35,11 +36,13 @@ availability and existing data as part of correctness....@@ -35,11 +36,13 @@ availability and existing data as part of correctness.
35| Surface | Connection / role | Configuration |36| Surface | Connection / role | Configuration |
36| --- | --- | --- |37| --- | --- | --- |
37| Mac checkout | `/Users/clo/dev/infra-2`; edit and upload here | This working copy |38| Mac checkout | `/Users/clo/dev/infra-2`; edit and upload here | This working copy |
38| Rehearsal NixOS VM | `root@127.0.0.1`, SSH port `2222` | `nixos/vm.nix`, `studio.test`, demo pool |39| Optional rehearsal NixOS VM | `root@127.0.0.1`, SSH port `2222` | `nixos/vm.nix`, `studio.test`, demo pool |
39| Zenith production | `root@10.0.0.1`, SSH port `22` | `nixos/zenith.nix`, `paperclover.net`, physical pool |40| Zenith production | `root@10.0.0.1`, SSH port `22` | `nixos/zenith.nix`, `paperclover.net`, physical pool |
4041
41`tools/deploy.py` reads `STUDIO_DEPLOY_HOST` and `STUDIO_DEPLOY_PORT`; defaults42`tools/deploy.py` reads `STUDIO_DEPLOY_HOST` and `STUDIO_DEPLOY_PORT`; defaults
42are the rehearsal connection above. Set both explicitly for production.43are the rehearsal connection above. Set both explicitly for Zenith staging
44and production. Prefer staging previews on Zenith for service and dashboard
45verification; use the VM when host or guest isolation is required.
43Host settings and dataset names have their canonical home in `nixos/zenith.nix`46Host settings and dataset names have their canonical home in `nixos/zenith.nix`
44and `config/site.pkl`; inspect them instead of copying values into new tooling.47and `config/site.pkl`; inspect them instead of copying values into new tooling.
4548
...@@ -99,7 +102,7 @@ Mac rehearsal routing. Do not hand-edit DNS, trust, or host files to bypass them...@@ -99,7 +102,7 @@ Mac rehearsal routing. Do not hand-edit DNS, trust, or host files to bypass them
993. Run the relevant local checks, then stage the working files:1023. Run the relevant local checks, then stage the working files:
100103
101 ```sh104 ```sh
102 STUDIO_DEPLOY_HOST=root@127.0.0.1 STUDIO_DEPLOY_PORT=2222 python3 tools/deploy.py stage <service>105 STUDIO_DEPLOY_HOST=root@10.0.0.1 STUDIO_DEPLOY_PORT=22 python3 tools/deploy.py stage <service>
103 ```106 ```
104107
105 The command prints `stage`, `release`, `host`, and `dataset`. Use that host;108 The command prints `stage`, `release`, `host`, and `dataset`. Use that host;
...@@ -135,7 +138,8 @@ read `tools/data-restore.md` before proposing it....@@ -135,7 +138,8 @@ read `tools/data-restore.md` before proposing it.
135138
136- Run checks that cover the change. Inspect a test's entry point first: some139- Run checks that cover the change. Inspect a test's entry point first: some
137 `tools/*-test.py` scripts mutate host services or require Linux/real guests.140 `tools/*-test.py` scripts mutate host services or require Linux/real guests.
138 Run those in rehearsal, not on production by default.141 Run those against isolated staging fixtures. Tests that alter host services,
142 storage, or guests require a disposable VM or explicit production scope.
139- Dashboard: `cd dashboard && npm run check && npm run build`; run Rust tests143- Dashboard: `cd dashboard && npm run check && npm run build`; run Rust tests
140 with `cargo test --manifest-path dashboard/Cargo.toml`. Host tests include144 with `cargo test --manifest-path dashboard/Cargo.toml`. Host tests include
141 `python3 tools/dashboard-host-test.py` and `python3 tools/dashboard-vms-test.py`.145 `python3 tools/dashboard-host-test.py` and `python3 tools/dashboard-vms-test.py`.
...@@ -169,6 +173,7 @@ read `tools/data-restore.md` before proposing it....@@ -169,6 +173,7 @@ read `tools/data-restore.md` before proposing it.
169 [tools/personal-cutover.md](tools/personal-cutover.md), and the service's173 [tools/personal-cutover.md](tools/personal-cutover.md), and the service's
170 `tools/*-migration.md` runbook.174 `tools/*-migration.md` runbook.
171- Provider behavior: [tools/service-dependencies.md](tools/service-dependencies.md).175- Provider behavior: [tools/service-dependencies.md](tools/service-dependencies.md).
176- Copyparty administration and folder moves: [tools/copyparty.md](tools/copyparty.md).
172- Local inference: [tools/local-ai.md](tools/local-ai.md).177- Local inference: [tools/local-ai.md](tools/local-ai.md).
173178
174Runbooks contain dated rehearsal evidence and migration history. Check current179Runbooks contain dated rehearsal evidence and migration history. Check current
dashboard/src/auth.rs+39-7
...@@ -571,6 +571,23 @@ impl Store {...@@ -571,6 +571,23 @@ impl Store {
571 }571 }
572}572}
573573
574fn file_groups(user: &Value) -> Option<String> {
575 if user["enabled"] != true
576 || !array(&user["requiredActions"]).is_empty()
577 || guest::is_guest(user)
578 || user["kind"] == "ai"
579 {
580 return None;
581 }
582 let id = uuid::Uuid::parse_str(string(&user["id"])).ok()?;
583 let mut groups = array(&user["groups"])
584 .iter()
585 .map(|g| string(&g["name"]).to_owned())
586 .collect::<Vec<_>>();
587 groups.push(format!("snow-user-{id}"));
588 Some(groups.join(","))
589}
590
574pub async fn route(State(app): State<Arc<App>>, request: Request) -> Result<Response> {591pub async fn route(State(app): State<Arc<App>>, request: Request) -> Result<Response> {
575 if request.uri().path() == "/auth/shale/page" {592 if request.uri().path() == "/auth/shale/page" {
576 return shale_page::proxy(app, request).await;593 return shale_page::proxy(app, request).await;
...@@ -591,14 +608,9 @@ pub async fn route(State(app): State<Arc<App>>, request: Request) -> Result<Resp...@@ -591,14 +608,9 @@ pub async fn route(State(app): State<Arc<App>>, request: Request) -> Result<Resp
591 let headers = request.headers().clone();608 let headers = request.headers().clone();
592 if path == "/auth/file/check" && method == Method::GET {609 if path == "/auth/file/check" && method == Method::GET {
593 let user = auth.session(&headers, "file")?;610 let user = auth.session(&headers, "file")?;
594 if user.is_null() || !array(&user["requiredActions"]).is_empty() {611 let Some(groups) = file_groups(&user) else {
595 return Ok(StatusCode::UNAUTHORIZED.into_response());612 return Ok(StatusCode::UNAUTHORIZED.into_response());
596 }613 };
597 let groups = array(&user["groups"])
598 .iter()
599 .map(|g| string(&g["name"]))
600 .collect::<Vec<_>>()
601 .join(",");
602 return Ok((614 return Ok((
603 StatusCode::NO_CONTENT,615 StatusCode::NO_CONTENT,
604 [616 [
...@@ -1284,3 +1296,23 @@ fn passkeys(db: &Connection, id: &str) -> Result<Vec<Passkey>> {...@@ -1284,3 +1296,23 @@ fn passkeys(db: &Connection, id: &str) -> Result<Vec<Passkey>> {
1284 })1296 })
1285 .collect()1297 .collect()
1286}1298}
1299
1300#[cfg(test)]
1301mod tests {
1302 use super::*;
1303
1304 #[test]
1305 fn folder_identity_tracks_the_account_id_and_requires_an_enabled_regular_account() {
1306 let mut account = json!({"id":"12345678-1234-1234-1234-123456789abc","username":"snow","enabled":true,"requiredActions":[],"groups":[{"name":"media"}]});
1307 assert_eq!(file_groups(&account).unwrap(), "media,snow-user-12345678-1234-1234-1234-123456789abc");
1308 account["username"] = json!("different");
1309 assert_eq!(file_groups(&account).unwrap(), "media,snow-user-12345678-1234-1234-1234-123456789abc");
1310 account["id"] = json!("12345678-1234-1234-1234-123456789abd");
1311 assert!(file_groups(&account).unwrap().ends_with("snow-user-12345678-1234-1234-1234-123456789abd"));
1312 for (field, value) in [("enabled", json!(false)), ("kind", json!("guest")), ("kind", json!("ai")), ("requiredActions", json!(["UPDATE_PASSWORD"])), ("id", json!("x,infra-admin"))] {
1313 let mut blocked = account.clone();
1314 blocked[field] = value;
1315 assert!(file_groups(&blocked).is_none());
1316 }
1317 }
1318}
dashboard/src/host.rs+2
...@@ -34,6 +34,8 @@ pub async fn open(request: Value) -> Result<(Value, tokio::net::UnixStream)> {...@@ -34,6 +34,8 @@ pub async fn open(request: Value) -> Result<(Value, tokio::net::UnixStream)> {
34 Some("vm.create" | "vm.media" | "vm.preset")34 Some("vm.create" | "vm.media" | "vm.preset")
35 ) {35 ) {
36 91036 910
37 } else if request["operation"] == "copyparty.manage" {
38 310
37 } else {39 } else {
38 7040 70
39 };41 };
dashboard/src/main.rs+3-1
...@@ -15,6 +15,7 @@ mod oidc;...@@ -15,6 +15,7 @@ mod oidc;
15mod relay;15mod relay;
16mod shale;16mod shale;
17mod shale_page;17mod shale_page;
18mod shares;
18mod storage;19mod storage;
19mod telemetry;20mod telemetry;
20mod users;21mod users;
...@@ -269,7 +270,7 @@ async fn api(State(app): State<Arc<App>>, request: Request) -> Result<Response>...@@ -269,7 +270,7 @@ async fn api(State(app): State<Arc<App>>, request: Request) -> Result<Response>
269 "services" if parts.len() == 1 => Some("metrics"),270 "services" if parts.len() == 1 => Some("metrics"),
270 "storage" if parts.len() == 1 => Some("metrics"),271 "storage" if parts.len() == 1 => Some("metrics"),
271 "ai" | "mcp" => Some("ai"),272 "ai" | "mcp" => Some("ai"),
272 "services" | "traces" | "storage" | "users" | "deploys" | "paper-clover" => {273 "services" | "traces" | "storage" | "users" | "shares" | "deploys" | "paper-clover" => {
273 Some("admin")274 Some("admin")
274 }275 }
275 "media" | "seedbox" | "youtube" => Some("media"),276 "media" | "seedbox" | "youtube" => Some("media"),
...@@ -322,6 +323,7 @@ async fn api(State(app): State<Arc<App>>, request: Request) -> Result<Response>...@@ -322,6 +323,7 @@ async fn api(State(app): State<Arc<App>>, request: Request) -> Result<Response>
322 "ai" => ai::route(app, &method, &parts[1..], &me, &headers).await,323 "ai" => ai::route(app, &method, &parts[1..], &me, &headers).await,
323 "mcp" => mcp::manage(app, &method, &parts[1..], &me, value, &headers).await,324 "mcp" => mcp::manage(app, &method, &parts[1..], &me, value, &headers).await,
324 "users" => users::route(app, &method, &parts[1..], &me, value).await,325 "users" => users::route(app, &method, &parts[1..], &me, value).await,
326 "shares" => shares::route(app, &method, &parts[1..], &query, &me, value).await,
325 "vms" | "seedbox" | "paper-clover" => {327 "vms" | "seedbox" | "paper-clover" => {
326 apps::route(app, &method, &parts, &query, &me, value).await328 apps::route(app, &method, &parts, &query, &me, value).await
327 }329 }
dashboard/src/shares.rs created+92
...@@ -0,0 +1,92 @@
1use crate::*;
2
3fn operation(method: &Method, parts: &[&str], mut body: Value) -> Result<(&'static str, Value)> {
4 let action = match parts {
5 [] if method == Method::GET => return Ok(("state", json!({}))),
6 ["folders"] if method == Method::GET => return Ok(("folders.list", body)),
7 ["accounts"] if method == Method::POST => "account.create",
8 ["accounts", _] if method == Method::PATCH => "account.update",
9 ["accounts", _] if method == Method::DELETE => "account.delete",
10 ["accounts", _, "rotate"] if method == Method::POST => "account.rotate",
11 ["grants"] if method == Method::POST => "grant.save",
12 ["grants", _] if method == Method::PATCH => "grant.save",
13 ["grants", _] if method == Method::DELETE => "grant.delete",
14 ["links"] if method == Method::POST => "link.create",
15 ["links", _] if method == Method::PATCH => "link.update",
16 ["links", _] if method == Method::DELETE => "link.delete",
17 _ => return Err(Error::new(404, "No share action here.")),
18 };
19 if body.is_null() {
20 body = json!({});
21 }
22 let fields = body
23 .as_object_mut()
24 .ok_or_else(|| Error::new(400, "Reload shares and try again."))?;
25 if let Some(id) = parts.get(1) {
26 let key = if parts[0] == "links" { "key" } else { "id" };
27 if fields.contains_key(key) {
28 return Err(Error::new(400, "Reload shares before saving this change."));
29 }
30 fields.insert(key.into(), json!(id));
31 } else if parts == ["grants"] {
32 fields.insert("id".into(), Value::Null);
33 }
34 Ok((action, body))
35}
36
37pub async fn route(
38 app: Arc<App>,
39 method: &Method,
40 parts: &[&str],
41 query: &HashMap<String, String>,
42 me: &Value,
43 body: Value,
44) -> Result<Response> {
45 need(me, "admin")?;
46 let body = if parts == ["folders"] && method == Method::GET {
47 json!({"path":query.get("path").map(String::as_str).unwrap_or_default()})
48 } else {
49 body
50 };
51 let (action, payload) = operation(method, parts, body)?;
52 let users = {
53 let db = app.auth.db.lock().unwrap();
54 let mut statement = db.prepare("SELECT id FROM users ORDER BY username")?;
55 statement
56 .query_map([], |row| row.get::<_, String>(0))?
57 .map(|id| {
58 let id = id?;
59 let profile = auth::user(&db, &id)?;
60 let groups = array(&profile["groups"]).iter().map(|group| group["name"].clone()).collect::<Vec<_>>();
61 Ok(json!({"id":id,"username":profile["username"],"enabled":profile["enabled"],"kind":profile["kind"],"requiredActions":profile["requiredActions"],"groups":groups}))
62 })
63 .collect::<Result<Vec<_>>>()?
64 };
65 let mut value = host::call(json!({
66 "operation":"copyparty.manage", "action":action, "payload":payload, "users":users,
67 }))
68 .await?;
69 if value.is_object() {
70 value["filesUrl"] = json!(env(
71 "STUDIO_FILES_URL",
72 &format!("https://file.{}", env("STUDIO_DOMAIN", "studio.test")),
73 ));
74 }
75 Ok(Document::new(value).response())
76}
77
78#[cfg(test)]
79mod tests {
80 use super::*;
81
82 #[test]
83 fn route_identity_cannot_be_overridden_by_the_body() {
84 assert!(operation(&Method::PATCH, &["links", "kept"], json!({"key":"other"})).is_err());
85 assert!(operation(&Method::PATCH, &["accounts", "kept"], json!({"id":"other"})).is_err());
86 let (action, payload) =
87 operation(&Method::DELETE, &["links", "kept"], Value::Null).unwrap();
88 assert_eq!(action, "link.delete");
89 assert_eq!(payload, json!({"key":"kept"}));
90 assert!(operation(&Method::GET, &["accounts", "kept", "rotate"], Value::Null).is_err());
91 }
92}
dashboard/src/users.rs+15-2
...@@ -35,8 +35,8 @@ fn profile(body: &Value, full: bool) -> Result<Value> {...@@ -35,8 +35,8 @@ fn profile(body: &Value, full: bool) -> Result<Value> {
35 let value = match key {35 let value = match key {
36 "username" => {36 "username" => {
37 let v = string(value).trim().to_lowercase();37 let v = string(value).trim().to_lowercase();
38 if v.starts_with("ai-") {38 if v.starts_with("ai-") || v.starts_with("api-") {
39 return Err(Error::new(400, "Usernames beginning with ai- are reserved for model accounts."));39 return Err(Error::new(400, "Choose a username without the ai- or api- prefix. Those are reserved for service accounts."));
40 }40 }
41 if v.len() > 254 || !username_pattern.is_match(&v) {41 if v.len() > 254 || !username_pattern.is_match(&v) {
42 return Err(Error::new(42 return Err(Error::new(
...@@ -635,3 +635,16 @@ pub async fn account(...@@ -635,3 +635,16 @@ pub async fn account(
635 Document::new(value).response()635 Document::new(value).response()
636 })636 })
637}637}
638
639#[cfg(test)]
640mod tests {
641 use super::*;
642
643 #[test]
644 fn people_cannot_claim_service_account_usernames() {
645 for username in ["api-account", "API-account", "ai-account"] {
646 assert!(profile(&json!({"username":username}), false).is_err());
647 }
648 assert!(profile(&json!({"username":"person"}), false).is_ok());
649 }
650}
dashboard/web/api.contract.ts+19
...@@ -9,6 +9,7 @@ import type { Domain, History, NewDomain, VMLibrary } from "./types/vms.ts";...@@ -9,6 +9,7 @@ import type { Domain, History, NewDomain, VMLibrary } from "./types/vms.ts";
9import type { Me, ServiceSummary, ServiceDetail, ServiceDefinition, HostInfo, Issue, Series, LogLine, TraceSummary, Trace } from "./types/model.ts";9import type { Me, ServiceSummary, ServiceDetail, ServiceDefinition, HostInfo, Issue, Series, LogLine, TraceSummary, Trace } from "./types/model.ts";
10import type { MapNode } from "./types/storage.index.ts";10import type { MapNode } from "./types/storage.index.ts";
11import type { ProgressNode, PaperCloverStats } from "./types/paperClover.ts";11import type { ProgressNode, PaperCloverStats } from "./types/paperClover.ts";
12import type { AccountSecret, FolderGrant, LinkInput, ShareState } from "./types/shares.ts";
1213
13type Endpoint<Output, Input = {}, Status extends number = 200, Format extends string = "json"> = {14type Endpoint<Output, Input = {}, Status extends number = 200, Format extends string = "json"> = {
14 input: Input; output: Output; status: Status; outputFormat: Format;15 input: Input; output: Output; status: Status; outputFormat: Format;
...@@ -54,6 +55,24 @@ type Explorer = {...@@ -54,6 +55,24 @@ type Explorer = {
54type ExplorerRoutes<Prefix extends string> = { [P in keyof Explorer as `${Prefix}${P}`]: Explorer[P] };55type ExplorerRoutes<Prefix extends string> = { [P in keyof Explorer as `${Prefix}${P}`]: Explorer[P] };
5556
56export type Api = Hono<{}, {57export type Api = Hono<{}, {
58 "/shares": { $get: Endpoint<ShareState>; };
59 "/shares/folders": { $get: Endpoint<{ path: string; directories: { name: string; path: string }[] }, { query: { path: string } }>; };
60 "/shares/accounts": { $post: Endpoint<AccountSecret, { json: { name: string } }>; };
61 "/shares/accounts/:id": {
62 $patch: Endpoint<ShareState, { param: { id: string }; json: { name: string } }>;
63 $delete: Endpoint<ShareState, { param: { id: string } }>;
64 };
65 "/shares/accounts/:id/rotate": { $post: Endpoint<AccountSecret, { param: { id: string } }>; };
66 "/shares/grants": { $post: Endpoint<ShareState, { json: Omit<FolderGrant, "id"> }>; };
67 "/shares/grants/:id": {
68 $patch: Endpoint<ShareState, { param: { id: string }; json: Omit<FolderGrant, "id"> }>;
69 $delete: Endpoint<ShareState, { param: { id: string } }>;
70 };
71 "/shares/links": { $post: Endpoint<ShareState, { json: LinkInput & { createFolder: boolean } }>; };
72 "/shares/links/:key": {
73 $patch: Endpoint<ShareState, { param: { key: string }; json: LinkInput & { mode: "retarget" | "move" } }>;
74 $delete: Endpoint<ShareState, { param: { key: string } }>;
75 };
57 "/ai": { $get: Endpoint<{ endpoint: string; model: string; context: number }>; };76 "/ai": { $get: Endpoint<{ endpoint: string; model: string; context: number }>; };
58 "/ai/key": { $post: Endpoint<{ key: string }>; };77 "/ai/key": { $post: Endpoint<{ key: string }>; };
59 "/mcp": { $get: Endpoint<Connections>; };78 "/mcp": { $get: Endpoint<Connections>; };
dashboard/web/components/Dialog.tsx+2
...@@ -20,6 +20,7 @@ interface DialogOptions {...@@ -20,6 +20,7 @@ interface DialogOptions {
20 canConfirm?: () => boolean;20 canConfirm?: () => boolean;
21 /** Takes focus once the dialog closes, instead of the element that opened it. */21 /** Takes focus once the dialog closes, instead of the element that opened it. */
22 returnFocus?: HTMLElement;22 returnFocus?: HTMLElement;
23 onClose?: () => void;
23 /**24 /**
24 * Gets the body's named fields. The dialog stays open with a spinner until this settles; a rejection shakes it25 * Gets the body's named fields. The dialog stays open with a spinner until this settles; a rejection shakes it
25 * and shows the reason.26 * and shows the reason.
...@@ -96,6 +97,7 @@ function open(options: DialogOptions, field?: TextDialogOptions) {...@@ -96,6 +97,7 @@ function open(options: DialogOptions, field?: TextDialogOptions) {
96 dispose();97 dispose();
97 host.remove();98 host.remove();
98 options.returnFocus?.focus();99 options.returnFocus?.focus();
100 options.onClose?.();
99 }}101 }}
100 onPointerDown={(event) => (pressedOutside = event.target === dialog)}102 onPointerDown={(event) => (pressedOutside = event.target === dialog)}
101 onClick={(event) => pressedOutside && event.target === dialog && close()}>103 onClick={(event) => pressedOutside && event.target === dialog && close()}>
dashboard/web/components/Sidebar.tsx+2
...@@ -14,6 +14,7 @@ import SquarePlay from "lucide-solid/icons/square-play";...@@ -14,6 +14,7 @@ import SquarePlay from "lucide-solid/icons/square-play";
14import UserRound from "lucide-solid/icons/user-round";14import UserRound from "lucide-solid/icons/user-round";
15import Plug from "lucide-solid/icons/plug";15import Plug from "lucide-solid/icons/plug";
16import Users from "lucide-solid/icons/users";16import Users from "lucide-solid/icons/users";
17import FolderHeart from "lucide-solid/icons/folder-heart";
17import { createSignal, For, type JSX, onCleanup, Show } from "solid-js";18import { createSignal, For, type JSX, onCleanup, Show } from "solid-js";
18import { type Health, type Me, type Section, VIEW_AS } from "../types/model.ts";19import { type Health, type Me, type Section, VIEW_AS } from "../types/model.ts";
19import { queries } from "../api.ts";20import { queries } from "../api.ts";
...@@ -59,6 +60,7 @@ const AFTER: Page[] = [...@@ -59,6 +60,7 @@ const AFTER: Page[] = [
59 },60 },
60 { href: "/paper-clover", label: "paper clover", icon: PaperCloverMark, section: "admin" },61 { href: "/paper-clover", label: "paper clover", icon: PaperCloverMark, section: "admin" },
61 { href: "/users", label: "users", icon: Users, section: "admin" },62 { href: "/users", label: "users", icon: Users, section: "admin" },
63 { href: "/shares", label: "shares", icon: FolderHeart, section: "admin" },
62 { href: "/deploys", label: "deploys", icon: Rocket, section: "admin" },64 { href: "/deploys", label: "deploys", icon: Rocket, section: "admin" },
63 { href: "/vms", label: "vms", icon: Monitor, section: "vms" },65 { href: "/vms", label: "vms", icon: Monitor, section: "vms" },
64];66];
dashboard/web/main.tsx+3
...@@ -16,6 +16,7 @@ import { Seedbox } from "./pages/Seedbox.tsx";...@@ -16,6 +16,7 @@ import { Seedbox } from "./pages/Seedbox.tsx";
16import { YouTube } from "./pages/YouTube.tsx";16import { YouTube } from "./pages/YouTube.tsx";
17import { PaperClover } from "./pages/PaperClover.tsx";17import { PaperClover } from "./pages/PaperClover.tsx";
18import { directory, Users } from "./pages/Users.tsx";18import { directory, Users } from "./pages/Users.tsx";
19import { shares, Shares } from "./pages/Shares.tsx";
19import { Deploys } from "./pages/Deploys.tsx";20import { Deploys } from "./pages/Deploys.tsx";
20import { Deploy } from "./pages/Deploy.tsx";21import { Deploy } from "./pages/Deploy.tsx";
21import { VMs } from "./pages/VMs.tsx";22import { VMs } from "./pages/VMs.tsx";
...@@ -83,6 +84,8 @@ render(() => (...@@ -83,6 +84,8 @@ render(() => (
83 <Route path="/youtube" component={YouTube} />84 <Route path="/youtube" component={YouTube} />
84 <Route path="/paper-clover" component={PaperClover} preload={preload(queries.paperClover, queries.paperCloverActivity)} />85 <Route path="/paper-clover" component={PaperClover} preload={preload(queries.paperClover, queries.paperCloverActivity)} />
85 <Route path="/users/:name?" component={Users} preload={preload(directory)} />86 <Route path="/users/:name?" component={Users} preload={preload(directory)} />
87 <Route path="/shares/accounts/:id" component={Shares} preload={preload(shares)} />
88 <Route path="/shares/:key?" component={Shares} preload={preload(shares)} />
86 <Route path="/deploys" component={Deploys} preload={preload(queries.deploys, queries.services)} />89 <Route path="/deploys" component={Deploys} preload={preload(queries.deploys, queries.services)} />
87 <Route path="/deploys/:id/:tab?" component={Deploy} preload={preload(queries.deploys, queries.services)} />90 <Route path="/deploys/:id/:tab?" component={Deploy} preload={preload(queries.deploys, queries.services)} />
88 <Route path="/vms" component={VMs} preload={preload(queries.vms)} />91 <Route path="/vms" component={VMs} preload={preload(queries.vms)} />
dashboard/web/pages/Shares.css created+45
...@@ -0,0 +1,45 @@
1.shares-toolbar { display: flex; align-items: center; flex-wrap: wrap; gap: 12px; margin-bottom: 12px; }
2.shares-toolbar .search-box { margin-left: auto; width: 240px; }
3.shares-toolbar .count { opacity: 0.7; }
4.shares-table { overflow: auto; }
5.shares-table th { white-space: nowrap; }
6.share-name { color: var(--accent); font-weight: 600; }
7.share-name:hover { text-decoration: underline; }
8.share-path { font: 12px/1.5 var(--mono); overflow-wrap: anywhere; }
9.shares-table .share-path { min-width: 170px; }
10.share-detail { display: block; color: var(--muted); font-size: 12px; margin-top: 2px; }
11.share-expiry { white-space: nowrap; }
12.share-account-folder { display: flex; flex-wrap: wrap; gap: 4px 12px; margin-block: 3px; }
13.share-detail-page .page-head { flex-wrap: wrap; }
14.share-detail-page .page-head h1 { overflow-wrap: anywhere; }
15.share-form { max-width: 800px; }
16.share-form > fieldset { display: grid; gap: 18px; margin: 0; padding: 0; border: 0; min-width: 0; }
17.share-form .field { display: grid; gap: 5px; min-width: 0; }
18.share-folder-field { display: flex; gap: 6px; }
19.share-current-folder { display: flex; align-items: center; flex-wrap: wrap; gap: 8px 12px; }
20.share-current-folder .share-path { flex: 1; }
21.dialog .share-form { display: grid; gap: 18px; }
22.share-folder-field input { flex: 1; min-width: 0; font-family: var(--mono); font-size: 12px; }
23.share-rights, .share-path-mode { border: 0; margin: 0; padding: 0; min-width: 0; }
24.share-rights legend, .share-path-mode legend { margin-bottom: 7px; padding: 0; }
25.share-rights > div { display: flex; flex-wrap: wrap; gap: 8px 18px; }
26.share-path-mode { display: grid; gap: 8px; }
27.share-path-mode label { display: grid; grid-template-columns: 18px 1fr; column-gap: 5px; align-items: center; }
28.share-path-mode input { grid-column: 1; grid-row: 1; margin: 0; accent-color: var(--accent); }
29.share-path-mode .hint { grid-column: 2; margin: 0; }
30.share-options { display: grid; grid-template-columns: 1fr 1fr; gap: 18px; }
31.share-options button { justify-self: start; }
32.share-selection { margin: 0; padding-left: 20px; overflow-wrap: anywhere; }
33.share-columns { display: grid; grid-template-columns: minmax(250px, 1fr) minmax(320px, 1.5fr); gap: 8px; align-items: start; }
34.share-columns .card-title { margin: 0 0 12px; align-items: center; color: var(--text); }
35.share-grant { display: flex; align-items: center; gap: 6px; padding-block: 7px; }
36.share-grant > div { display: grid; flex: 1; min-width: 0; gap: 2px; }
37.share-grant .muted { font-size: 12px; }
38.share-grants p { font-size: 13px; }
39.share-secret dd { overflow-wrap: anywhere; }
40.share-picker .field { display: grid; gap: 5px; }
41.share-picker-list { max-height: 300px; min-height: 100px; overflow: auto; margin-top: 10px; }
42.share-picker-list button { display: flex; align-items: center; gap: 8px; text-align: left; border: 0; background: none; width: 100%; padding: 7px 8px; cursor: pointer; border-radius: 4px; overflow-wrap: anywhere; }
43.share-picker-list button:hover { background: var(--hover); }
44@media (max-width: 1000px) { .share-columns { grid-template-columns: 1fr; } }
45@media (max-width: 600px) { .share-options { grid-template-columns: 1fr; } .shares-toolbar .search-box { margin-left: 0; width: 100%; } }
dashboard/web/pages/Shares.tsx created+318
...@@ -0,0 +1,318 @@
1import { A, useNavigate, useParams, useSearchParams } from "@solidjs/router";
2import ArrowLeft from "lucide-solid/icons/arrow-left";
3import Folder from "lucide-solid/icons/folder";
4import Plus from "lucide-solid/icons/plus";
5import Search from "lucide-solid/icons/search";
6import { createMemo, createResource, createSignal, For, onMount, Show } from "solid-js";
7import { parseResponse } from "hono/client";
8import { api, query, reason } from "../api.ts";
9import { Checkbox } from "../components/Checkbox.tsx";
10import { Copy } from "../components/Copy.tsx";
11import { showConfirmDialog, showTextDialog } from "../components/Dialog.tsx";
12import { ListPage } from "../components/ListPage.tsx";
13import { lastGood, Loaded, SkeletonRows } from "../components/Loaded.tsx";
14import { TabBar } from "../components/TabBar.tsx";
15import type { AccountSecret, FileAccount, FolderGrant, ShareState, SharedLink } from "../types/shares.ts";
16import { datetime, plural } from "../format.ts";
17import "./Shares.css";
18
19export const shares = query("shares", () => parseResponse(api.shares.$get()));
20
21const RIGHTS = [["r", "read"], ["w", "upload"], ["m", "move / rename"], ["d", "delete"], [".", "hidden files"], ["a", "Copyparty admin"]] as const;
22const permissions = (value: string) => RIGHTS.filter(([key]) => value.includes(key)).map(([, label]) => label).join(", ");
23const macPath = (path: string, root: string) => path === root || path.startsWith(root + "/") ? "/Volumes/clover" + path.slice(root.length) : path;
24const linkUrl = (state: ShareState, key: string) => state.filesUrl ? `${state.filesUrl.replace(/\/$/, "")}/shr/${encodeURIComponent(key)}` : null;
25
26function Rights(props: { value: string; onChange: (value: string) => void; link?: boolean }) {
27 return <fieldset class="share-rights"><legend>permissions</legend><div>
28 <For each={props.link ? RIGHTS.slice(0, 2) : RIGHTS}>{([key, label]) =>
29 <Checkbox checked={props.value.includes(key)} onChange={(checked) => props.onChange(checked ? props.value + key : props.value.replace(key, ""))}>{label}</Checkbox>
30 }</For>
31 </div></fieldset>;
32}
33
34function FolderField(props: { value: string; root: string; create?: boolean; onChange: (value: string, existing?: boolean) => void }) {
35 const browse = () => {
36 const [path, setPath] = createSignal(props.create && !props.value.endsWith("/") ? props.value.replace(/\/[^/]+$/, "") || props.root : props.value || props.root);
37 showConfirmDialog({
38 title: "Choose folder", confirmLabel: "use folder",
39 body: () => {
40 const [folders, { refetch }] = createResource(path, (path) => parseResponse(api.shares.folders.$get({ query: { path } })));
41 return <div class="share-picker">
42 <label class="field">folder<input class="search" value={path()} onChange={(e) => setPath(e.currentTarget.value)} /></label>
43 <button type="button" class="button small" disabled={path() === props.root || path() === "/Volumes/clover"}
44 onClick={() => setPath(path().replace(/\/[^/]+\/?$/, "") || props.root)}>parent folder</button>
45 <Loaded data={folders} what="folders" retry={refetch} skeleton={<SkeletonRows count={4} />}>
46 {(data) => <div class="share-picker-list"><For each={data().directories} fallback={<p class="muted">No folders here</p>}>
47 {(folder) => <button type="button" onClick={() => setPath(macPath(folder.path, props.root))}><Folder size={16} aria-hidden="true" />{folder.name}</button>}
48 </For></div>}
49 </Loaded>
50 </div>;
51 },
52 onConfirm: () => props.onChange(macPath(path(), props.root), true),
53 });
54 };
55 return <div class="field">folder<div class="share-folder-field">
56 <input aria-label="Folder" class="search" value={props.value} required autocomplete="off" spellcheck={false}
57 onInput={(event) => props.onChange(event.currentTarget.value)} />
58 <button type="button" class="button" onClick={browse}>browse</button>
59 </div></div>;
60}
61
62function showSecret(result: AccountSecret, returnFocus?: HTMLElement) {
63 showConfirmDialog({
64 title: "Save account password", confirmLabel: "done",
65 returnFocus,
66 description: "This password is shown once. Save it before closing.",
67 body: () => <dl class="kv share-secret"><dt>username</dt><dd><Copy value={result.account.username} /></dd><dt>password</dt><dd><Copy value={result.password} /></dd></dl>,
68 onConfirm: () => {},
69 });
70}
71
72function editGrant(props: { subject: string; state: ShareState; grant?: FolderGrant; saved: (state: ShareState) => void }) {
73 const [path, setPath] = createSignal(macPath(props.grant?.path ?? props.state.defaultRoot, props.state.root));
74 const [rights, setRights] = createSignal(props.grant?.permissions ?? "r");
75 showConfirmDialog({
76 title: props.grant ? "Edit folder access" : "Add folder access", confirmLabel: "save access",
77 returnFocus: document.querySelector<HTMLElement>(".folder-access .card-title button") ?? undefined,
78 canConfirm: () => !!path().trim() && !!rights(),
79 body: () => <div class="share-form">
80 <FolderField value={path()} root={props.state.root} onChange={setPath} />
81 <Rights value={rights()} onChange={setRights} />
82 <p class="hint">Folder permissions add to existing access.</p>
83 </div>,
84 onConfirm: async () => {
85 const json = { subject: props.subject, path: path(), permissions: rights() };
86 const state = await parseResponse(props.grant
87 ? api.shares.grants[":id"].$patch({ param: { id: props.grant.id }, json })
88 : api.shares.grants.$post({ json }));
89 props.saved(state);
90 },
91 });
92}
93
94export function FolderAccess(props: { subject: string; enabled?: boolean }) {
95 const [state, { mutate, refetch }] = shares.use();
96 const ready = lastGood(state);
97 return <section class="card folder-access">
98 <h2 class="card-title">folder access<span class="spacer" />
99 <button class="button small" disabled={!ready() || props.enabled === false} onClick={() => editGrant({ subject: props.subject, state: ready()!, saved: mutate })}>add folder</button>
100 </h2>
101 <Loaded data={state} what="folder access" retry={refetch} skeleton={<SkeletonRows count={2} />}>
102 {(data) => <GrantList subject={props.subject} state={data()} saved={mutate} enabled={props.enabled} />}
103 </Loaded>
104 </section>;
105}
106
107function GrantList(props: { subject: string; state: ShareState; saved: (state: ShareState) => void; enabled?: boolean }) {
108 const grants = () => props.state.grants.filter((grant) => grant.subject === props.subject);
109 const remove = (grant: FolderGrant) => showConfirmDialog({
110 title: "Remove folder access?", description: macPath(grant.path, props.state.root), confirmLabel: "remove access", destructive: true,
111 returnFocus: document.querySelector<HTMLElement>(".folder-access .card-title button") ?? undefined,
112 onConfirm: async () => props.saved(await parseResponse(api.shares.grants[":id"].$delete({ param: { id: grant.id } }))),
113 });
114 return <div class="share-grants">
115 <For each={grants()} fallback={<p class="muted">No folder permissions assigned here</p>}>{(grant) => <div class="share-grant">
116 <div><span class="share-path">{macPath(grant.path, props.state.root)}</span><span class="muted">{permissions(grant.permissions)}</span></div>
117 <button class="button small" aria-label={`Edit access to ${macPath(grant.path, props.state.root)}`} disabled={props.enabled === false} onClick={() => editGrant({ subject: props.subject, state: props.state, grant, saved: props.saved })}>edit</button>
118 <button class="button small danger" aria-label={`Remove access to ${macPath(grant.path, props.state.root)}`} onClick={() => remove(grant)}>remove</button>
119 </div>}</For>
120 <Show when={grants().length}><p class="hint">Folder permissions add to existing access.</p></Show>
121 </div>;
122}
123
124export function Shares() {
125 const params = useParams<{ key?: string; id?: string }>();
126 const [search, setSearch] = useSearchParams<{ tab?: string; q?: string }>();
127 const [state, { refetch, mutate }] = shares.use();
128 const navigate = useNavigate();
129 const accounts = () => search.tab === "accounts";
130 const createAccount = () => {
131 let credentials: AccountSecret | undefined;
132 showTextDialog({
133 title: "New API account", label: "name", placeholder: "T3Code", confirmLabel: "create account",
134 onConfirm: async (name) => {
135 const result = await parseResponse(api.shares.accounts.$post({ json: { name } }));
136 mutate(result);
137 navigate(`/shares/accounts/${result.account.id}`);
138 credentials = result;
139 },
140 onClose: () => { if (credentials) showSecret(credentials, document.querySelector<HTMLElement>(".share-detail-page h1") ?? undefined); },
141 });
142 };
143 return <Loaded data={state} what="shares" retry={refetch} skeleton={<div class="page"><SkeletonRows count={6} /></div>}>
144 {(data) => <Show when={params.id} keyed fallback={<Show when={params.key} keyed fallback={
145 <ListPage id="shares" class="shares-page" flush head={<div class="page-head"><h1>shares</h1><span class="spacer" />
146 <Show when={accounts()} fallback={<A class="button primary" href="/shares/new"><Plus size={14} aria-hidden="true" />new share</A>}>
147 <button class="button primary" onClick={createAccount}><Plus size={14} aria-hidden="true" />new account</button>
148 </Show>
149 </div>}>
150 <div class="shares-toolbar">
151 <TabBar label="Share manager"><button aria-pressed={!accounts()} onClick={() => setSearch({ tab: undefined, q: undefined })}>shared links<span class="count">{data().links.length}</span></button>
152 <button aria-pressed={accounts()} onClick={() => setSearch({ tab: "accounts", q: undefined })}>API accounts<span class="count">{data().accounts.length}</span></button>
153 </TabBar>
154 <label class="search-box"><Search size={14} aria-hidden="true" /><input type="search" aria-label="Search shares" placeholder="name or folder" value={search.q ?? ""} onInput={(e) => setSearch({ q: e.currentTarget.value || undefined }, { replace: true })} /></label>
155 </div>
156 <div class="fill shares-table"><Show when={accounts()} fallback={<LinkList state={data()} filter={search.q ?? ""} />}>
157 <AccountList state={data()} filter={search.q ?? ""} create={createAccount} />
158 </Show></div>
159 </ListPage>
160 }>{(key) => <LinkPage key={key} state={data()} saved={mutate} />}</Show>}>
161 {(id) => <Show when={data().accounts.find((account) => account.id === id)} fallback={<div class="page empty">This account no longer exists. <A href="/shares?tab=accounts">back to accounts</A></div>}>
162 {(account) => <AccountPage account={account()} state={data()} saved={mutate} />}
163 </Show>}
164 </Show>}
165 </Loaded>;
166}
167
168function LinkList(props: { state: ShareState; filter: string }) {
169 const links = () => props.state.links.filter((link) => `${link.name} ${link.path} ${link.key}`.toLowerCase().includes(props.filter.toLowerCase()));
170 return <Show when={links().length} fallback={<div class="empty">{props.filter ? "No shares match" : "No shared links yet"} <A class="button small" href="/shares/new">new share</A></div>}>
171 <table class="data"><thead><tr><th>share</th><th>folder</th><th>access</th><th>expires</th><th>public link</th></tr></thead><tbody>
172 <For each={links()}>{(link) => <tr>
173 <td><A class="share-name" href={`/shares/${encodeURIComponent(link.key)}`}>{link.name || link.key}</A>
174 <Show when={link.files.length}><span class="share-detail">{plural(link.files.length, "selected file")}</span></Show>
175 </td>
176 <td class="share-path">{macPath(link.path, props.state.root)}</td>
177 <td>{permissions(link.permissions)}<Show when={link.hasPassword}><span class="share-detail">password protected</span></Show></td>
178 <td class="share-expiry">{link.expires ? datetime(link.expires) : "never"}</td>
179 <td><Show when={linkUrl(props.state, link.key)}>{(url) => <Copy value={url()} label="link">copy link</Copy>}</Show></td>
180 </tr>}</For>
181 </tbody></table>
182 </Show>;
183}
184
185function AccountList(props: { state: ShareState; filter: string; create: () => void }) {
186 const accounts = () => props.state.accounts.filter((account) => `${account.name} ${account.username}`.toLowerCase().includes(props.filter.toLowerCase()));
187 return <Show when={accounts().length} fallback={<div class="empty">{props.filter ? "No accounts match" : "No API accounts yet"} <button class="button small" onClick={props.create}>new account</button></div>}>
188 <table class="data"><thead><tr><th>account</th><th>username</th><th>folders</th></tr></thead><tbody>
189 <For each={accounts()}>{(account) => <tr><td><A class="share-name" href={`/shares/accounts/${account.id}`}>{account.name}</A></td><td><Copy value={account.username} /></td>
190 <td><For each={props.state.grants.filter((grant) => grant.subject === account.subject)} fallback={<span class="muted">no folders assigned</span>}>{(grant) => <span class="share-account-folder"><span class="share-path">{macPath(grant.path, props.state.root)}</span><span class="muted">{permissions(grant.permissions)}</span></span>}</For></td>
191 </tr>}</For>
192 </tbody></table>
193 </Show>;
194}
195
196function AccountPage(props: { account: FileAccount; state: ShareState; saved: (state: ShareState) => void }) {
197 const navigate = useNavigate();
198 let heading!: HTMLHeadingElement;
199 onMount(() => heading.focus());
200 const rename = () => showTextDialog({ title: "Rename account", label: "name", initialValue: props.account.name, confirmLabel: "save name",
201 onConfirm: async (name) => props.saved(await parseResponse(api.shares.accounts[":id"].$patch({ param: { id: props.account.id }, json: { name } }))),
202 });
203 const rotate = () => {
204 let credentials: AccountSecret | undefined;
205 showConfirmDialog({ title: "Replace account password?", description: "The old password and open sessions stop working.", confirmLabel: "replace password", destructive: true,
206 onConfirm: async () => {
207 const result = await parseResponse(api.shares.accounts[":id"].rotate.$post({ param: { id: props.account.id } }));
208 props.saved(result); credentials = result;
209 },
210 onClose: () => { if (credentials) showSecret(credentials); },
211 });
212 };
213 const remove = () => showConfirmDialog({ title: `Delete ${props.account.name}?`, description: "This account loses access to its folders. The files stay in place.", confirmLabel: "delete account", destructive: true,
214 onConfirm: async () => {
215 props.saved(await parseResponse(api.shares.accounts[":id"].$delete({ param: { id: props.account.id } })));
216 navigate("/shares?tab=accounts");
217 },
218 });
219 return <div class="page share-detail-page"><div class="page-head"><A class="button icon-only" href="/shares?tab=accounts" aria-label="Back to API accounts"><ArrowLeft size={16} /></A>
220 <h1 ref={heading} tabindex="-1">{props.account.name}</h1><span class="sub">API account</span><span class="spacer" /><button class="button danger" onClick={remove}>delete account</button></div>
221 <div class="share-columns"><section class="card"><h2 class="card-title">connection<span class="spacer" /><button class="button small" onClick={rename}>rename</button></h2>
222 <dl class="kv"><dt>username</dt><dd><Copy value={props.account.username} /></dd><dt>server</dt><dd><Show when={props.state.filesUrl} fallback="unavailable">{(url) => <Copy value={url()} />}</Show></dd></dl>
223 <p class="hint">Use this account with HTTP, WebDAV, or an AI client.</p><button class="button small" onClick={rotate}>replace password</button>
224 </section><section class="card folder-access"><h2 class="card-title">folder access<span class="spacer" /><button class="button small" onClick={() => editGrant({ subject: props.account.subject, state: props.state, saved: props.saved })}>add folder</button></h2>
225 <GrantList subject={props.account.subject} state={props.state} saved={props.saved} />
226 </section></div>
227 </div>;
228}
229
230function LinkPage(props: { key: string; state: ShareState; saved: (state: ShareState) => void }) {
231 const navigate = useNavigate();
232 const original = () => props.state.links.find((link) => link.key === props.key);
233 const fresh = props.key === "new";
234 return <Show when={fresh || original()} fallback={<div class="page empty">This share no longer exists. <A href="/shares">back to shares</A></div>}>
235 <LinkForm original={original()} fresh={fresh} state={props.state} saved={(state, key) => { props.saved(state); if (fresh) navigate(`/shares/${encodeURIComponent(key)}`); }} />
236 </Show>;
237}
238
239function LinkForm(props: { original?: SharedLink; fresh: boolean; state: ShareState; saved: (state: ShareState, key: string) => void }) {
240 const navigate = useNavigate();
241 let heading!: HTMLHeadingElement;
242 onMount(() => heading.focus());
243 const [name, setName] = createSignal(props.original?.name ?? "");
244 const [customPath, setCustomPath] = createSignal("");
245 const path = () => customPath() || `${macPath(props.state.defaultRoot, props.state.root)}/${name().trim()}`;
246 const [rights, setRights] = createSignal(props.original?.permissions ?? "r");
247 const [createFolder, setCreateFolder] = createSignal(true);
248 const [password, setPassword] = createSignal<string | null>(null);
249 const dateValue = (seconds: number) => {
250 const date = new Date(seconds * 1000);
251 return new Date(date.getTime() - date.getTimezoneOffset() * 60_000).toISOString().slice(0, 16);
252 };
253 const [expires, setExpires] = createSignal(props.original?.expires ? dateValue(props.original.expires) : "");
254 const [pending, setPending] = createSignal(false);
255 const [error, setError] = createSignal("");
256 const dirty = createMemo(() => props.fresh || name() !== props.original?.name || rights() !== props.original?.permissions || password() !== null || expires() !== (props.original?.expires ? dateValue(props.original.expires) : ""));
257 const save = async () => {
258 if (pending() || !dirty()) return;
259 setPending(true); setError("");
260 try {
261 const json = { name: name().trim(), path: props.original?.path ?? path(), permissions: rights(), expires: expires() ? new Date(expires()).getTime() / 1000 : 0, password: password() };
262 const result = await parseResponse(props.original
263 ? api.shares.links[":key"].$patch({ param: { key: props.original.key }, json: { ...json, mode: "retarget" } })
264 : api.shares.links.$post({ json: { ...json, createFolder: createFolder() } }));
265 const key = props.original?.key ?? result.links.find((link) => !props.state.links.some((old) => old.key === link.key))?.key;
266 if (key) props.saved(result, key);
267 setPassword(null);
268 } catch (failure) { setError(reason(failure)); }
269 finally { setPending(false); }
270 };
271 const changeFolder = () => {
272 const link = props.original!;
273 const [destination, setDestination] = createSignal(macPath(link.path, props.state.root));
274 const [mode, setMode] = createSignal<"retarget" | "move">("retarget");
275 showConfirmDialog({
276 title: "Change folder",
277 get confirmLabel() { return mode() === "move" ? "move folder" : "use folder"; },
278 canConfirm: () => !!destination().trim() && destination() !== macPath(link.path, props.state.root),
279 body: () => <div class="share-form">
280 <FolderField value={destination()} root={props.state.root} onChange={setDestination} />
281 <fieldset class="share-path-mode"><legend>change folder</legend>
282 <label><input type="radio" name="path-mode" checked={mode() === "retarget"} onChange={() => setMode("retarget")} />use another folder<span class="hint">Share the files already at the new path</span></label>
283 <label><input type="radio" name="path-mode" checked={mode() === "move"} onChange={() => setMode("move")} />move files<span class="hint">Move the current folder to the new path</span></label>
284 </fieldset>
285 <p class="hint">Public share links stay the same. Other shares inside a moved folder follow it.</p>
286 </div>,
287 onConfirm: async () => props.saved(await parseResponse(api.shares.links[":key"].$patch({ param: { key: link.key }, json: {
288 name: link.name, path: destination(), permissions: link.permissions, expires: link.expires, password: null, mode: mode(),
289 } })), link.key),
290 });
291 };
292 const remove = () => showConfirmDialog({ title: `Delete ${props.original?.name || "share"}?`, description: "The public link stops working. The files stay in place.", confirmLabel: "delete share", destructive: true,
293 onConfirm: async () => { const state = await parseResponse(api.shares.links[":key"].$delete({ param: { key: props.original!.key } })); props.saved(state, props.original!.key); navigate("/shares"); },
294 });
295 return <div class="page share-detail-page"><div class="page-head"><A class="button icon-only" href="/shares" aria-label="Back to shares"><ArrowLeft size={16} /></A>
296 <h1 ref={heading} tabindex="-1">{props.fresh ? "new share" : props.original?.name || props.original?.key}</h1><span class="spacer" />
297 <Show when={!props.fresh}><button class="button danger" disabled={pending()} onClick={remove}>delete share</button></Show>
298 <button form="share-form" type="submit" class="button primary" disabled={!dirty() || pending()} aria-busy={pending()}>{props.fresh ? "create share" : "save changes"}</button></div>
299 <form id="share-form" class="share-form card" onSubmit={(event) => { event.preventDefault(); void save(); }}>
300 <fieldset disabled={pending()}>
301 <label class="field">name<input class="search" required value={name()} placeholder="Name - Description" autocomplete="off" onInput={(event) => setName(event.currentTarget.value)} /></label>
302 <Show when={props.fresh} fallback={<div class="field">folder<div class="share-current-folder"><span class="share-path">{macPath(props.original!.path, props.state.root)}</span><button type="button" class="button" onClick={changeFolder}>change folder</button></div></div>}>
303 <FolderField value={path()} root={props.state.root} create={createFolder()} onChange={(value, existing) => { setCustomPath(value); if (existing) setCreateFolder(false); }} />
304 <Checkbox checked={createFolder()} onChange={setCreateFolder}>create this folder</Checkbox>
305 </Show>
306 <Show when={props.original?.files.length}><div class="field">selected files<ul class="share-selection"><For each={props.original?.files}>{(file) => <li>{file}</li>}</For></ul><span class="hint">This share keeps its file selection when its folder changes</span></div></Show>
307 <Rights value={rights()} onChange={setRights} link />
308 <div class="share-options"><label class="field">expires<input type="datetime-local" class="search" value={expires()} onInput={(event) => setExpires(event.currentTarget.value)} /><span class="hint">Leave empty for no expiry</span></label>
309 <label class="field">password<input class="search" type="password" autocomplete="new-password" value={password() ?? ""} placeholder={props.original?.hasPassword ? "Keep current password" : "No password"} onInput={(event) => setPassword(event.currentTarget.value)} />
310 <Show when={props.original?.hasPassword}><button type="button" class="button small" onClick={() => setPassword("")}>remove password</button></Show>
311 <Show when={props.original?.hasPassword && password() === ""}><span class="hint">Password will be removed when saved</span></Show>
312 </label></div>
313 <Show when={props.original && linkUrl(props.state, props.original.key)}>{(url) => <div class="field">public link<Copy value={url()} /></div>}</Show>
314 </fieldset>
315 <Show when={error()}><p class="error" role="alert">{error()}</p></Show>
316 </form>
317 </div>;
318}
dashboard/web/pages/Users.tsx+2
...@@ -23,6 +23,7 @@ import { toast } from "../components/Toast.tsx";...@@ -23,6 +23,7 @@ import { toast } from "../components/Toast.tsx";
23import { ago, count, date, datetime, plural } from "../format.ts";23import { ago, count, date, datetime, plural } from "../format.ts";
24import "./Users.css";24import "./Users.css";
25import { CommitIdentity } from "./Account.tsx";25import { CommitIdentity } from "./Account.tsx";
26import { FolderAccess } from "./Shares.tsx";
2627
27const STEPS: [string, string][] = [["SETUP", "finish setup"], ["UPDATE_PASSWORD", "new password"], ["UPDATE_PROFILE", "check profile"]];28const STEPS: [string, string][] = [["SETUP", "finish setup"], ["UPDATE_PASSWORD", "new password"], ["UPDATE_PROFILE", "check profile"]];
2829
...@@ -592,6 +593,7 @@ function Profile(props: { user: User; data: Data; refetch: () => unknown }) {...@@ -592,6 +593,7 @@ function Profile(props: { user: User; data: Data; refetch: () => unknown }) {
592 </Show>593 </Show>
593 </Show>594 </Show>
594 </section>595 </section>
596 <Show when={props.user.kind !== "guest"}><FolderAccess subject={`user:${props.user.id}`} enabled={props.user.enabled} /></Show>
595 <section class="card">597 <section class="card">
596 <h2 class="card-title">598 <h2 class="card-title">
597 sign-in599 sign-in
dashboard/web/types/shares.ts created+44
...@@ -0,0 +1,44 @@
1export interface FileAccount {
2 id: string;
3 name: string;
4 username: string;
5 subject: string;
6}
7
8export interface FolderGrant {
9 id: string;
10 subject: string;
11 path: string;
12 permissions: string;
13}
14
15export interface SharedLink {
16 key: string;
17 name: string;
18 path: string;
19 permissions: string;
20 owner: string;
21 created: number;
22 expires: number;
23 hasPassword: boolean;
24 files: string[];
25}
26
27export interface ShareState {
28 root: string;
29 defaultRoot: string;
30 filesUrl: string | null;
31 accounts: FileAccount[];
32 grants: FolderGrant[];
33 links: SharedLink[];
34}
35
36export interface LinkInput {
37 name: string;
38 path: string;
39 permissions: string;
40 expires: number;
41 password: string | null;
42}
43
44export type AccountSecret = ShareState & { account: FileAccount; password: string };
nixos/configuration.nix+3-3
...@@ -6,7 +6,7 @@ let...@@ -6,7 +6,7 @@ let
6 proxyToken = "/var/lib/studio/dashboard-proxy.token";6 proxyToken = "/var/lib/studio/dashboard-proxy.token";
7 hostTools = lib.fileset.toSource {7 hostTools = lib.fileset.toSource {
8 root = ../.;8 root = ../.;
9 fileset = lib.fileset.unions [ ../tools/dashboard-host.py ../tools/shale-labels.py ../config/shale.json ../dashboard/src/host-protocol.json ../tools/vms.py ../tools/vm-ssh.py ../tools/dashboard-run.py ../tools/release.py ../guest/protocol.json ];9 fileset = lib.fileset.unions [ ../tools/dashboard-host.py ../service/copyparty/manage.py ../service/copyparty/copyparty.conf ../tools/shale-labels.py ../config/shale.json ../dashboard/src/host-protocol.json ../tools/vms.py ../tools/vm-ssh.py ../tools/dashboard-run.py ../tools/release.py ../guest/protocol.json ];
10 };10 };
11 nativePkl = pkgs.callPackage ./pkl.nix { };11 nativePkl = pkgs.callPackage ./pkl.nix { };
12 screenPython = pkgs.python3.withPackages (packages: [ packages.pygobject3 packages.gst-python ]);12 screenPython = pkgs.python3.withPackages (packages: [ packages.pygobject3 packages.gst-python ]);
...@@ -203,11 +203,11 @@ in...@@ -203,11 +203,11 @@ in
203 RuntimeDirectoryPreserve = "yes";203 RuntimeDirectoryPreserve = "yes";
204 UMask = "0077";204 UMask = "0077";
205 ProtectSystem = "strict";205 ProtectSystem = "strict";
206 ReadWritePaths = [ "/srv/vm" "-/srv/prod/shale/data" "/var/lib/studio/dashboard" ] ++ lib.optionals (config.networking.hostName != "clover-demo") [ "/srv/clover/Media/vm" ];206 ReadWritePaths = [ "/srv/vm" "-/srv/prod/shale/data" "-/srv/prod/copyparty/cfg" "/srv/staging" "/srv/clover" "/var/lib/studio/dashboard" ] ++ lib.optionals (config.networking.hostName != "clover-demo") [ "/srv/clover/Media/vm" ];
207 ProtectHome = true;207 ProtectHome = true;
208 PrivateTmp = true;208 PrivateTmp = true;
209 NoNewPrivileges = true;209 NoNewPrivileges = true;
210 CapabilityBoundingSet = [ "CAP_SYS_ADMIN" "CAP_DAC_READ_SEARCH" "CAP_DAC_OVERRIDE" ];210 CapabilityBoundingSet = [ "CAP_SYS_ADMIN" "CAP_DAC_READ_SEARCH" "CAP_DAC_OVERRIDE" "CAP_CHOWN" ];
211 MemoryMax = "256M";211 MemoryMax = "256M";
212 TasksMax = 64;212 TasksMax = 64;
213 DevicePolicy = "closed";213 DevicePolicy = "closed";
service/copyparty/copyparty.conf+1
...@@ -9,6 +9,7 @@...@@ -9,6 +9,7 @@
9 name: clover's nas9 name: clover's nas
10 stats, nos-dup10 stats, nos-dup
11 dav-auth11 dav-auth
12 xvol
12 cookie-cmax: 6553613 cookie-cmax: 65536
13 ah-alg: argon214 ah-alg: argon2
14 xbu: c,/hooks/reject-apple-cruft.py15 xbu: c,/hooks/reject-apple-cruft.py
service/copyparty/manage.py created+697
...@@ -0,0 +1,697 @@
1import copy
2import ctypes
3import errno
4import fcntl
5import importlib
6import json
7import os
8from pathlib import Path
9import re
10import secrets
11import sqlite3
12import subprocess
13import time
14import urllib.error
15import urllib.request
16import uuid
17
18runs = importlib.import_module('dashboard-run')
19Error = runs.Error
20PODMAN = ['podman', '--remote', '--url', 'unix:///run/podman/podman.sock']
21
22ACTIONS = {
23 'state': set(), 'folders.list': {'path'},
24 'account.create': {'name'}, 'account.update': {'id', 'name'},
25 'account.rotate': {'id'}, 'account.delete': {'id'},
26 'grant.save': {'id', 'subject', 'path', 'permissions'}, 'grant.delete': {'id'},
27 'link.create': {'name', 'path', 'permissions', 'expires', 'password', 'createFolder'},
28 'link.update': {'key', 'name', 'path', 'permissions', 'expires', 'password', 'mode'},
29 'link.delete': {'key'},
30}
31
32
33def identifier(value):
34 if not isinstance(value, str):
35 raise Error(400, 'Choose an account or folder permission from the list.')
36 try:
37 if str(uuid.UUID(value)) == value:
38 return value
39 except ValueError:
40 pass
41 raise Error(400, 'Choose an account or folder permission from the list.')
42
43
44def label(value):
45 if (not isinstance(value, str) or not value.strip() or len(value) > 200
46 or any(ord(c) < 32 or c in '/\\' for c in value)
47 or value.strip() in {'.', '..'}):
48 raise Error(400, 'Enter a name under 200 characters without slashes.')
49 return value.strip()
50
51
52def permissions(value, choices):
53 if not isinstance(value, str) or not value or set(value) - set(choices):
54 raise Error(400, 'Choose at least one supported permission.')
55 return ''.join(c for c in choices if c in value)
56
57
58def atomic(path, data, owner=None):
59 pending = path.with_name('.' + path.name + '.pending')
60 fd = os.open(pending, os.O_WRONLY | os.O_CREAT | os.O_TRUNC | os.O_NOFOLLOW, 0o600)
61 try:
62 os.fchmod(fd, 0o600)
63 with os.fdopen(fd, 'wb') as target:
64 target.write(data)
65 target.flush()
66 os.fsync(target.fileno())
67 if owner is not None:
68 os.chown(pending, *owner)
69 os.replace(pending, path)
70 finally:
71 pending.unlink(missing_ok=True)
72
73
74def volumes(base):
75 sections = re.split(r'(?m)^(?=\[/)', base)
76 result = []
77 for section in sections[1:]:
78 lines = [line.strip() for line in section.splitlines() if line.strip()]
79 match = re.fullmatch(r'\[(/[^\r\n]*)\]', lines[0])
80 if not match or len(lines) < 2:
81 raise Error(503, 'Copyparty configuration needs repair before editing shares.')
82 entry = {'url': match[1].rstrip('/') or '/', 'path': lines[1], 'accs': [], 'flags': []}
83 target = None
84 for line in lines[2:]:
85 if line == 'accs:':
86 target = entry['accs']
87 elif line == 'flags:':
88 target = entry['flags']
89 elif target is not None:
90 target.append(line)
91 else:
92 raise Error(503, 'Copyparty configuration needs repair before editing shares.')
93 result.append(entry)
94 return sections[0], result
95
96
97def render(base, state, hashes):
98 global_config, entries = volumes(base)
99 accounts = state['accounts']
100 if accounts:
101 global_config += '\n[accounts]\n' + ''.join(
102 ' api-' + account['id'] + ': ' + hashes['file_api_' + account['id'].replace('-', '')] + '\n'
103 for account in accounts
104 )
105 api_names = ['api-' + account['id'] for account in accounts]
106 base_entries = copy.deepcopy(entries)
107 paths = {grant['path'] for grant in state['grants']}
108 for entry in base_entries:
109 if entry['url'] == '/clover' or entry['url'].startswith('/clover/'):
110 host = str(Path(state['root']) / entry['url'][len('/clover'):].lstrip('/'))
111 if any(host == grant['path'] or host.startswith(grant['path'].rstrip('/') + '/') for grant in state['grants']):
112 paths.add(host)
113 for path in sorted(paths):
114 relative = Path(path).relative_to(state['root'])
115 url = '/clover' + ('/' + relative.as_posix() if relative.parts else '')
116 parent = max((entry for entry in base_entries if url == entry['url'] or url.startswith(entry['url'].rstrip('/') + '/')),
117 key=lambda entry: len(entry['url']))
118 exact = next((entry for entry in entries if entry['url'] == url), None)
119 entry = exact if exact is not None else {
120 'url': url, 'path': '/w/clover' + ('/' + relative.as_posix() if relative.parts else ''),
121 'accs': list(parent['accs']), 'flags': list(parent['flags']),
122 }
123 for grant in state['grants']:
124 if path == grant['path'] or path.startswith(grant['path'].rstrip('/') + '/'):
125 kind, subject = grant['subject'].split(':', 1)
126 principal = '@snow-user-' + subject if kind == 'user' else 'api-' + subject
127 entry['accs'].append(grant['permissions'] + ': ' + principal)
128 if 'xvol' not in entry['flags']:
129 entry['flags'].append('xvol')
130 if exact is None:
131 entries.append(entry)
132 rendered = global_config
133 for entry in entries:
134 allowed = {}
135 if entry['url'] == '/clover' or entry['url'].startswith('/clover/'):
136 host = str(Path(state['root']) / entry['url'][len('/clover'):].lstrip('/'))
137 for grant in state['grants']:
138 if grant['subject'].startswith('api:') and (host == grant['path'] or host.startswith(grant['path'].rstrip('/') + '/')):
139 for perm in grant['permissions']:
140 allowed.setdefault(perm, set()).add('api-' + grant['subject'][4:])
141 access = []
142 for line in entry['accs']:
143 perms, names = line.split(':', 1)
144 names = [name.strip() for name in names.split(',')]
145 if api_names and ('*' in names or '@acct' in names):
146 perms = perms.replace('A', 'rwmda.')
147 for perm in dict.fromkeys(perms):
148 denied = [name for name in api_names if name not in allowed.get(perm, set())]
149 access.append(perm + ': ' + ','.join(names + ['-' + name for name in denied]))
150 else:
151 access.append(line)
152 entry['accs'] = access
153 rendered += '\n[' + entry['url'] + ']\n ' + entry['path'] + '\n accs:\n'
154 rendered += ''.join(' ' + line + '\n' for line in entry['accs'])
155 if entry['flags']:
156 rendered += ' flags:\n' + ''.join(' ' + line + '\n' for line in entry['flags'])
157 return rendered
158
159
160def configuration(cfg, state, items, source=None):
161 source = source or Path(__file__).with_name('copyparty.conf')
162 base = source.read_text()
163 proof = items.get('idp_header', '')
164 if not re.fullmatch(r'[0-9a-f]{32}', proof):
165 raise Error(503, 'Copyparty sign-in configuration is unavailable. Redeploy the service.')
166 capacity = os.statvfs(cfg)
167 reserve = max(1, min(16, capacity.f_blocks * capacity.f_frsize // (4 * 10**9)))
168 base = base.replace('STUDIO_IDP_HEADER', 'X-Studio-Idp-' + proof).replace('STUDIO_MIN_FREE_GB', str(reserve))
169 return render(base, state, items).encode()
170
171
172def remaining(deadline, maximum=10):
173 timeout = min(maximum, deadline - time.monotonic())
174 if timeout <= 0:
175 raise Error(504, 'Copyparty took too long to respond. Check the service before retrying.')
176 return timeout
177
178
179def rename(source, destination):
180 # RENAME_NOREPLACE keeps a concurrently created destination from being overwritten.
181 libc = ctypes.CDLL(None, use_errno=True)
182 if hasattr(libc, 'renameat2'):
183 result = libc.renameat2(-100, os.fsencode(source), -100, os.fsencode(destination), 1)
184 elif hasattr(libc, 'renamex_np'):
185 result = libc.renamex_np(os.fsencode(source), os.fsencode(destination), 4)
186 else:
187 raise Error(503, 'This host cannot safely move folders. Retarget the share instead.')
188 if result:
189 code = ctypes.get_errno()
190 if code == errno.EEXIST:
191 raise Error(409, 'The destination already exists. Choose a new folder name.')
192 if code == errno.EXDEV:
193 raise Error(409, 'Choose a destination on the same filesystem to move this folder.')
194 raise OSError(code, os.strerror(code))
195
196
197class Manager:
198 def __init__(self, users):
199 root = Path(os.environ.get('STUDIO_ROOT', '/srv'))
200 self.root = Path(os.environ.get('STUDIO_CLOVER_ROOT', str(root / 'clover'))).resolve()
201 self.cfg = Path(os.environ.get('STUDIO_COPYPARTY_CFG', str(root / 'prod/copyparty/cfg')))
202 self.native = self.cfg / 'copyparty'
203 self.service = os.environ.get('STUDIO_COPYPARTY_SERVICE', 'copyparty')
204 self.users = users
205 self.path = self.cfg / 'manager.json'
206 self.changed = set()
207 self.undo = []
208 self.secret_before = {}
209 self.container_id = None
210 self.db = None
211 self.job = None
212 self.stopped = False
213 self.offline_at = None
214 self.deadline = None
215
216 def path_for(self, value, missing=False):
217 if not isinstance(value, str) or len(value) > 4096 or any(ord(c) < 32 or c in '#[]\\' for c in value) or ' ' in value:
218 raise Error(400, 'Choose a folder without control characters, brackets, or repeated spaces.')
219 if value == '/Volumes/clover' or value.startswith('/Volumes/clover/'):
220 value = str(self.root) + value[len('/Volumes/clover'):]
221 path = Path(value)
222 if not path.is_absolute() or '..' in path.parts or not path.is_relative_to(self.root):
223 raise Error(400, 'Choose a folder inside the Clover file store.')
224 current = self.root
225 try:
226 for part in path.relative_to(self.root).parts:
227 current /= part
228 if current.is_symlink():
229 raise Error(400, 'Choose a folder directly instead of a symbolic link.')
230 if not missing and not path.is_dir():
231 raise Error(404, 'That folder no longer exists. Choose another folder.')
232 if missing and not path.parent.is_dir():
233 raise Error(404, 'The parent folder does not exist. Choose an existing parent.')
234 if path.exists() and not path.is_dir():
235 raise Error(400, 'Choose a folder instead of a file.')
236 except OSError:
237 raise Error(400, 'That folder cannot be opened. Choose another folder.') from None
238 return path
239
240 def container(self):
241 if self.container_id:
242 return self.container_id
243 result = subprocess.run([*PODMAN, 'ps', '--format', '{{.ID}} {{.Image}}'], capture_output=True, text=True, timeout=10, check=True)
244 for row in result.stdout.splitlines():
245 fields = row.split()
246 if len(fields) != 2 or '/copyparty/' not in fields[1]:
247 continue
248 result = subprocess.run([*PODMAN, 'inspect', '--format', '{{json .Mounts}}', fields[0]],
249 capture_output=True, text=True, timeout=10, check=True)
250 mounts = {mount['Destination']: mount['Source'] for mount in json.loads(result.stdout)}
251 if mounts.get('/cfg') == str(self.cfg):
252 if mounts.get('/w/clover') != str(self.root):
253 raise Error(503, 'The share manager points at a different file store. Check its host configuration.')
254 self.container_id = fields[0]
255 return self.container_id
256 raise Error(503, 'Copyparty is offline. Start it before editing shares.')
257
258 def hash(self, password):
259 if not isinstance(password, str) or len(password) > 255 or any(c in password for c in '\r\n\0'):
260 raise Error(400, 'Use a single-line password under 256 characters.')
261 if not password:
262 return ''
263 script = ('import argparse,sys; from pathlib import Path; from copyparty.pwhash import PWHash; '
264 'args=argparse.Namespace(ah_alg="argon2",ah_salt=Path("/cfg/copyparty/ah-salt.txt").read_text().strip()); '
265 'print(PWHash(args).hash(sys.stdin.read()))')
266 result = subprocess.run([*PODMAN, 'exec', '-i', self.container(), 'python3', '-c', script],
267 input=password, capture_output=True, text=True, timeout=25)
268 value = result.stdout.strip()
269 if result.returncode or not re.fullmatch(r'\+[A-Za-z0-9_-]{32}', value):
270 raise Error(502, 'The password could not be secured. Check Copyparty and try again.')
271 return value
272
273 def reload(self, items, timeout=30):
274 deadline = time.monotonic() + timeout
275 instances = runs.nomad('service/' + self.service, timeout=remaining(deadline)) or []
276 if len(instances) != 1:
277 raise Error(503, 'Copyparty is offline or changing allocations. Try again shortly.')
278 address, port = instances[0]['Address'], instances[0]['Port']
279 if not re.fullmatch(r'[0-9a-fA-F:.]+', address) or type(port) is not int or not 1 <= port <= 65535:
280 raise Error(503, 'Copyparty address is unavailable. Redeploy its service.')
281 address = '[' + address + ']' if ':' in address else address
282 request = urllib.request.Request('http://' + address + ':' + str(port) + '/?reload=cfg', headers={
283 'User-Name': 'snow', 'User-Groups': '', 'X-Studio-Idp-' + items['idp_header']: '1',
284 })
285 try:
286 with urllib.request.urlopen(request, timeout=remaining(deadline, 30)) as response:
287 body = response.read(262144)
288 if response.status != 200 or b'config has been reloaded' not in body:
289 raise Error(502, 'Copyparty did not confirm the update. Try again after checking the service.')
290 except (urllib.error.URLError, TimeoutError):
291 raise Error(502, 'Copyparty could not apply the update. Check the service and try again.') from None
292
293 def stop(self):
294 if self.offline_at is None:
295 self.offline_at = time.monotonic()
296 if self.deadline is None:
297 self.deadline = self.offline_at + 160
298 deadline = min(self.deadline, time.monotonic() + 40)
299 if self.job is None:
300 self.job = runs.nomad('job/' + self.service, timeout=remaining(deadline))
301 if self.job is None:
302 raise Error(503, 'Copyparty is offline. Start it before moving folders.')
303 self.job['Stop'] = False
304 runs.nomad('job/' + self.service + '?purge=false', 'DELETE', timeout=remaining(deadline))
305 self.stopped = True
306 while time.monotonic() < deadline:
307 allocations = runs.nomad('job/' + self.service + '/allocations', timeout=remaining(deadline)) or []
308 if all(entry.get('ClientStatus') in {'complete', 'failed', 'lost'} for entry in allocations):
309 self.container_id = None
310 return
311 time.sleep(0.5)
312 raise Error(502, 'Copyparty did not stop in time. Check the service before retrying the move.')
313
314 def start(self):
315 if self.job is None:
316 raise Error(503, 'Copyparty restart settings are unavailable. Redeploy its service.')
317 deadline = min(self.deadline, time.monotonic() + 70)
318 runs.nomad('jobs', 'POST', {'Job': self.job}, timeout=remaining(deadline))
319 items = getattr(self, 'items', None)
320 if items is None:
321 variable = runs.nomad('var/nomad/jobs/' + self.service, timeout=remaining(deadline))
322 if variable is None:
323 raise Error(503, 'Copyparty secrets are unavailable. Redeploy its service.')
324 items = variable['Items']
325 while time.monotonic() < deadline:
326 try:
327 self.reload(items, timeout=remaining(deadline, 30))
328 self.stopped = False
329 return
330 except Error as error:
331 if error.status not in {502, 503}:
332 raise
333 time.sleep(0.5)
334 raise Error(502, 'Copyparty could not restart in time. Check its service logs before retrying.')
335
336 def virtual(self, path):
337 return 'clover' + ('/' + path.relative_to(self.root).as_posix() if path != self.root else '')
338
339 def real(self, virtual):
340 if virtual == 'clover' or virtual.startswith('clover/'):
341 return self.root / virtual[len('clover'):].lstrip('/')
342 if virtual == 'media' or virtual.startswith('media/'):
343 media = Path(os.environ.get('STUDIO_MEDIA_ROOT', str(self.root / 'Media')))
344 return media / virtual[len('media'):].lstrip('/')
345 return None
346
347 def state(self):
348 links = []
349 for key, password, path, perms, count, owner, created, expires in self.db.execute('SELECT * FROM sh ORDER BY t0 DESC,k'):
350 real = self.real(path)
351 links.append({'key': key, 'name': self.data['names'].get(key, real.name if real is not None else Path(path).name or key),
352 'path': str(real) if real is not None else '/' + path,
353 'permissions': perms, 'owner': owner, 'created': created, 'expires': expires,
354 'hasPassword': bool(password), 'files': [row[0] for row in self.db.execute('SELECT vp FROM sf WHERE k=? ORDER BY vp', (key,))] if count else []})
355 return {'root': str(self.root), 'defaultRoot': str(self.root / 'Share'),
356 'accounts': [{**account, 'subject': 'api:' + account['id'], 'username': 'api-' + account['id']} for account in self.data['accounts']],
357 'grants': self.data['grants'], 'links': links}
358
359 def secret(self, key, value):
360 before = self.items.get(key)
361 if value is None:
362 self.items.pop(key, None)
363 else:
364 self.items[key] = value
365 self.secret_before[key] = before
366
367 def account(self, value):
368 value = identifier(value)
369 account = next((entry for entry in self.data['accounts'] if entry['id'] == value), None)
370 if account is None:
371 raise Error(404, 'That API account no longer exists. Reload accounts.')
372 return account
373
374 def grant(self, payload):
375 subject = payload['subject']
376 if not isinstance(subject, str) or ':' not in subject:
377 raise Error(400, 'Choose a Snow Sign On user or API account.')
378 kind, value = subject.split(':', 1)
379 identifier(value)
380 if kind == 'api':
381 self.account(value)
382 elif kind == 'user':
383 if not any(user['id'] == value and user.get('enabled') is True and user.get('kind') in {None, 'user'} for user in self.users):
384 raise Error(400, 'Choose an enabled Snow Sign On user.')
385 else:
386 raise Error(400, 'Choose a Snow Sign On user or API account.')
387 path = str(self.path_for(payload['path']))
388 perms = permissions(payload['permissions'], 'rwmda.g')
389 entry = {'id': identifier(payload['id']) if payload['id'] is not None else str(uuid.uuid4()),
390 'subject': subject, 'path': path, 'permissions': perms}
391 found = next((grant for grant in self.data['grants'] if grant['id'] == entry['id']), None)
392 if payload['id'] is not None and found is None:
393 raise Error(404, 'That folder permission no longer exists. Reload permissions.')
394 if any(grant['id'] != entry['id'] and grant['subject'] == subject and grant['path'] == path for grant in self.data['grants']):
395 raise Error(409, 'This account already has permissions for that folder. Edit its existing permission.')
396 self.data['grants'] = [grant for grant in self.data['grants'] if grant['id'] != entry['id']] + [entry]
397
398 def owner_permissions(self, owner, path, required):
399 _, entries = volumes(configuration(self.cfg, self.data, self.items).decode())
400 url = '/' + self.virtual(path)
401 entry = max((entry for entry in entries if url == entry['url'] or url.startswith(entry['url'].rstrip('/') + '/')),
402 key=lambda entry: len(entry['url']))
403 groups = {'acct'}
404 user = next((user for user in self.users if user.get('username') == owner), None)
405 if user is not None:
406 if user.get('enabled') is not True or user.get('kind') not in {None, 'user'} or user.get('requiredActions'):
407 raise Error(409, 'The share owner cannot sign in. Enable their account before editing this share.')
408 groups.update(user.get('groups', []))
409 groups.add('snow-user-' + identifier(user['id']))
410 effective = set()
411 excluded_permissions = set()
412 for rule in entry['accs']:
413 perms, principals = rule.split(':', 1)
414 names = {name.strip() for name in principals.split(',')}
415 selected = owner in names or '*' in names or any('@' + group in names for group in groups)
416 excluded = '-' + owner in names or any('-@' + group in names for group in groups)
417 expanded = perms.replace('A', 'rwmda.')
418 if selected:
419 effective.update(expanded)
420 if excluded:
421 excluded_permissions.update(expanded)
422 if set(required) - (effective - excluded_permissions):
423 raise Error(409, 'The share owner needs these permissions at the destination. Grant them access before changing this share.')
424
425 def link(self, action, payload):
426 key = payload['key'] if action == 'link.update' else secrets.token_urlsafe(12)
427 if not isinstance(key, str) or not re.fullmatch(r'[A-Za-z0-9_-]{1,200}', key):
428 raise Error(400, 'Choose a share from the list.')
429 old = self.db.execute('SELECT * FROM sh WHERE k=?', (key,)).fetchone()
430 if action == 'link.update' and old is None:
431 raise Error(404, 'That share no longer exists. Reload shares.')
432 name = label(payload['name'])
433 perms = permissions(payload['permissions'], 'rwmd')
434 expires = payload['expires']
435 if type(expires) is not int or expires < 0 or expires > 2**53:
436 raise Error(400, 'Choose an expiration date or keep the share indefinitely.')
437 password = payload['password']
438 if password is not None and not isinstance(password, str):
439 raise Error(400, 'Enter a password or leave the existing password unchanged.')
440 if password is None:
441 password = old[1] if old is not None else ''
442 else:
443 password = self.hash(password)
444 create = action == 'link.create' and payload['createFolder'] is True
445 if action == 'link.create' and type(payload['createFolder']) is not bool:
446 raise Error(400, 'Choose an existing folder or create a new one.')
447 destination = payload['path']
448 share_root = self.root / 'Share'
449 if create and (not destination or Path(destination).parent == share_root or str(destination).startswith('/Volumes/clover/Share/')):
450 self.path_for(str(share_root), missing=True)
451 if not share_root.exists():
452 share_root.mkdir(mode=0o775)
453 share_root.chmod(0o775)
454 owner = self.root.stat()
455 os.chown(share_root, owner.st_uid, owner.st_gid)
456 self.undo.append(lambda: share_root.rmdir())
457 if not destination:
458 destination = str(share_root / name)
459 destination = self.path_for(destination, missing=create or action == 'link.update' and payload['mode'] == 'move')
460 files = [row[0] for row in self.db.execute('SELECT vp FROM sf WHERE k=?', (key,))] if old is not None and old[4] else []
461 mode = payload.get('mode', 'retarget')
462 if mode not in {'retarget', 'move'}:
463 raise Error(400, 'Choose whether to move the folder or use another folder.')
464 if create:
465 try:
466 destination.mkdir(mode=0o775)
467 destination.chmod(0o775)
468 except FileExistsError:
469 raise Error(409, 'That folder already exists. Choose it as an existing folder or change its name.') from None
470 owner = destination.parent.stat()
471 os.chown(destination, owner.st_uid, owner.st_gid)
472 self.undo.append(lambda: destination.rmdir())
473 if old is not None and mode == 'move':
474 source = self.real(old[2])
475 if source is None:
476 raise Error(400, 'Only folders inside the Clover file store can be moved here.')
477 source = self.path_for(str(source))
478 if source != destination:
479 for grant in self.data['grants']:
480 if Path(grant['path']).is_relative_to(source):
481 grant['path'] = str(destination / Path(grant['path']).relative_to(source))
482 self.owner_permissions(old[5], destination, perms)
483 self.move(source, destination)
484 for filename in files:
485 if (not isinstance(filename, str) or filename in {'.', '..'} or '/' in filename or '\\' in filename
486 or not (destination / filename).is_file() or (destination / filename).is_symlink()):
487 raise Error(409, 'The destination must contain every selected file. Choose a matching folder.')
488 if old is not None:
489 self.owner_permissions(old[5], destination, perms)
490 self.changed.add(key)
491 self.db.execute('UPDATE sh SET pw=?,vp=?,pr=?,t1=? WHERE k=?', (password, self.virtual(destination), perms, expires, key))
492 else:
493 self.changed.add(key)
494 self.db.execute('INSERT INTO sh VALUES(?,?,?,?,?,?,?,?)', (key, password, self.virtual(destination), perms, 0, 'snow', int(time.time()), expires))
495 self.data['names'][key] = name
496
497 def move(self, source, destination):
498 if source == self.root or source == self.root / 'Share' or destination.is_relative_to(source) or source.is_relative_to(destination):
499 raise Error(400, 'Choose separate folders below the Clover file store.')
500 if destination.exists():
501 raise Error(409, 'The destination already exists. Choose a new folder name.')
502 if source.stat().st_dev != destination.parent.stat().st_dev:
503 raise Error(409, 'Choose a destination on the same filesystem to move this folder.')
504 mounts = Path('/proc/self/mountinfo')
505 if mounts.exists():
506 for row in mounts.read_text().splitlines():
507 mount = Path(re.sub(r'\\([0-7]{3})', lambda match: chr(int(match[1], 8)), row.split()[4]))
508 if mount.is_relative_to(source):
509 raise Error(409, 'A mounted filesystem cannot be moved. Choose a folder without mounts inside it.')
510 elif os.path.ismount(source):
511 raise Error(409, 'A mounted filesystem cannot be moved. Choose a folder inside it.')
512 rows = self.db.execute('SELECT k,vp,st FROM sh').fetchall()
513 for key, virtual, count in rows:
514 real = self.real(virtual)
515 if real is not None and source != real and source.is_relative_to(real):
516 selected = {row[0] for row in self.db.execute('SELECT vp FROM sf WHERE k=?', (key,))} if count else None
517 if selected is None or source.relative_to(real).parts[0] in selected:
518 raise Error(409, 'Another share includes this folder through its parent. Retarget that share before moving the folder.')
519 self.stop()
520 for key, virtual, _ in rows:
521 real = self.real(virtual)
522 if real is not None and real.is_relative_to(source):
523 self.changed.add(key)
524 self.db.execute('UPDATE sh SET vp=? WHERE k=?', (self.virtual(destination / real.relative_to(source)), key))
525 for grant in self.data['grants']:
526 if Path(grant['path']).is_relative_to(source):
527 grant['path'] = str(destination / Path(grant['path']).relative_to(source))
528 rename(source, destination)
529 self.undo.append(lambda: rename(destination, source))
530
531 def dispatch(self, action, payload):
532 extra = {}
533 if action.startswith('account.'):
534 account = {'id': str(uuid.uuid4()), 'name': label(payload['name'])} if action == 'account.create' else self.account(payload['id'])
535 value = account['id']
536 if action == 'account.create':
537 if any(user.get('username') == 'api-' + value for user in self.users):
538 raise Error(409, 'That API username already belongs to a Snow Sign On user. Try creating the account again.')
539 self.data['accounts'].append(account)
540 if action in {'account.update', 'account.create'}:
541 account['name'] = label(payload['name'])
542 if action in {'account.create', 'account.rotate'}:
543 password = secrets.token_urlsafe(32)
544 self.secret('file_api_' + value.replace('-', ''), self.hash(password))
545 extra = {'account': {**account, 'subject': 'api:' + value, 'username': 'api-' + value}, 'password': password}
546 if action in {'account.delete', 'account.rotate'}:
547 session_path = self.native / 'sessions.db'
548 if session_path.exists():
549 with sqlite3.connect(session_path) as sessions:
550 sessions.execute('DELETE FROM us WHERE un=?', ('api-' + value,))
551 if action == 'account.delete':
552 self.data['accounts'].remove(account)
553 self.data['grants'] = [grant for grant in self.data['grants'] if grant['subject'] != 'api:' + value]
554 self.secret('file_api_' + value.replace('-', ''), None)
555 elif action == 'grant.save':
556 self.grant(payload)
557 elif action == 'grant.delete':
558 value = identifier(payload['id'])
559 if not any(grant['id'] == value for grant in self.data['grants']):
560 raise Error(404, 'That folder permission no longer exists. Reload permissions.')
561 self.data['grants'] = [grant for grant in self.data['grants'] if grant['id'] != value]
562 elif action in {'link.create', 'link.update'}:
563 self.link(action, payload)
564 elif action == 'link.delete':
565 key = payload['key']
566 if not isinstance(key, str) or not self.db.execute('SELECT 1 FROM sh WHERE k=?', (key,)).fetchone():
567 raise Error(404, 'That share no longer exists. Reload shares.')
568 self.changed.add(key)
569 self.db.execute('DELETE FROM sh WHERE k=?', (key,))
570 self.db.execute('DELETE FROM sf WHERE k=?', (key,))
571 self.data['names'].pop(key, None)
572 return extra
573
574 def apply(self, action, payload):
575 self.data = json.loads(runs.read(self.path, missing='{"accounts":[],"grants":[],"names":{}}'))
576 origin = Path(self.data.get('root', os.environ.get('STUDIO_SOURCE_CLOVER_ROOT', '/srv/clover')))
577 if 'root' not in self.data and self.data.get('grants'):
578 origin = Path(os.environ.get('STUDIO_SOURCE_CLOVER_ROOT', str(self.root)))
579 if origin != self.root:
580 for grant in self.data['grants']:
581 grant['path'] = str(self.root / Path(grant['path']).relative_to(origin))
582 self.data['root'] = str(self.root)
583 if not isinstance(self.data.get('accounts'), list) or not isinstance(self.data.get('grants'), list) or not isinstance(self.data.get('names'), dict):
584 raise Error(503, 'Share settings need repair before editing. Check the host service.')
585 database = self.native / 'shares.db'
586 if not database.is_file():
587 raise Error(503, 'Copyparty shares are not ready. Start Copyparty and try again.')
588 self.db = sqlite3.connect(database.as_uri() + '?mode=rw', uri=True, timeout=5)
589 self.db.row_factory = None
590 try:
591 version = self.db.execute("SELECT v FROM kv WHERE k='sver'").fetchone()
592 if version != (2,) or [row[1] for row in self.db.execute('PRAGMA table_info(sh)')] != ['k', 'pw', 'vp', 'pr', 'st', 'un', 't0', 't1']:
593 raise Error(503, 'This Copyparty share format is unsupported. Update Snowglobe before editing shares.')
594 if action == 'state':
595 return self.state()
596 if action == 'folders.list':
597 path = self.path_for(payload['path'])
598 with os.scandir(path) as entries:
599 directories = sorted(({'name': entry.name, 'path': str(path / entry.name)} for entry in entries
600 if not entry.name.startswith('.') and entry.is_dir(follow_symlinks=False)), key=lambda entry: entry['name'].casefold())
601 return {'path': str(path), 'directories': directories}
602 self.container()
603 variable_path = 'nomad/jobs/' + self.service
604 variable = runs.nomad('var/' + variable_path)
605 if variable is None:
606 raise Error(503, 'Copyparty secrets are unavailable. Redeploy its service.')
607 self.items = dict(variable['Items'])
608 original_items = dict(self.items)
609 config_path = self.cfg / 'copyparty.conf'
610 old_config = config_path.read_bytes()
611 metadata = config_path.stat()
612 owner = (metadata.st_uid, metadata.st_gid)
613 self.db.execute('BEGIN IMMEDIATE')
614 original_rows = {row[0]: row for row in self.db.execute('SELECT * FROM sh')}
615 original_files = {key: self.db.execute('SELECT * FROM sf WHERE k=?', (key,)).fetchall() for key in original_rows}
616 committed = False
617 wrote_files = False
618 old_manager = self.path.read_bytes() if self.path.exists() else None
619 changed_secrets = False
620 after_rows = {}
621 try:
622 extra = self.dispatch(action, payload)
623 rendered = configuration(self.cfg, self.data, self.items)
624 if self.secret_before:
625 runs.nomad('var/' + variable_path + '?cas=' + str(variable['ModifyIndex']), 'PUT',
626 {'Namespace': 'default', 'Path': variable_path, 'Items': self.items})
627 changed_secrets = True
628 atomic(config_path, rendered, owner)
629 wrote_files = True
630 atomic(self.path, json.dumps(self.data, ensure_ascii=False).encode())
631 after_rows = {key: self.db.execute('SELECT * FROM sh WHERE k=?', (key,)).fetchone() for key in self.changed}
632 self.db.commit()
633 committed = True
634 if self.job is not None:
635 self.start()
636 else:
637 self.reload(self.items)
638 return {**self.state(), **extra}
639 except Exception:
640 if self.job is not None:
641 self.deadline = self.offline_at + 280
642 self.stop()
643 self.db.rollback()
644 if committed and self.changed:
645 self.db.execute('BEGIN IMMEDIATE')
646 for key in self.changed:
647 current = self.db.execute('SELECT * FROM sh WHERE k=?', (key,)).fetchone()
648 if current != after_rows[key]:
649 raise Error(502, 'The update could not finish and a share changed during recovery. Check Copyparty before retrying.') from None
650 self.db.execute('DELETE FROM sh WHERE k=?', (key,))
651 self.db.execute('DELETE FROM sf WHERE k=?', (key,))
652 if key in original_rows:
653 self.db.execute('INSERT INTO sh VALUES(?,?,?,?,?,?,?,?)', original_rows[key])
654 self.db.executemany('INSERT INTO sf VALUES(?,?)', original_files[key])
655 self.db.commit()
656 for undo in reversed(self.undo):
657 undo()
658 if wrote_files:
659 atomic(config_path, old_config, owner)
660 if old_manager is None:
661 self.path.unlink(missing_ok=True)
662 else:
663 atomic(self.path, old_manager)
664 if changed_secrets:
665 latest = runs.nomad('var/' + variable_path)
666 items = dict(latest['Items'])
667 for key, value in self.secret_before.items():
668 if items.get(key) != self.items.get(key):
669 raise Error(502, 'The API credential changed during recovery. Check Copyparty before retrying.') from None
670 if value is None:
671 items.pop(key, None)
672 else:
673 items[key] = value
674 runs.nomad('var/' + variable_path + '?cas=' + str(latest['ModifyIndex']), 'PUT',
675 {'Namespace': 'default', 'Path': variable_path, 'Items': items})
676 if self.job is not None:
677 self.items = original_items
678 self.start()
679 elif committed:
680 self.reload(original_items)
681 raise
682 finally:
683 self.db.close()
684
685
686def handle(request):
687 action, payload, users = request.get('action'), request.get('payload'), request.get('users')
688 if (not isinstance(action, str) or action not in ACTIONS or not isinstance(payload, dict)
689 or set(payload) != ACTIONS[action] or not isinstance(users, list)):
690 raise Error(400, 'Use the fields required by this share action.')
691 manager = Manager(users)
692 if not manager.cfg.is_dir():
693 raise Error(503, 'Copyparty settings are unavailable. Deploy its service before editing shares.')
694 lock = os.open(manager.cfg / '.manager.lock', os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW, 0o600)
695 with os.fdopen(lock, 'w') as locked:
696 fcntl.flock(locked, fcntl.LOCK_EX)
697 return manager.apply(action, payload)
service/copyparty/prepare.py+35-25
...@@ -2,35 +2,45 @@...@@ -2,35 +2,45 @@
2import json2import json
3import os3import os
4from pathlib import Path4from pathlib import Path
5import re
6import subprocess5import subprocess
7import sys6import sys
87
8root = Path(__file__).resolve().parents[2]
9sys.path.insert(0, str(root / 'tools'))
10import manage as copyparty
11
912
10data = json.load(sys.stdin)13data = json.load(sys.stdin)
11root = Path(data["hostRoot"]) / "cfg"14cfg = Path(data['hostRoot']) / 'cfg'
12service = Path(data["hostRoot"]).name15service = Path(data['hostRoot']).name
13secret = json.loads(subprocess.check_output(16variable = json.loads(subprocess.check_output(
14 ["nomad", "var", "get", "-out=json", f"nomad/jobs/{service}"],17 ['nomad', 'var', 'get', '-out=json', 'nomad/jobs/' + service],
15 text=True, stderr=subprocess.DEVNULL,18 text=True, stderr=subprocess.DEVNULL,
16))["Items"]["idp_header"]19))
17if not re.fullmatch(r"[0-9a-f]{32}", secret):20items = variable['Items']
18 raise ValueError("invalid Copyparty identity secret")21state_path = cfg / 'manager.json'
19config = Path(__file__).with_name("copyparty.conf").read_text()22state = json.loads(state_path.read_text()) if state_path.exists() else {'accounts': [], 'grants': [], 'names': {}}
20if config.count("STUDIO_IDP_HEADER") != 1:23canonical_root = Path(os.environ.get('STUDIO_CLOVER_ROOT', str(Path(os.environ.get('STUDIO_ROOT', '/srv')) / 'clover')))
21 raise ValueError("expected one Copyparty identity header marker")24origin = Path(state.get('root', str(canonical_root)))
22config = config.replace("STUDIO_IDP_HEADER", "X-Studio-Idp-" + secret)25if origin != canonical_root:
23if config.count("STUDIO_MIN_FREE_GB") != 1:26 for grant in state['grants']:
24 raise ValueError("expected one Copyparty free-space marker")27 grant['path'] = str(canonical_root / Path(grant['path']).relative_to(origin))
25capacity = os.statvfs(data["hostRoot"])28state['root'] = str(canonical_root)
26reserve = max(1, min(16, capacity.f_blocks * capacity.f_frsize // (4 * 10**9)))29missing = ['file_api_' + account['id'].replace('-', '') for account in state['accounts']
27config = config.replace("STUDIO_MIN_FREE_GB", str(reserve))30 if 'file_api_' + account['id'].replace('-', '') not in items]
28target = root / "copyparty.conf"31if missing and service != 'copyparty':
29if not target.exists() or target.read_text() != config:32 source = copyparty.runs.nomad('var/nomad/jobs/copyparty')
30 pending = root / ".copyparty.conf.pending"33 for key in missing:
31 pending.write_text(config)34 if source is None or key not in source['Items']:
32 os.chmod(pending, 0o600)35 raise RuntimeError('Copyparty API credentials are missing from the source Nomad variables')
33 os.chown(pending, data["uid"], data["uid"])36 items[key] = source['Items'][key]
34 os.replace(pending, target)37 copyparty.runs.nomad('var/nomad/jobs/' + service + '?cas=' + str(variable['ModifyIndex']), 'PUT',
38 {'Namespace': 'default', 'Path': 'nomad/jobs/' + service, 'Items': items})
39elif missing:
40 raise RuntimeError('Copyparty API credentials are missing; restore its Nomad variables before starting it')
41rendered = copyparty.configuration(cfg, state, items, Path(__file__).with_name('copyparty.conf'))
42target = cfg / 'copyparty.conf'
43if not target.exists() or target.read_bytes() != rendered:
44 copyparty.atomic(target, rendered, (data['uid'], data['uid']))
35os.chmod(target, 0o600)45os.chmod(target, 0o600)
36os.chown(target, data["uid"], data["uid"])46os.chown(target, data['uid'], data['uid'])
tools/copyparty-cutover-test.py created+299
...@@ -0,0 +1,299 @@
1#!/usr/bin/env python3
2import importlib.util
3import json
4import os
5from pathlib import Path
6import sqlite3
7import sys
8import tempfile
9import unittest
10from unittest.mock import patch
11
12spec = importlib.util.spec_from_file_location('cutover', Path(__file__).with_name('copyparty-cutover.py'))
13cutover = importlib.util.module_from_spec(spec)
14spec.loader.exec_module(cutover)
15copyparty = cutover.copyparty
16public_statuses = cutover.public_statuses
17
18
19class Cutover(unittest.TestCase):
20 def setUp(self):
21 self.temporary = tempfile.TemporaryDirectory(prefix='copyparty-cutover-')
22 self.addCleanup(self.temporary.cleanup)
23 self.base = Path(self.temporary.name)
24 self.root = self.base / 'clover'
25 self.cfg = self.base / 'cfg'
26 (self.root / 'Share').mkdir(parents=True)
27 (self.cfg / 'copyparty').mkdir(parents=True)
28 self.environment = patch.dict(os.environ, {'STUDIO_CLOVER_ROOT': str(self.root), 'STUDIO_COPYPARTY_CFG': str(self.cfg)})
29 self.environment.start()
30 self.addCleanup(self.environment.stop)
31 self.manager = copyparty.Manager([])
32 self.manager.job = {'ID': 'copyparty', 'Stop': False, 'Meta': {'studio_hostname': 'copyparty.example.invalid'}}
33 self.events = []
34 edge = patch.object(cutover, 'public_statuses', side_effect=lambda host, keys, deadline: {key: 200 for key in keys})
35 edge.start()
36 self.addCleanup(edge.stop)
37
38 def stop():
39 self.events.append('stop')
40 if self.manager.offline_at is None:
41 self.manager.offline_at = cutover.time.monotonic()
42 self.manager.stopped = True
43
44 def start():
45 self.events.append('start')
46 self.manager.stopped = False
47
48 self.manager.stop = stop
49 self.manager.start = start
50 for source, _ in cutover.MOVES:
51 path = self.root / source
52 (path / 'nested').mkdir(parents=True)
53 (path / 'selected.txt').write_text('Disposable fixture')
54 (path / 'nested' / 'child.txt').write_text('Disposable child')
55 (self.cfg / 'copyparty.conf').write_text('Fixture configuration')
56 self.database = self.cfg / 'copyparty/shares.db'
57 with sqlite3.connect(self.database) as db:
58 db.executescript("CREATE TABLE kv(k text,v int); INSERT INTO kv VALUES('sver',2); CREATE TABLE sh(k text,pw text,vp text,pr text,st int,un text,t0 int,t1 int); CREATE TABLE sf(k text,vp text);")
59 db.executemany('INSERT INTO sh VALUES(?,?,?,?,?,?,?,?)', [
60 ('folder-key', 'opaque-hash', 'clover/Documents/Public/Snowbound', 'rw', 0, 'snow', 10, 20),
61 ('nested-key', '', 'clover/Documents/Public/Snowbound/nested', 'r', 0, 'snow', 30, 0),
62 ('selected-key', 'another-hash', 'clover/Documents/EvilTakeout', 'r', 1, 'snow', 40, 50),
63 ('unaffected-key', '', 'clover/Other', 'r', 0, 'snow', 60, 0),
64 ])
65 db.execute('INSERT INTO sf VALUES(?,?)', ('selected-key', 'selected.txt'))
66 with sqlite3.connect(self.database) as db:
67 self.original = cutover.rows(db)
68 self.before_content = {str(path.relative_to(self.root)): path.read_bytes() for path in self.root.rglob('*.txt')}
69
70 def test_move_preserves_keys_selections_passwords_inodes_and_content(self):
71 expected = cutover.plan(self.manager)
72 result = cutover.apply(self.manager, expected, self.base / 'backups')
73 self.assertEqual(result['result'], 'moved')
74 self.assertEqual(self.events, ['stop', 'start'])
75 with sqlite3.connect(self.database) as db:
76 actual = cutover.rows(db)
77 self.assertEqual(actual, cutover.remapped(self.manager, self.original, cutover.mappings(self.manager)))
78 self.assertEqual(actual['sf'], self.original['sf'])
79 for detail in expected['moves']:
80 source, destination = Path(detail['source']), Path(detail['destination'])
81 self.assertFalse(source.exists())
82 self.assertEqual(destination.stat().st_ino, detail['inode'])
83 for old_path, contents in self.before_content.items():
84 old = self.root / old_path
85 if old.is_relative_to(source):
86 self.assertEqual((destination / old.relative_to(source)).read_bytes(), contents)
87 backup = Path(result['backup'])
88 for child in backup.iterdir():
89 self.assertEqual(child.stat().st_mode & 0o777, 0o600)
90 self.assertEqual(backup.stat().st_mode & 0o777, 0o700)
91
92 def test_public_probe_sends_bounded_verified_head_and_preserves_redirect_status(self):
93 response = cutover.subprocess.CompletedProcess([], 0, stdout='302', stderr='')
94 with patch.object(cutover.subprocess, 'run', return_value=response) as run:
95 actual = public_statuses('copyparty.example.invalid', ['selected file'], cutover.time.monotonic() + 1)
96 self.assertEqual(actual, {'selected file': 302})
97 arguments = run.call_args.args[0]
98 self.assertEqual(arguments[:2], ['curl', '--disable'])
99 self.assertIn('--head', arguments)
100 self.assertEqual(arguments[arguments.index('--output') + 1], '/dev/null')
101 self.assertEqual(arguments[arguments.index('--write-out') + 1], '%{http_code}')
102 self.assertEqual(arguments[arguments.index('--noproxy') + 1], '*')
103 self.assertEqual(arguments[-2:], ['--', 'https://copyparty.example.invalid/shr/selected%20file/'])
104 self.assertLessEqual(run.call_args.kwargs['timeout'], 1)
105 self.assertEqual(float(arguments[arguments.index('--max-time') + 1]), run.call_args.kwargs['timeout'])
106 for flag in ['--insecure', '-k', '--location', '-L', '--include', '-i']:
107 self.assertNotIn(flag, arguments)
108
109 def test_public_probe_treats_network_tls_and_process_timeout_as_unavailable(self):
110 for response in [
111 cutover.subprocess.CompletedProcess([], 35, stdout='000', stderr='TLS problem'),
112 cutover.subprocess.CompletedProcess([], 28, stdout='200', stderr='Timeout'),
113 cutover.subprocess.CompletedProcess([], 0, stdout='000', stderr=''),
114 cutover.subprocess.CompletedProcess([], 0, stdout='200 headers', stderr=''),
115 ]:
116 with patch.object(cutover.subprocess, 'run', return_value=response):
117 self.assertEqual(public_statuses('host', ['key'], cutover.time.monotonic() + 1), {'key': None})
118 with patch.object(cutover.subprocess, 'run', side_effect=cutover.subprocess.TimeoutExpired('curl', 0.1)):
119 self.assertEqual(public_statuses('host', ['key'], cutover.time.monotonic() + 0.1), {'key': None})
120 with patch.object(cutover.subprocess, 'run') as run:
121 self.assertEqual(public_statuses('host', ['key'], cutover.time.monotonic() - 1), {'key': None})
122 run.assert_not_called()
123
124 def test_stalled_probe_children_cannot_hold_the_executor_past_its_budget(self):
125 run = cutover.subprocess.run
126 def stalled(arguments, **options):
127 return run([sys.executable, '-c', 'import time; time.sleep(10)'], **options)
128 started = cutover.time.monotonic()
129 with patch.object(cutover.subprocess, 'run', side_effect=stalled):
130 actual = public_statuses('host', ['one', 'two'], started + 0.15)
131 self.assertEqual(actual, {'one': None, 'two': None})
132 self.assertLess(cutover.time.monotonic() - started, 1.5)
133
134 def test_public_poll_recovers_and_timeout_obeys_its_budget(self):
135 clock = [0.0]
136 def sleep(delay):
137 clock[0] += delay
138 with patch.object(cutover.time, 'monotonic', side_effect=lambda: clock[0]), patch.object(cutover.time, 'sleep', side_effect=sleep):
139 with patch.object(cutover, 'public_statuses', side_effect=[{'key': 503}, {'key': 200}]):
140 cutover.wait_public('host', {'key': 200}, 1)
141 self.assertEqual(clock[0], 0.5)
142 with patch.object(cutover, 'public_statuses', return_value={'key': None}), self.assertRaises(TimeoutError):
143 cutover.wait_public('host', {'key': 200}, 1)
144 self.assertEqual(clock[0], 1)
145
146 def test_edge_timeout_rolls_back_and_records_recovery_evidence(self):
147 expected = cutover.plan(self.manager)
148 with patch.object(cutover, 'wait_public', side_effect=[TimeoutError('injected edge timeout'), None]):
149 with self.assertRaisesRegex(TimeoutError, 'edge timeout'):
150 cutover.apply(self.manager, expected, self.base / 'backups')
151 self.assertEqual(cutover.plan(self.manager), expected)
152 self.assertEqual(self.events, ['stop', 'start', 'stop', 'start'])
153 evidence = json.loads(next((self.base / 'backups').glob('*/result.json')).read_text())
154 self.assertEqual(evidence['result'], 'rolled-back')
155 self.assertEqual(evidence['publicStatuses'], {key: 200 for key in expected['affectedKeys']})
156
157 def test_existing_grants_follow_moves_and_configuration_is_rerendered(self):
158 source, destination = cutover.mappings(self.manager)[0]
159 state = {'accounts': [], 'grants': [{'id': 'fixture', 'subject': 'user:fixture', 'path': str(source), 'permissions': 'r'}], 'names': {}}
160 (self.cfg / 'manager.json').write_text(json.dumps(state))
161 expected = cutover.plan(self.manager)
162 with patch.object(copyparty.runs, 'nomad', return_value={'Items': {}}), patch.object(copyparty, 'configuration', return_value=b'Updated configuration') as render:
163 cutover.apply(self.manager, expected, self.base / 'backups')
164 saved = json.loads((self.cfg / 'manager.json').read_text())
165 self.assertEqual(saved['grants'][0]['path'], str(destination))
166 self.assertEqual(render.call_args.args[1]['grants'], saved['grants'])
167 self.assertEqual((self.cfg / 'copyparty.conf').read_bytes(), b'Updated configuration')
168
169 def test_failed_restart_restores_files_and_database_before_restart(self):
170 def start():
171 self.events.append('start')
172 if self.events.count('start') == 1:
173 raise RuntimeError('injected startup failure')
174 self.manager.stopped = False
175 self.manager.start = start
176 expected = cutover.plan(self.manager)
177 with self.assertRaisesRegex(RuntimeError, 'injected startup failure'):
178 cutover.apply(self.manager, expected, self.base / 'backups')
179 self.assertEqual(self.events, ['stop', 'start', 'stop', 'start'])
180 self.assertEqual(cutover.plan(self.manager), expected)
181 with sqlite3.connect(self.database) as db:
182 self.assertEqual(cutover.rows(db), self.original)
183 for path, content in self.before_content.items():
184 self.assertEqual((self.root / path).read_bytes(), content)
185
186 def test_rollback_uses_actual_helper_stop_and_start_with_frozen_nomad_job(self):
187 self.manager.stop = copyparty.Manager.stop.__get__(self.manager)
188 self.manager.start = copyparty.Manager.start.__get__(self.manager)
189 self.manager.job = None
190 requests = []
191 def nomad(path, method='GET', body=None, **_):
192 requests.append((path, method, body))
193 if path == 'job/copyparty' and method == 'GET':
194 return {'ID': 'copyparty', 'Stop': False, 'Meta': {'immutable': 'fixture', 'studio_hostname': 'copyparty.example.invalid'}}
195 if path.endswith('/allocations'):
196 return [{'ClientStatus': 'complete'}]
197 if path.startswith('var/'):
198 return {'Items': {'idp_header': 'fixture'}}
199 return {}
200 expected = cutover.plan(self.manager)
201 with patch.object(copyparty.runs, 'nomad', side_effect=nomad), patch.object(self.manager, 'reload', side_effect=[copyparty.Error(400, 'injected reload rejection'), None]):
202 with self.assertRaisesRegex(copyparty.Error, 'injected reload rejection'):
203 cutover.apply(self.manager, expected, self.base / 'backups')
204 self.assertEqual(cutover.plan(self.manager), expected)
205 registrations = [body for path, method, body in requests if path == 'jobs' and method == 'POST']
206 self.assertEqual(registrations, [{'Job': self.manager.job}, {'Job': self.manager.job}])
207 self.assertEqual(len([path for path, method, _ in requests if method == 'DELETE']), 2)
208 self.assertFalse(self.manager.stopped)
209
210 def test_partial_rename_failure_rolls_back_before_start(self):
211 native_rename = copyparty.rename
212 calls = 0
213 def rename(source, destination):
214 nonlocal calls
215 calls += 1
216 if calls == 3:
217 raise RuntimeError('injected third rename failure')
218 self.assertTrue(self.manager.stopped)
219 native_rename(source, destination)
220 expected = cutover.plan(self.manager)
221 with patch.object(copyparty, 'rename', side_effect=rename), self.assertRaisesRegex(RuntimeError, 'third rename'):
222 cutover.apply(self.manager, expected, self.base / 'backups')
223 self.assertEqual(cutover.plan(self.manager), expected)
224 self.assertEqual(self.events, ['stop', 'start'])
225
226 def test_stop_timeout_still_restarts_the_frozen_job(self):
227 def stop():
228 self.manager.offline_at = cutover.time.monotonic()
229 self.manager.stopped = True
230 raise RuntimeError('injected stop timeout')
231 self.manager.stop = stop
232 expected = cutover.plan(self.manager)
233 with self.assertRaisesRegex(RuntimeError, 'stop timeout'):
234 cutover.apply(self.manager, expected, self.base / 'backups')
235 self.assertEqual(self.events, ['start'])
236 self.assertEqual(cutover.plan(self.manager), expected)
237
238 def test_ambiguous_stop_response_reregisters_even_without_stopped_flag(self):
239 def stop():
240 self.manager.offline_at = cutover.time.monotonic()
241 self.manager.stopped = False
242 raise TimeoutError('lost Nomad stop response')
243 self.manager.stop = stop
244 expected = cutover.plan(self.manager)
245 with self.assertRaisesRegex(TimeoutError, 'lost Nomad stop response'):
246 cutover.apply(self.manager, expected, self.base / 'backups')
247 self.assertEqual(self.events, ['start'])
248 self.assertEqual(cutover.plan(self.manager), expected)
249
250 def test_verification_budget_reserves_time_for_restart(self):
251 expected = cutover.plan(self.manager)
252 original_tree = cutover.tree
253 def timed_tree(path, deadline=None):
254 if deadline is not None:
255 raise TimeoutError('injected verification deadline')
256 return original_tree(path)
257 with patch.object(cutover, 'tree', side_effect=timed_tree), self.assertRaisesRegex(TimeoutError, 'verification deadline'):
258 cutover.apply(self.manager, expected, self.base / 'backups')
259 self.assertEqual(self.events, ['stop', 'start'])
260 self.assertEqual(cutover.plan(self.manager), expected)
261
262 def test_source_and_database_drift_stop_before_mutation(self):
263 expected = cutover.plan(self.manager)
264 (self.root / cutover.MOVES[0][0] / 'changed.txt').write_text('Concurrent disposable write')
265 with self.assertRaisesRegex(ValueError, 'plan changed'):
266 cutover.apply(self.manager, expected, self.base / 'backups')
267 self.assertEqual(self.events, [])
268
269 def test_parent_shares_and_parent_selections_require_explicit_url_preservation(self):
270 for selected in [0, 1]:
271 with sqlite3.connect(self.database) as db:
272 db.execute('INSERT OR REPLACE INTO sh VALUES(?,?,?,?,?,?,?,?)', ('parent', '', 'clover/Documents/Public', 'r', selected, 'snow', 1, 0))
273 if selected:
274 db.execute('INSERT INTO sf VALUES(?,?)', ('parent', 'Snowbound'))
275 with self.assertRaisesRegex(ValueError, 'parent of a moved folder'):
276 cutover.plan(self.manager)
277 with sqlite3.connect(self.database) as db:
278 db.execute("DELETE FROM sh WHERE k='parent'")
279 db.execute("DELETE FROM sf WHERE k='parent'")
280
281 def test_symlinks_mounts_and_existing_destinations_reject_before_downtime(self):
282 source, destination = cutover.mappings(self.manager)[0]
283 destination.mkdir()
284 with self.assertRaises(ValueError):
285 cutover.plan(self.manager)
286 destination.rmdir()
287 link = source / 'link'
288 link.symlink_to('selected.txt')
289 with self.assertRaisesRegex(ValueError, 'symbolic link'):
290 cutover.plan(self.manager)
291 link.unlink()
292 with patch.object(cutover.os.path, 'ismount', side_effect=lambda path: Path(path) == source):
293 with self.assertRaises(ValueError):
294 cutover.plan(self.manager)
295 self.assertEqual(self.events, [])
296
297
298if __name__ == '__main__':
299 unittest.main()
tools/copyparty-cutover.py created+279
...@@ -0,0 +1,279 @@
1#!/usr/bin/env python3
2import argparse
3import concurrent.futures
4import copy
5import hashlib
6import importlib.util
7import json
8import os
9import re
10from pathlib import Path
11import sqlite3
12import socket
13import subprocess
14import stat
15import time
16import uuid
17from urllib.parse import quote
18
19
20spec = importlib.util.spec_from_file_location("copyparty", Path(__file__).resolve().parent.parent / "service/copyparty/manage.py")
21copyparty = importlib.util.module_from_spec(spec)
22spec.loader.exec_module(copyparty)
23
24
25MOVES = (
26 ('Documents/Public/Snowbound', 'Clover - Snowbound'),
27 ('Documents/Public/Meadow', 'Clover - Meadow Chat'),
28 ('Documents/Public/Bill Wurtz Music', 'Xavier - Bill Wurtz Music'),
29 ('Documents/HOTEWIG Reanimated', 'Collab - HOTEWIG Reanimated'),
30 ('Documents/EvilTakeout', 'Lino - Evil Inc Data Takeout'),
31 ('Documents/Windows7', 'Clover - Windows 7'),
32)
33
34
35def digest(value):
36 return hashlib.sha256(json.dumps(value, ensure_ascii=False, separators=(',', ':')).encode()).hexdigest()
37
38
39def rows(database):
40 if database.execute("SELECT v FROM kv WHERE k='sver'").fetchone() != (2,):
41 raise ValueError('Unsupported Copyparty share database version.')
42 return {
43 'sh': database.execute('SELECT * FROM sh ORDER BY k').fetchall(),
44 'sf': database.execute('SELECT * FROM sf ORDER BY k,vp').fetchall(),
45 }
46
47
48def mappings(manager):
49 return [(manager.root / source, manager.root / 'Share' / name) for source, name in MOVES]
50
51
52def remapped(manager, before, moves):
53 after = copy.deepcopy(before)
54 for index, row in enumerate(before['sh']):
55 source_path = manager.real(row[2].strip('/'))
56 for source, destination in moves:
57 if source_path is not None and source_path.is_relative_to(source):
58 changed = list(row)
59 changed[2] = manager.virtual(destination / source_path.relative_to(source))
60 after['sh'][index] = tuple(changed)
61 break
62 if (source_path is not None and source.is_relative_to(source_path)) or row[2].strip('/') == '':
63 raise ValueError('A share includes a parent of a moved folder. Preserve its URL paths before cutover: ' + row[0])
64 return after
65
66
67def tree(path, deadline=None):
68 def unreadable(error):
69 raise error
70
71 result = []
72 for directory, dirs, files in os.walk(path, followlinks=False, onerror=unreadable):
73 if deadline is not None and time.monotonic() >= deadline:
74 raise TimeoutError("Folder verification exceeded the cutover budget.")
75 base = Path(directory)
76 if base != path and os.path.ismount(base):
77 raise ValueError('A source contains a mounted filesystem: ' + str(base))
78 for entry in [base] + [base / name for name in files] + [base / name for name in dirs if (base / name).is_symlink()]:
79 if deadline is not None and time.monotonic() >= deadline:
80 raise TimeoutError("Folder verification exceeded the cutover budget.")
81 info = entry.lstat()
82 if stat.S_ISLNK(info.st_mode):
83 raise ValueError('Review this source symbolic link before cutover: ' + str(entry))
84 result.append((str(entry.relative_to(path)), info.st_dev, info.st_ino, info.st_mode, info.st_size if not stat.S_ISDIR(info.st_mode) else 0, info.st_mtime_ns))
85 dirs[:] = [name for name in dirs if not (base / name).is_symlink()]
86 return sorted(result)
87
88
89def plan(manager, deadline=None):
90 moves = mappings(manager)
91 details = []
92 for source, destination in moves:
93 manager.path_for(str(source))
94 manager.path_for(str(destination), missing=True)
95 if source.is_symlink() or os.path.ismount(source) or os.path.lexists(destination):
96 raise ValueError('A source is mounted or a destination already exists: ' + str(source))
97 source_info, parent_info = source.stat(), destination.parent.stat()
98 if source_info.st_dev != parent_info.st_dev:
99 raise ValueError('Source and destination must be on the same filesystem: ' + str(source))
100 manifest = tree(source, deadline)
101 details.append({'source': str(source), 'destination': str(destination), 'device': source_info.st_dev,
102 'inode': source_info.st_ino, 'entries': len(manifest), 'tree': digest(manifest)})
103 with sqlite3.connect((manager.native / 'shares.db').as_uri() + '?mode=ro', uri=True) as database:
104 before = rows(database)
105 after = remapped(manager, before, moves)
106 return {'host': socket.gethostname(), 'service': manager.service, 'configRoot': str(manager.cfg), 'moves': details, 'shares': len(before['sh']), 'selections': len(before['sf']),
107 'affectedKeys': [old[0] for old, new in zip(before['sh'], after['sh']) if old != new],
108 'sharesBefore': digest(before), 'sharesAfter': digest(after),
109 'config': digest((manager.cfg / 'copyparty.conf').read_text()),
110 'manager': digest((manager.cfg / 'manager.json').read_text() if (manager.cfg / 'manager.json').exists() else None)}
111
112
113def write_database(database, data):
114 current = rows(database)
115 if current['sf'] != data['sf'] or len(current['sh']) != len(data['sh']) or any(
116 old[:2] + old[3:] != new[:2] + new[3:]
117 for old, new in zip(current['sh'], data['sh'])
118 ):
119 raise ValueError('Share keys, selections, passwords, or permissions changed during cutover.')
120 with database:
121 database.executemany('UPDATE sh SET vp=? WHERE k=?', [(row[2], row[0]) for row in data['sh']])
122 if rows(database) != data:
123 raise ValueError('Copyparty share paths changed while the database was being written.')
124
125
126def public_statuses(host, keys, deadline):
127 def head(key):
128 timeout = min(3, deadline - time.monotonic())
129 if timeout <= 0:
130 return key, None
131 try:
132 response = subprocess.run(
133 ['curl', '--disable', '--noproxy', '*', '--silent', '--show-error',
134 '--head', '--output', '/dev/null', '--write-out', '%{http_code}',
135 '--max-time', str(timeout), '--', 'https://' + host + '/shr/' + quote(key, safe='') + '/'],
136 capture_output=True, text=True, timeout=timeout,
137 )
138 status = response.stdout.strip()
139 return key, int(status) if response.returncode == 0 and re.fullmatch(r'[1-5][0-9]{2}', status) else None
140 except (OSError, subprocess.TimeoutExpired):
141 return key, None
142
143 with concurrent.futures.ThreadPoolExecutor(max_workers=4) as workers:
144 return dict(workers.map(head, keys))
145
146
147def wait_public(host, expected, deadline):
148 while time.monotonic() < deadline:
149 if public_statuses(host, expected, deadline) == expected:
150 return
151 time.sleep(min(0.5, max(0, deadline - time.monotonic())))
152 raise TimeoutError('Public share links did not recover before the cutover deadline.')
153
154
155def apply(manager, expected, backup_root):
156 if plan(manager) != expected:
157 raise ValueError('The cutover plan changed. Review a new plan before applying.')
158 if manager.job is None:
159 manager.job = copyparty.runs.nomad('job/' + manager.service)
160 host = (manager.job or {}).get('Meta', {}).get('studio_hostname', '')
161 if not isinstance(host, str) or not re.fullmatch(r'[a-zA-Z0-9.-]+', host):
162 raise ValueError('Copyparty has no public hostname. Check its deployed service route.')
163 manager.job['Stop'] = False
164 baseline = public_statuses(host, expected['affectedKeys'], time.monotonic() + 15)
165 if any(status is None for status in baseline.values()):
166 raise ValueError('Public shares are unavailable. Restore their current routes before cutover.')
167 backup_root.mkdir(parents=True, exist_ok=True, mode=0o700)
168 metadata = backup_root.lstat()
169 if not stat.S_ISDIR(metadata.st_mode) or metadata.st_uid != os.geteuid() or metadata.st_mode & 0o077:
170 raise ValueError('Cutover backups require an owner-only directory.')
171 backup = backup_root / (time.strftime('%Y%m%dT%H%M%SZ', time.gmtime()) + '-' + uuid.uuid4().hex[:8])
172 backup.mkdir(mode=0o700)
173 database_path = manager.native / 'shares.db'
174 saved_files = {}
175 for path in (manager.cfg / 'copyparty.conf', manager.cfg / 'manager.json'):
176 saved_files[path] = path.read_bytes() if path.exists() else None
177 if saved_files[path] is not None:
178 copyparty.atomic(backup / path.name, saved_files[path])
179 copyparty.atomic(backup / 'plan.json', json.dumps(expected, ensure_ascii=False).encode())
180 moved = []
181 before = None
182 database = None
183 manager.deadline = time.monotonic() + 285
184 mutation_deadline = manager.deadline - 205
185 restart_attempted = False
186 try:
187 manager.stop()
188 copyparty.atomic(backup / 'nomad-job.json', json.dumps(manager.job).encode())
189 if plan(manager, mutation_deadline) != expected:
190 raise ValueError('The source files or share rows changed before Copyparty stopped.')
191 database = sqlite3.connect(database_path)
192 before = rows(database)
193 with sqlite3.connect(backup / 'shares.db') as target:
194 database.backup(target)
195 (backup / 'shares.db').chmod(0o600)
196 after = remapped(manager, before, mappings(manager))
197 data_path = manager.cfg / 'manager.json'
198 data = json.loads(saved_files[data_path]) if saved_files[data_path] is not None else None
199 for source, destination in mappings(manager):
200 copyparty.rename(source, destination)
201 moved.append((source, destination))
202 write_database(database, after)
203 if data is not None:
204 for grant in data.get('grants', []):
205 original = Path(grant['path'])
206 for source, destination in mappings(manager):
207 if original.is_relative_to(source):
208 grant['path'] = str(destination / original.relative_to(source))
209 break
210 copyparty.atomic(data_path, json.dumps(data, ensure_ascii=False).encode())
211 variable = copyparty.runs.nomad('var/nomad/jobs/' + manager.service)
212 rendered = copyparty.configuration(manager.cfg, {**data, 'root': str(manager.root)}, variable['Items'])
213 info = (manager.cfg / 'copyparty.conf').stat()
214 copyparty.atomic(manager.cfg / 'copyparty.conf', rendered, (info.st_uid, info.st_gid))
215 for detail in expected['moves']:
216 destination = Path(detail['destination'])
217 if digest(tree(destination, mutation_deadline)) != detail['tree']:
218 raise ValueError('Moved folder metadata changed: ' + str(destination))
219 if digest(rows(database)) != expected['sharesAfter']:
220 raise ValueError('Share keys, selections, passwords, or permissions changed during cutover.')
221 if time.monotonic() >= mutation_deadline:
222 raise TimeoutError("Folder updates exceeded the cutover budget.")
223 restart_attempted = True
224 manager.start()
225 wait_public(host, baseline, manager.offline_at + 150)
226 result = {'result': 'moved', 'moves': len(moved), 'affectedKeys': expected['affectedKeys'], 'backup': str(backup),
227 'downtimeSeconds': time.monotonic() - manager.offline_at, 'publicStatuses': baseline}
228 copyparty.atomic(backup / 'result.json', json.dumps(result).encode())
229 return result
230 except BaseException as failure:
231 if moved or before is not None:
232 if restart_attempted:
233 manager.stop()
234 if database is None:
235 database = sqlite3.connect(database_path)
236 if before is not None:
237 write_database(database, before)
238 for source, destination in reversed(moved):
239 copyparty.rename(destination, source)
240 for path, content in saved_files.items():
241 if content is None:
242 path.unlink(missing_ok=True)
243 else:
244 owner = path.stat()
245 copyparty.atomic(path, content, (owner.st_uid, owner.st_gid))
246 if manager.job is not None:
247 manager.start()
248 wait_public(host, baseline, manager.deadline)
249 evidence = {'result': 'rolled-back', 'reason': str(failure), 'downtimeSeconds': time.monotonic() - manager.offline_at,
250 'publicStatuses': baseline}
251 copyparty.atomic(backup / 'result.json', json.dumps(evidence).encode())
252 raise
253 finally:
254 if database is not None:
255 database.close()
256
257
258def main():
259 parser = argparse.ArgumentParser(description='Review or apply the six Clover share-folder moves.')
260 parser.add_argument('--apply', action='store_true')
261 parser.add_argument('--plan', type=Path, help='Save the reviewed plan, or read it when applying.')
262 args = parser.parse_args()
263 manager = copyparty.Manager([])
264 if args.apply:
265 if args.plan is None:
266 parser.error('--apply requires --plan from the reviewed preflight')
267 lock = os.open(manager.cfg / '.manager.lock', os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW, 0o600)
268 with os.fdopen(lock, 'w') as locked:
269 copyparty.fcntl.flock(locked, copyparty.fcntl.LOCK_EX)
270 print(json.dumps(apply(manager, json.loads(args.plan.read_text()), Path('/var/lib/studio/copyparty-cutover'))))
271 else:
272 value = plan(manager)
273 if args.plan is not None:
274 copyparty.atomic(args.plan, json.dumps(value, ensure_ascii=False, indent=2).encode())
275 print(json.dumps(value, ensure_ascii=False))
276
277
278if __name__ == '__main__':
279 main()
tools/copyparty-test.py created+323
...@@ -0,0 +1,323 @@
1#!/usr/bin/env python3
2import copy
3import importlib.util
4import json
5import os
6from pathlib import Path
7import sqlite3
8import tempfile
9import unittest
10from unittest.mock import patch
11import uuid
12
13spec = importlib.util.spec_from_file_location('shares', Path(__file__).resolve().parent.parent / 'service/copyparty/manage.py')
14shares = importlib.util.module_from_spec(spec)
15spec.loader.exec_module(shares)
16
17
18class Shares(unittest.TestCase):
19 def setUp(self):
20 self.temporary = tempfile.TemporaryDirectory()
21 self.addCleanup(self.temporary.cleanup)
22 self.root = Path(self.temporary.name) / 'clover'
23 self.cfg = Path(self.temporary.name) / 'cfg'
24 self.root.mkdir()
25 self.root = self.root.resolve()
26 (self.cfg / 'copyparty').mkdir(parents=True)
27 (self.cfg / 'copyparty.conf').write_text('original configuration\n')
28 self.database = self.cfg / 'copyparty/shares.db'
29 with sqlite3.connect(self.database) as db:
30 db.executescript("CREATE TABLE kv(k text,v int); INSERT INTO kv VALUES('sver',2); CREATE TABLE sh(k text,pw text,vp text,pr text,st int,un text,t0 int,t1 int); CREATE TABLE sf(k text,vp text);")
31 self.subject = str(uuid.uuid4())
32 self.users = [{'id': self.subject, 'username': 'test-user', 'enabled': True, 'kind': None, 'requiredActions': []}]
33 self.variable = {'ModifyIndex': 1, 'Items': {'idp_header': 'a' * 32, 'unrelated': 'retained'}}
34 self.environment = patch.dict(os.environ, {'STUDIO_CLOVER_ROOT': str(self.root), 'STUDIO_COPYPARTY_CFG': str(self.cfg)})
35 self.environment.start()
36 self.addCleanup(self.environment.stop)
37 self.nomad = patch.object(shares.runs, 'nomad', side_effect=self.request)
38 self.nomad.start()
39 self.addCleanup(self.nomad.stop)
40 self.reloads = []
41 self.reload = patch.object(shares.Manager, 'reload', autospec=True, side_effect=lambda manager, items, **kwargs: self.reloads.append(copy.deepcopy(items)))
42 self.reload_mock = self.reload.start()
43 self.addCleanup(self.reload.stop)
44 self.hash = patch.object(shares.Manager, 'hash', side_effect=lambda password: '+' + 'a' * 32 if password else '')
45 self.hash.start()
46 self.container = patch.object(shares.Manager, 'container', return_value='disposable')
47 self.container.start()
48 self.addCleanup(self.container.stop)
49 self.addCleanup(self.hash.stop)
50
51 def request(self, path, method='GET', body=None, timeout=10):
52 self.assertTrue(path.startswith('var/nomad/jobs/copyparty'))
53 if method == 'GET':
54 return copy.deepcopy(self.variable)
55 self.assertEqual(method, 'PUT')
56 self.assertEqual(path.split('?cas=')[1], str(self.variable['ModifyIndex']))
57 self.variable = {'Items': copy.deepcopy(body['Items']), 'ModifyIndex': self.variable['ModifyIndex'] + 1}
58 return copy.deepcopy(self.variable)
59
60 def call(self, action, **payload):
61 return shares.handle({'action': action, 'payload': payload, 'users': self.users})
62
63 def link(self, key='public', path='clover/source', count=0, files=(), owner='snow'):
64 with sqlite3.connect(self.database) as db:
65 db.execute('INSERT INTO sh VALUES(?,?,?,?,?,?,?,?)', (key, '', path, 'rw', count, owner, 100, 0))
66 db.executemany('INSERT INTO sf VALUES(?,?)', [(key, file) for file in files])
67
68 def update(self, path, mode='retarget', key='public'):
69 return self.call('link.update', key=key, name='Clover - Example', path=str(path), permissions='rw', expires=0, password=None, mode=mode)
70
71 def offline(self):
72 def stop(manager):
73 manager.job = {'ID': 'copyparty'}
74 manager.offline_at = shares.time.monotonic()
75 manager.stopped = True
76 def start(manager):
77 manager.stopped = False
78 return patch.object(shares.Manager, 'stop', autospec=True, side_effect=stop), patch.object(shares.Manager, 'start', autospec=True, side_effect=start)
79
80 def test_account_password_is_once_only_and_nomad_preserves_other_secrets(self):
81 created = self.call('account.create', name='Local AI')
82 self.assertTrue(created['password'])
83 identifier = created['account']['id']
84 key = 'file_api_' + identifier.replace('-', '')
85 self.assertEqual(self.variable['Items'][key], '+' + 'a' * 32)
86 self.assertEqual(self.variable['Items']['unrelated'], 'retained')
87 state = self.call('state')
88 self.assertNotIn('password', state)
89 self.assertNotIn(created['password'], (self.cfg / 'manager.json').read_text())
90 self.assertNotIn(created['password'], (self.cfg / 'copyparty.conf').read_text())
91 self.assertIn('w: @acct,-api-' + identifier, (self.cfg / 'copyparty.conf').read_text())
92 self.call('account.delete', id=identifier)
93 self.assertNotIn(key, self.variable['Items'])
94
95 def test_rotation_revokes_existing_copyparty_session(self):
96 created = self.call('account.create', name='Local AI')
97 identifier = created['account']['id']
98 session_path = self.cfg / 'copyparty/sessions.db'
99 with sqlite3.connect(session_path) as db:
100 db.executescript('CREATE TABLE us(un text,si text,t0 int);')
101 db.execute('INSERT INTO us VALUES(?,?,?)', ('api-' + identifier, 'old-cookie', 100))
102 db.execute('INSERT INTO us VALUES(?,?,?)', ('snow', 'retained-cookie', 100))
103 self.call('account.rotate', id=identifier)
104 with sqlite3.connect(session_path) as db:
105 self.assertEqual(db.execute('SELECT un,si FROM us').fetchall(), [('snow', 'retained-cookie')])
106
107 def test_human_grant_uses_uuid_group_and_preserves_ancestor_access(self):
108 folder = self.root / 'work'
109 child = folder / 'child'
110 child.mkdir(parents=True)
111 first = self.call('grant.save', id=None, subject='user:' + self.subject, path=str(folder), permissions='rw')
112 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(child), permissions='r')
113 config = (self.cfg / 'copyparty.conf').read_text()
114 self.assertIn('rw: @snow-user-' + self.subject, config)
115 self.assertNotIn('test-user', config)
116 self.assertIn('xvol', config)
117 child_config = config.split('[/clover/work/child]')[1]
118 self.assertIn('A: snow', child_config)
119 self.assertIn('rw: @snow-user-' + self.subject, child_config)
120 self.call('grant.delete', id=first['grants'][0]['id'])
121 config = (self.cfg / 'copyparty.conf').read_text()
122 self.assertNotIn('[/clover/work]\n', config)
123 self.assertNotIn('rw: @snow-user-' + self.subject, config)
124
125 def test_exact_base_mount_is_rendered_once(self):
126 (self.root / 'Public').mkdir()
127 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(self.root / 'Public'), permissions='d')
128 config = (self.cfg / 'copyparty.conf').read_text()
129 self.assertEqual(config.count('[/clover/Public]'), 1)
130 self.assertIn('r.: *', config)
131 self.assertIn('d: @snow-user-' + self.subject, config)
132
133 def test_untrusted_boundary_and_symlink_escape_reject_before_nomad(self):
134 folder = self.root / 'work'
135 folder.mkdir()
136 (self.root / 'escape').symlink_to(Path(self.temporary.name))
137 cases = [dict(id=None, subject='user:' + self.subject, path=str(self.root / 'escape'), permissions='r'),
138 dict(id=None, subject='user:' + self.subject, path=str(self.root / '../outside'), permissions='r'),
139 dict(id=None, subject='user:' + self.subject, path=str(folder), permissions='x'),
140 dict(id=None, subject='user:' + str(uuid.uuid4()), path=str(folder), permissions='r')]
141 for payload in cases:
142 with self.subTest(payload=payload), self.assertRaises(shares.Error):
143 self.call('grant.save', **payload)
144 with self.assertRaises(shares.Error):
145 self.call('state', extra=True)
146 for user in [dict(self.users[0], enabled=False), dict(self.users[0], kind='ai'), dict(self.users[0], kind='guest')]:
147 self.users = [user]
148 with self.assertRaises(shares.Error):
149 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(folder), permissions='r')
150
151 def test_pending_invited_user_can_receive_permissions_before_setup(self):
152 folder = self.root / 'invited-work'
153 folder.mkdir()
154 self.users[0]['requiredActions'] = ['SETUP', 'UPDATE_PASSWORD']
155 state = self.call('grant.save', id=None, subject='user:' + self.subject, path=str(folder), permissions='rw')
156 self.assertEqual(state['grants'][0]['subject'], 'user:' + self.subject)
157 self.assertEqual(state['grants'][0]['path'], str(folder))
158 self.assertIn('rw: @snow-user-' + self.subject, (self.cfg / 'copyparty.conf').read_text())
159
160 def test_retarget_preserves_selected_files_and_rejects_missing_selection(self):
161 destination = self.root / 'destination'
162 destination.mkdir()
163 (destination / 'kept.txt').write_text('disposable')
164 self.link(count=1, files=['kept.txt'])
165 self.update(destination)
166 state = self.call('state')
167 self.assertEqual(state['links'][0]['files'], ['kept.txt'])
168 self.assertEqual(state['links'][0]['path'], str(destination))
169 missing = self.root / 'missing'
170 missing.mkdir()
171 with self.assertRaises(shares.Error):
172 self.update(missing)
173 self.assertEqual(self.call('state')['links'][0]['path'], str(destination))
174
175 def test_default_folder_create_and_link_delete_keep_data(self):
176 result = self.call('link.create', name='Clover - Example', path='', permissions='rw', expires=0, password='', createFolder=True)
177 path = self.root / 'Share/Clover - Example'
178 self.assertTrue(path.is_dir())
179 self.call('link.delete', key=result['links'][0]['key'])
180 self.assertTrue(path.is_dir())
181 self.assertEqual(self.call('state')['links'], [])
182
183 def test_move_preserves_key_selection_nested_links_and_grants(self):
184 source = self.root / 'source'
185 (source / 'nested').mkdir(parents=True)
186 (source / 'kept.txt').write_text('disposable')
187 destination = self.root / 'destination'
188 self.link(count=1, files=['kept.txt'])
189 self.link(key='nested', path='clover/source/nested')
190 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(source / 'nested'), permissions='rw')
191 stop, start = self.offline()
192 with stop as stopped, start as started:
193 self.update(destination, mode='move')
194 stopped.assert_called_once()
195 started.assert_called_once()
196 self.assertFalse(source.exists())
197 self.assertEqual((destination / 'kept.txt').read_text(), 'disposable')
198 state = self.call('state')
199 self.assertEqual({link['key']: link['path'] for link in state['links']}, {'public': str(destination), 'nested': str(destination / 'nested')})
200 self.assertEqual(state['grants'][0]['path'], str(destination / 'nested'))
201 self.assertEqual(next(link for link in state['links'] if link['key'] == 'public')['files'], ['kept.txt'])
202
203 def test_move_rejects_existing_destination_and_ancestor_selection(self):
204 source = self.root / 'source'
205 source.mkdir()
206 destination = self.root / 'destination'
207 destination.mkdir()
208 self.link()
209 with self.assertRaises(shares.Error):
210 self.update(destination, mode='move')
211 destination.rmdir()
212 self.link(key='ancestor', path='clover', count=0)
213 with self.assertRaises(shares.Error):
214 self.update(destination, mode='move')
215 self.assertTrue(source.is_dir())
216 self.assertFalse(destination.exists())
217
218 def test_reload_failure_restores_config_state_database_and_secret(self):
219 original = (self.cfg / 'copyparty.conf').read_bytes()
220 calls = 0
221 def reload(manager, items):
222 nonlocal calls
223 calls += 1
224 if calls == 1:
225 raise shares.Error(502, 'test rejection')
226 self.reload_mock.side_effect = reload
227 with self.assertRaises(shares.Error):
228 self.call('account.create', name='Local AI')
229 self.reload_mock.side_effect = lambda manager, items, **kwargs: self.reloads.append(copy.deepcopy(items))
230 self.assertEqual((self.cfg / 'copyparty.conf').read_bytes(), original)
231 self.assertEqual(self.call('state')['accounts'], [])
232 self.assertEqual(self.variable['Items'], {'idp_header': 'a' * 32, 'unrelated': 'retained'})
233 self.assertEqual(calls, 2)
234 destination = self.root / 'target'
235 destination.mkdir()
236 self.link()
237 self.reload_mock.side_effect = [shares.Error(502, 'test rejection'), None]
238 with self.assertRaises(shares.Error):
239 self.update(destination)
240 self.assertEqual(self.call('state')['links'][0]['path'], str(self.root / 'source'))
241
242 def test_failed_offline_restart_restores_folder_and_mappings(self):
243 source = self.root / 'source'
244 source.mkdir()
245 destination = self.root / 'destination'
246 self.link()
247 stop, start = self.offline()
248 with stop as stopped, patch.object(shares.Manager, 'start', autospec=True, side_effect=[shares.Error(502, 'test restart'), None]) as started:
249 with self.assertRaises(shares.Error):
250 self.update(destination, mode='move')
251 self.assertEqual(stopped.call_count, 2)
252 self.assertEqual(started.call_count, 2)
253 self.assertTrue(source.is_dir())
254 self.assertFalse(destination.exists())
255 self.assertEqual(self.call('state')['links'][0]['path'], str(source))
256
257 def test_new_folders_work_under_host_private_umask(self):
258 previous = os.umask(0o077)
259 try:
260 self.call('link.create', name='Clover - Restricted', path=str(self.root / 'Share/Clover - Restricted'),
261 permissions='r', expires=0, password='', createFolder=True)
262 finally:
263 os.umask(previous)
264 self.assertEqual((self.root / 'Share').stat().st_mode & 0o777, 0o775)
265 self.assertEqual((self.root / 'Share/Clover - Restricted').stat().st_mode & 0o777, 0o775)
266
267 def test_api_scope_excludes_wildcards_but_explicit_child_grant_works(self):
268 account = self.call('account.create', name='Scoped')['account']
269 (self.root / 'Public').mkdir()
270 self.call('grant.save', id=None, subject=account['subject'], path=str(self.root), permissions='r')
271 config = (self.cfg / 'copyparty.conf').read_text()
272 public = config.split('[/clover/Public]')[1].split('[/logs]')[0]
273 self.assertIn('r: *\n', public)
274 self.assertIn('w: @acct,-' + account['username'], public)
275 self.assertIn('r: ' + account['username'], public)
276 logs = config.split('[/logs]')[1]
277 self.assertIn('g: *,-' + account['username'], logs)
278 manager = shares.Manager(self.users)
279 manager.data = json.loads((self.cfg / 'manager.json').read_text())
280 manager.items = self.variable['Items']
281 manager.owner_permissions(account['username'], self.root / 'Public', 'r')
282 with self.assertRaises(shares.Error):
283 manager.owner_permissions(account['username'], self.root / 'Public', 'w')
284
285 def test_admin_can_retarget_other_owner_with_destination_permissions(self):
286 destination = self.root / 'other'
287 destination.mkdir()
288 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(destination), permissions='rw')
289 self.link(owner='test-user')
290 self.update(destination)
291 self.assertEqual(self.call('state')['links'][0]['owner'], 'test-user')
292 denied = self.root / 'denied'
293 denied.mkdir()
294 with self.assertRaises(shares.Error):
295 self.update(denied)
296 self.assertEqual(self.call('state')['links'][0]['path'], str(destination))
297
298 def test_stage_rebases_grants_without_changing_native_virtual_sources(self):
299 folder = self.root / 'work'
300 folder.mkdir()
301 self.call('grant.save', id=None, subject='user:' + self.subject, path=str(folder), permissions='r')
302 self.link()
303 clone = self.root.parent / 'clover-clone'
304 clone.mkdir()
305 (clone / 'work').mkdir()
306 with patch.dict(os.environ, {'STUDIO_CLOVER_ROOT': str(clone)}):
307 state = self.call('state')
308 self.assertEqual(state['grants'][0]['path'], str(clone / 'work'))
309 self.assertEqual(state['links'][0]['path'], str(clone / 'source'))
310
311 def test_no_clobber_rename_keeps_both_folders(self):
312 source = self.root / 'source'
313 destination = self.root / 'destination'
314 source.mkdir()
315 destination.mkdir()
316 with self.assertRaises(shares.Error):
317 shares.rename(source, destination)
318 self.assertTrue(source.is_dir())
319 self.assertTrue(destination.is_dir())
320
321
322if __name__ == '__main__':
323 unittest.main()
tools/copyparty.md created+49
...@@ -0,0 +1,49 @@
1# Copyparty administration and folder cutover
2
3Snowglobe's admin Shares page manages native public links and separate API
4accounts. A person's page assigns folder permissions. Existing Copyparty
5permissions remain additive; changing a folder is an explicit retarget or move
6action. New shared folders default to `Share/Name - Description` under Clover.
7
8Copyparty owns its native `cfg/copyparty/shares.db` and selected-file records.
9`cfg/manager.json` stores API account labels, UUID-based folder grants, and link
10labels. API password hashes live in the service's Nomad variable under
11`file_api_<UUID without hyphens>`. Restore these together; restoring only the
12config file cannot recover managed accounts. `prepare.py` regenerates config
13from the canonical service policy and managed state on every deployment.
14
15## Six-folder production move
16
17`copyparty-cutover.py` contains the requested six source/destination pairs.
18It requires a reviewed plan, absent destinations, and same-filesystem renames.
19The plan binds the host, service, roots, source metadata, and complete native
20share projection. No file contents are read. Public HEAD responses are captured
21before stopping the service and verified after restart; existing keys,
22passwords, permissions, expirations, and selected-file records must agree.
23
24Run the script from the reviewed immutable release on the confirmed target:
25
26```sh
27umask 077
28python3 tools/copyparty-cutover.py --plan /root/copyparty-cutover-plan.json
29python3 tools/copyparty-cutover.py --apply --plan /root/copyparty-cutover-plan.json
30```
31
32Copyparty remains stopped through all folder and database changes. A mismatch
33restores the old paths and records before restarting the frozen original job.
34Mutation verification has an 80-second cutoff; public readiness has a
35150-second forward deadline, leaving recovery time within the 285-second
36overall budget. Restricted backups and result evidence live under
37`/var/lib/studio/copyparty-cutover`. Code rollback does not undo this file move.
38
39## 2026-10-16 staging evidence
40
41Zenith preview `copyparty-preview-9909de81` verified API account scope,
42permission changes, password/session revocation, signed UUID identity grants,
43symlink confinement, selected-file retention, and nested public links. A
44disposable offline move completed in 6.94 seconds. The six requested clone
45folders were then renamed with unchanged inode/metadata manifests: all 20
46share records and four selected-file rows matched the expected path-only
47projection, and all nine affected public URLs returned their original 200
48responses within a 78.2-second upper bound. Production source metadata and
49share records remained unchanged during rehearsal.
tools/dashboard-auth-test.py+11-1
...@@ -119,6 +119,7 @@ def main():...@@ -119,6 +119,7 @@ def main():
119 assert picture_headers['content-type'] == 'image/png'119 assert picture_headers['content-type'] == 'image/png'
120 request('/api/account/pictures/missing', status=404)120 request('/api/account/pictures/missing', status=404)
121 request('/api/me', status=401, extra={'User-Name': name, 'User-Groups': 'infra-admin'})121 request('/api/me', status=401, extra={'User-Name': name, 'User-Groups': 'infra-admin'})
122 request('/api/shares', status=401, extra={'User-Name': name, 'User-Groups': 'infra-admin'})
122 request('/auth/status', status=403, extra={'Studio-Proxy-Token': 'wrong'})123 request('/auth/status', status=403, extra={'Studio-Proxy-Token': 'wrong'})
123 csrf = request('/auth/status', cookies=cookies)['csrf']124 csrf = request('/auth/status', cookies=cookies)['csrf']
124 csrf_headers = request('/auth/status', cookies=cookies, include_headers=True)125 csrf_headers = request('/auth/status', cookies=cookies, include_headers=True)
...@@ -142,6 +143,11 @@ def main():...@@ -142,6 +143,11 @@ def main():
142 assert 'Max-Age=2592000' in fields['set-cookie']143 assert 'Max-Age=2592000' in fields['set-cookie']
143 assert 'admin' in request('/api/me', cookies=cookies)['sections']144 assert 'admin' in request('/api/me', cookies=cookies)['sections']
144 request('/api/users', 'POST', {}, cookies, 403, {'Origin': 'https://evil.example'})145 request('/api/users', 'POST', {}, cookies, 403, {'Origin': 'https://evil.example'})
146 request('/api/shares/accounts', 'POST', {'name': 'Agent'}, cookies, 403, {'Origin': 'https://evil.example'})
147 preview_cookies = {**cookies, 'view-as': 'media-manage'}
148 request('/api/shares', cookies=preview_cookies, status=403)
149 request('/api/shares/links', 'POST', {}, preview_cookies, 403)
150 request('/api/users', 'POST', {'profile': {'username': 'API-agent', 'email': '', 'firstName': '', 'lastName': ''}, 'groups': [], 'setup': {'kind': 'invite'}}, cookies, 400)
145 assert request('/auth/password', 'POST', {**login, 'next': '//evil.example'}, cookies)['next'] == '/'151 assert request('/auth/password', 'POST', {**login, 'next': '//evil.example'}, cookies)['next'] == '/'
146 # Import's password format must verify with the original Keycloak parameters.152 # Import's password format must verify with the original Keycloak parameters.
147 with sqlite3.connect(data / 'accounts.sqlite') as db:153 with sqlite3.connect(data / 'accounts.sqlite') as db:
...@@ -196,7 +202,9 @@ def main():...@@ -196,7 +202,9 @@ def main():
196 finished = request('/auth/file/callback?' + callback.split('?', 1)[1], cookies=file_cookies, status=302, host=file)202 finished = request('/auth/file/callback?' + callback.split('?', 1)[1], cookies=file_cookies, status=302, host=file)
197 assert finished['location'] == file + '/clover/Public/'203 assert finished['location'] == file + '/clover/Public/'
198 request('/auth/file/callback?' + callback.split('?', 1)[1], cookies=file_cookies, status=403, host=file)204 request('/auth/file/callback?' + callback.split('?', 1)[1], cookies=file_cookies, status=403, host=file)
199 request('/auth/file/check', cookies=file_cookies, status=204, host=file)205 file_identity = request('/auth/file/check', cookies=file_cookies, status=204, host=file)
206 assert file_identity['x-auth-request-preferred-username'] == name
207 assert 'snow-user-' + actor in file_identity['x-auth-request-groups'].split(',')
200 request('/auth/file/check', cookies=cookies, status=401, host=file)208 request('/auth/file/check', cookies=cookies, status=401, host=file)
201 request('/api/me', cookies=file_cookies, status=401)209 request('/api/me', cookies=file_cookies, status=401)
202 request('/auth/file/sign-in?rd=https://evil.example/', status=400, host=file)210 request('/auth/file/sign-in?rd=https://evil.example/', status=400, host=file)
...@@ -206,6 +214,8 @@ def main():...@@ -206,6 +214,8 @@ def main():
206 request('/auth/setup', 'POST', {'csrf': new_csrf, 'setup': setup, 'email': 'new@example.invalid', 'password': 'another-password'}, newcomer)214 request('/auth/setup', 'POST', {'csrf': new_csrf, 'setup': setup, 'email': 'new@example.invalid', 'password': 'another-password'}, newcomer)
207 request('/auth/setup', 'POST', {'csrf': new_csrf, 'setup': setup, 'email': 'new@example.invalid', 'password': 'another-password'}, newcomer, 410)215 request('/auth/setup', 'POST', {'csrf': new_csrf, 'setup': setup, 'email': 'new@example.invalid', 'password': 'another-password'}, newcomer, 410)
208 request('/api/users', cookies=newcomer, status=403)216 request('/api/users', cookies=newcomer, status=403)
217 request('/api/shares', cookies=newcomer, status=403)
218 request('/api/shares/grants', 'POST', {}, newcomer, 403)
209 key = {'publicKey': Path('config/admin.pub').read_text().strip(), 'label': 'identity fixture'}219 key = {'publicKey': Path('config/admin.pub').read_text().strip(), 'label': 'identity fixture'}
210 request('/api/account/signing-keys', 'POST', key, newcomer, 204)220 request('/api/account/signing-keys', 'POST', key, newcomer, 204)
211 pending_key = request('/api/account', cookies=newcomer)['commitIdentity']['signingKeys'][0]221 pending_key = request('/api/account', cookies=newcomer)['commitIdentity']['signingKeys'][0]
tools/dashboard-host-test.py+14
...@@ -78,6 +78,20 @@ class Boundary(unittest.TestCase):...@@ -78,6 +78,20 @@ class Boundary(unittest.TestCase):
78 self.host.handle(request)78 self.host.handle(request)
79 command.assert_not_called()79 command.assert_not_called()
8080
81 def test_share_management_requires_only_its_declared_context(self):
82 request = {"operation": "copyparty.manage", "action": "state", "payload": {}, "users": []}
83 with patch.object(host_module.copyparty, "handle", return_value={"links": []}) as handle:
84 self.assertEqual(self.host.handle(request), {"links": []})
85 handle.assert_called_once_with(request)
86 for changed in [
87 {key: value for key, value in request.items() if key != "users"},
88 {**request, "root": "/srv/private"},
89 {**request, "env": {"NOMAD_TOKEN": "test"}},
90 ]:
91 with self.assertRaises(host_module.Rejected):
92 self.host.handle(changed)
93 self.assertEqual(handle.call_count, 1)
94
81 def test_reclaim_is_always_a_dry_run(self):95 def test_reclaim_is_always_a_dry_run(self):
82 with patch.object(host_module, "command", return_value="reclaim\t1024") as command:96 with patch.object(host_module, "command", return_value="reclaim\t1024") as command:
83 self.host.handle({"operation": "storage.reclaim", "dataset": "studio-demo/a",97 self.host.handle({"operation": "storage.reclaim", "dataset": "studio-demo/a",
tools/dashboard-host.py+9-1
...@@ -2,7 +2,7 @@...@@ -2,7 +2,7 @@
2import concurrent.futures2import concurrent.futures
3import contextlib3import contextlib
4import fcntl4import fcntl
5import importlib5import importlib.util
6import json6import json
7import math7import math
8import os8import os
...@@ -21,6 +21,11 @@ import time...@@ -21,6 +21,11 @@ import time
21import vms21import vms
22import tty22import tty
2323
24spec = importlib.util.spec_from_file_location("copyparty", Path(__file__).resolve().parent.parent / "service/copyparty/manage.py")
25copyparty = importlib.util.module_from_spec(spec)
26spec.loader.exec_module(copyparty)
27
28
24dashboard_runs = importlib.import_module("dashboard-run")29dashboard_runs = importlib.import_module("dashboard-run")
25shale_labels = importlib.import_module("shale-labels")30shale_labels = importlib.import_module("shale-labels")
2631
...@@ -90,6 +95,7 @@ class Host:...@@ -90,6 +95,7 @@ class Host:
90 "shale.write": {"grant", "repository", "action", "arguments"},95 "shale.write": {"grant", "repository", "action", "arguments"},
91 "host.sample": set(),96 "host.sample": set(),
92 "host.usage": {"refresh"},97 "host.usage": {"refresh"},
98 "copyparty.manage": {"action", "payload", "users"},
93 **dashboard_runs.FIELDS,99 **dashboard_runs.FIELDS,
94 **{"vm." + action: set() if fields is None else {"payload"} for action, fields in vms.ACTION_FIELDS.items()},100 **{"vm." + action: set() if fields is None else {"payload"} for action, fields in vms.ACTION_FIELDS.items()},
95 }101 }
...@@ -97,6 +103,8 @@ class Host:...@@ -97,6 +103,8 @@ class Host:
97 raise Rejected("Choose a supported host operation.")103 raise Rejected("Choose a supported host operation.")
98 if set(request) != parameters[operation] | {"operation"}:104 if set(request) != parameters[operation] | {"operation"}:
99 raise Rejected("Use only the fields required by this host operation.")105 raise Rejected("Use only the fields required by this host operation.")
106 if operation == "copyparty.manage":
107 return copyparty.handle(request)
100 if operation == "shale.metadata":108 if operation == "shale.metadata":
101 return shale_labels.metadata(request)109 return shale_labels.metadata(request)
102 if operation == "shale.write":110 if operation == "shale.write":
tools/dashboard-run.py+2-2
...@@ -74,7 +74,7 @@ def service(target, key=None):...@@ -74,7 +74,7 @@ def service(target, key=None):
74 raise Error(400, "Choose a secret from this service's list.")74 raise Error(400, "Choose a secret from this service's list.")
7575
7676
77def nomad(path, method="GET", body=None):77def nomad(path, method="GET", body=None, timeout=10):
78 token = read(release.STATE / "nomad.token", missing="").strip()78 token = read(release.STATE / "nomad.token", missing="").strip()
79 if not token:79 if not token:
80 raise Error(503, "Nomad credentials are unavailable. Check the host service configuration.")80 raise Error(503, "Nomad credentials are unavailable. Check the host service configuration.")
...@@ -82,7 +82,7 @@ def nomad(path, method="GET", body=None):...@@ -82,7 +82,7 @@ def nomad(path, method="GET", body=None):
82 data=json.dumps(body).encode() if body is not None else None,82 data=json.dumps(body).encode() if body is not None else None,
83 headers={"X-Nomad-Token": token, "Content-Type": "application/json"})83 headers={"X-Nomad-Token": token, "Content-Type": "application/json"})
84 try:84 try:
85 with urllib.request.urlopen(request, timeout=10) as response:85 with urllib.request.urlopen(request, timeout=timeout) as response:
86 data = response.read(MAX_LOG + 1)86 data = response.read(MAX_LOG + 1)
87 except urllib.error.HTTPError as error:87 except urllib.error.HTTPError as error:
88 if error.code == 404 and method == "GET":88 if error.code == 404 and method == "GET":