| 1 | # Snow Globe |
| 2 | |
| 3 | Snow Globe is Clover's physical home computer: NixOS, Nomad/Podman services, |
| 4 | ZFS storage, and virtual machines. Pkl describes services; the Rust/Solid |
| 5 | web app manages the system. Availability and existing data are part of correctness. |
| 6 | |
| 7 | ## Default workflow |
| 8 | |
| 9 | Finish a requested change through verification without another permission round: |
| 10 | |
| 11 | 1. Inspect the current code, Jujutsu state, and relevant service/runbook. |
| 12 | 2. Implement the change and run the checks that cover it. |
| 13 | 3. Automatically deploy the working changes to staging on Snow Globe: |
| 14 | `python3 tools/deploy.py stage <service>` or `stage dashboard`. |
| 15 | 4. Test the real preview: app behavior, authentication, persistence, dependencies, |
| 16 | logs/restarts, and effects on other services. For UI, use the normal browser |
| 17 | panel, check the keyboard, and capture screenshots or recordings. |
| 18 | 5. Present the working preview URL and evidence. Keep it available for review. |
| 19 | Wait for Clover to request production promotion. |
| 20 | |
| 21 | A request to change, fix, test, stage, or commit does not authorize production. |
| 22 | Once Clover requests production, finish the normal deployment without asking |
| 23 | again: put the reviewed changes on a described, conflict-free **main**, run |
| 24 | `python3 tools/deploy.py prod`, and verify the actual production paths and history. |
| 25 | Production exports **main**; it never promotes a preview database or the working |
| 26 | copy. Do not move main or push merely because a local commit was requested. |
| 27 | |
| 28 | Two surfaces use the real machine during iteration: |
| 29 | |
| 30 | - **Local AI:** deploy and test changes directly in production without waiting |
| 31 | for a separate promotion request. The physical GPU is required; do not start a |
| 32 | competing preview model. Use disposable prompts, credentials, and workspaces. |
| 33 | - **Virtual machines:** iterate on Snow Globe using disposable guests and disks. |
| 34 | Test actual guest/host integration and consoles. Preserve active personal |
| 35 | guests; a VM task does not authorize unrelated guest or storage changes. |
| 36 | |
| 37 | Other service/dashboard changes follow staging first. Host/OS changes that cannot |
| 38 | be exercised in a service preview need isolated host testing; see |
| 39 | [host rehearsal](tools/rehearsal.md). Do not substitute an isolated test for |
| 40 | production verification after an authorized promotion. |
| 41 | |
| 42 | ## Target and state |
| 43 | |
| 44 | The deployment CLI defaults to Snow Globe, `root@10.0.0.1`, SSH port `22`. |
| 45 | `SNOWGLOBE_DEPLOY_HOST` and `SNOWGLOBE_DEPLOY_PORT` override it; set both when targeting |
| 46 | another machine. Confirm the connected host before remote mutations. |
| 47 | The configured hostname is `snowglobe`, with pretty name `snow globe`. |
| 48 | |
| 49 | | Path | Purpose | |
| 50 | | --- | --- | |
| 51 | | `nixos/snowglobe.nix`, `config/site.pkl` | Host and site configuration | |
| 52 | | `/opt/snowglobe/releases/<digest>` | Immutable uploaded releases | |
| 53 | | `/opt/snowglobe/main`, `/opt/snowglobe/current` | Published main and active release links | |
| 54 | | `/etc/nixos` | Installed host configuration, managed by release activation | |
| 55 | | `/srv/prod/<service>` | Persistent service ZFS datasets | |
| 56 | | `/srv/staging/<stage>` | Disposable preview data and external mount clones | |
| 57 | | `/srv/clover` (`/Volumes/clover` on Mac) | Live personal files | |
| 58 | | `/srv/clover/Media` | Separate media dataset | |
| 59 | | `/srv/vm` | Guest disks, access files, installation library | |
| 60 | | `/var/lib/snowglobe` | Identities, previews, history, backups, dashboard state, logs | |
| 61 | |
| 62 | Preserve persistent state, numeric owners, and existing consumers through |
| 63 | namespace changes. Read current configuration and host state before changing |
| 64 | operational assumptions. |
| 65 | |
| 66 | ## Data and credentials |
| 67 | |
| 68 | - Infrastructure access and directory listings do not authorize reading private |
| 69 | contents. Get authorization per source; keep private material and derived model |
| 70 | output inside its store. Preview clones do not broaden that authorization. |
| 71 | - Keep credentials and encryption keys out of chat, command arguments, logs, and |
| 72 | tracked files. Use private files/stdin. Nomad variables own service secrets; |
| 73 | configuration declares references. Restrict private files to their consumers. |
| 74 | - Preserve assigned numeric owners in `/var/lib/snowglobe/identities.json`, seeded |
| 75 | from `config/identities.json`. Never renumber services or recursively chown data |
| 76 | as a shortcut. |
| 77 | - Before destructive storage or OS work, verify key recovery and application/ |
| 78 | database restores with at least two restricted copies off the server, stored |
| 79 | separately. Snapshots or a key archive alone do not establish recovery. |
| 80 | - Use the release workflow for activation and rollback, not ad hoc replacement |
| 81 | containers or edits inside `/opt/snowglobe/current`. Code rollback does not restore |
| 82 | data. Read [data restore](tools/data-restore.md) before proposing data loss. |
| 83 | |
| 84 | ## Implementation and checks |
| 85 | |
| 86 | - `config/Service.pkl` owns the schema; `config/site.pkl` owns site settings. |
| 87 | Services are discovered from `service/*/*.pkl`. Declare routes, mounts, health, |
| 88 | authentication, resources, and provider requirements in the service definition. |
| 89 | - `tools/snowglobe.py` owns rendering/provisioning; `tools/deploy.py` uploads; |
| 90 | `tools/release.py` activates; `tools/data.py` backs up/restores; |
| 91 | `tools/router.py` derives Caddy routes. Change sources, not generated jobs/routes. |
| 92 | - `nixos/` owns the host. `dashboard/src/` and `dashboard/web/` own the app; |
| 93 | `tools/dashboard-host.py` and `tools/vms.py` own privileged operations. |
| 94 | Read `guest/DESIGN.md` before changing the guest/host boundary. |
| 95 | - Review outbound mail, federation, downloads, automation, fixed ports, host |
| 96 | networking, and GPU use before staging. Use preview-aware settings or fresh |
| 97 | isolation. Fresh previews reject writable external mounts. Read-only mounts |
| 98 | can still expose their source; inspect actual bindings. |
| 99 | - Repeating `stage <service>` updates its preview and retains preview data. Use |
| 100 | the printed hostname. Clean up superseded previews with |
| 101 | `python3 tools/deploy.py destroy <stage-id>`; retain evidence first. |
| 102 | - Dashboard: `cd dashboard && npm run check && npm run build` and |
| 103 | `cargo test --manifest-path dashboard/Cargo.toml`. Deployment: |
| 104 | `python3 tools/deploy-test.py`; target-side `tools/snowglobe.py validate` checks |
| 105 | jobs and `check` checks routes. Rendering can allocate identities. |
| 106 | - Read a test's entry point before running it. Host/service/storage/guest mutation |
| 107 | tests need disposable fixtures or the explicit production scope above. A health |
| 108 | response alone does not prove the app works. Metrics/traces must be observed. |
| 109 | - Prefer fewer moving parts. Derive state, remove dead code and false comments, |
| 110 | and keep one home for each artifact. When a trade depends on how likely a |
| 111 | default or interface is to change, explain the trade and ask Clover. |
| 112 | - Use Jujutsu, never standalone `git`. Inspect `jj st`, the current description, |
| 113 | and `jj diff --git`; preserve unrelated work and split independent changes. |
| 114 | Attribution uses `Assisted-by: <model>`, never `Co-authored-by`. |
| 115 | |
| 116 | ## References |
| 117 | |
| 118 | [Storage layout](readme.md) · [Physical host/recovery](tools/snowglobe-hardware.md) · |
| 119 | [Service dependencies](tools/service-dependencies.md) · [Files](tools/copyparty.md) · |
| 120 | [AI steering](service/local-ai/steering/README.md) · [AI cache](tools/local-ai-cache.md) |