Files
installer/src/backup.rs
wtclaude 6c49217e9c
All checks were successful
PR Checks / rust-gates (pull_request) Successful in 1m49s
docs(backup): correct why the sidecar database is not backed up
backup.rs justified skipping the sidecar's database on two claims. Protocol 4
falsifies one and reveals the other was already wrong.

It said the database is safe because store.rs creates every table IF NOT EXISTS.
That held only while every schema change added a whole table — which, up to and
including Protocol 3.0, every one of them did. Protocol 4 adds a COLUMN to a table
that already exists, which IF NOT EXISTS cannot do, so link now carries a real
migration. A run can change the database's structure, not only its contents.

It also said every table holds state the sweeps repopulate. `events` does not: it
is never pruned, and the website backfills what it missed from GET /history on
every reconnect. So a lost database costs the gap-recovery window for whatever
happened while the site was down. That claim was untrue before this workstream
existed.

The behaviour does not change — the database is still not copied — because the
argument against backing up unbounded bulk survives both corrections: `events`
grows without limit, the migration is transactional and additive, and the website
holds its own durable copy of everything already ingested. Only the reasoning was
wrong, and a wrong reason left in place is what lets the next person extend it to
a case it never covered.

Whether that unbounded table should be pruned or protected belongs to link, on its
own merits, rather than being settled inside a backup policy.

No logic change; docs only.

Refs: docs/website/TEAMS.md Part 12 Phase 1

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:58:24 -05:00

503 lines
20 KiB
Rust

//! Copies of what a run is about to overwrite (PLAN.md §5.3).
//!
//! ## Scoped by what cannot be fetched again
//!
//! Most of what this installer writes is replaceable: the sidecar binary and every overlay file are
//! re-downloadable and hash-named in the bundle, and the sidecar's database is overwhelmingly a
//! projection of shard state that the sweeps repopulate. Backing it up would be bulk with little
//! recovery value, and the bulk is not free: it would bury the two things that matter.
//!
//! That reasoning used to be stated two ways that are no longer true, and the correction is worth
//! keeping rather than quietly deleting:
//!
//! - It said the database is safe because `store.rs` creates every table `IF NOT EXISTS`. That held
//! only while every schema change added a whole *table*. Protocol 4 adds a *column* to a table
//! that already exists, which `IF NOT EXISTS` cannot do, so `link` now carries a real migration
//! (`PRAGMA user_version` steps). A run can therefore change the database's structure, not just
//! its contents.
//! - It said every table holds state the sweeps repopulate. `events` does not: it is never pruned,
//! and the website backfills the events it missed from `GET /history` on every reconnect. So a
//! lost database costs the gap-recovery window for anything that happened while the site was down.
//!
//! The decision is unchanged — this still does not copy the database — because the argument against
//! backing up unbounded bulk survives both corrections: `events` grows without limit, the migration
//! is transactional and additive, and the website holds its own durable copy of everything it has
//! already ingested. Only the *reason* was wrong. Whether that table should be pruned or protected
//! is a question for `link`, on its own merits, not something to settle inside a backup policy.
//!
//! What a run can destroy irrecoverably is short:
//!
//! 1. **The operator's own edits to a file the overlay owns.** `Bridge.cfg` is deliberately kept
//! (PLAN.md §5 Phase 1), but every `.cs` file and `Scripts.csproj` is overwritten
//! *unconditionally and by design* — so the one place this tool knowingly discards work is the
//! one place it should keep a copy first.
//! 2. **A stock ServUO file the patch tier edits.** `patches/originals/` already holds the
//! pre-*tier* copy and is never overwritten, which is the right revert target; it is not a
//! record of what the file looked like *this morning*, after the operator's own later edits.
//! 3. **`sidecar.toml`**, whose token the website already holds. Mint a new one and the site's
//! saved configuration starts answering `401` with nothing on the sidecar to explain why.
//!
//! ## What decides whether a backup happens
//!
//! **Whether this run is about to overwrite something** — not which verb was typed and not whether
//! a prior record exists. PLAN.md §5.3 framed it as "`update`, and `install` over an existing
//! record", on the reasoning that a first install overwrites nothing. That reasoning does not
//! survive contact with `INSTALL.md` Appendix A2, which documents deploying the overlay **by hand**:
//! a first `install` over such a tree finds `.cs` files that differ, plans them as `Change`, and
//! overwrites them with no record anywhere of what was there. So the test is the direct one, and a
//! genuine first install onto a clean tree still writes nothing because there is nothing to copy.
//!
//! ## Restoring is printed, not done
//!
//! Same rule as the uninstall report, and for the same reason: the installer cannot know what has
//! changed since, and a restore that puts an old `.cs` file back over a newer overlay eats work
//! rather than saving it. The path and the manifest are what this module hands over.
use anyhow::{Context, Result};
use serde::{Deserialize, Serialize};
use std::fs;
use std::path::{Path, PathBuf};
use crate::paths::Layout;
use crate::util;
/// How many backup directories survive. Older ones are pruned as new ones are written.
///
/// An unbounded directory of ServUO source copies on a shard host is its own support problem, and
/// the value of an old backup falls off a cliff: what an operator reaches for is "before this
/// upgrade", occasionally "before the one before". Three is that, plus one.
pub const KEEP: usize = 3;
/// The `schema` written into `manifest.json`, so a future reader can tell shapes apart.
const SCHEMA: u32 = 1;
/// Why a file was copied. Recorded per entry, because "what did this upgrade touch" is answered
/// very differently by an overlay file and by a stock ServUO file the patch tier edited.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Reason {
/// An overlay file whose on-disk content the sync is about to replace.
OverlayChange,
/// A stock ServUO file the patch tier is about to edit.
PatchTarget,
/// A companion `.cs` the tier copies in, which already existed in the tree.
PatchCompanion,
/// `sidecar.toml` — the token the website holds.
SidecarConfig,
}
impl Reason {
fn as_str(self) -> &'static str {
match self {
Self::OverlayChange => "overlay-change",
Self::PatchTarget => "patch-target",
Self::PatchCompanion => "patch-companion",
Self::SidecarConfig => "sidecar-config",
}
}
/// Which sub-directory of the backup the copy lands under.
///
/// The two roots are kept apart because a path is only meaningful relative to one of them, and
/// without the split a state file could collide with a tree file of the same name.
fn root_dir(self) -> &'static str {
match self {
Self::SidecarConfig => "state",
_ => "servuo",
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Manifest {
pub schema: u32,
/// RFC 3339, when the backup was taken.
pub taken: String,
/// `install` or `update` — the verb that displaced these files.
pub command: String,
pub installer: String,
/// The bundle in the record before this run, when there was one.
#[serde(skip_serializing_if = "Option::is_none")]
pub bundle_from: Option<String>,
/// The bundle this run is moving to.
pub bundle_to: String,
pub servuo_root: String,
pub files: Vec<Entry>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct Entry {
/// Where the copy sits inside the backup directory, `/`-separated.
pub path: String,
/// Where it was copied from, absolute, as it was on this host.
pub source: String,
pub sha256: String,
pub reason: String,
}
/// One run's backup. Created up front and handed to each stage that writes.
///
/// **The directory is created lazily, on the first capture.** A run that overwrites nothing must
/// leave nothing behind — an empty dated directory per run would be indistinguishable from a
/// backup that failed to record anything, and would push real ones out of the retention window.
pub struct Session {
dir: PathBuf,
enabled: bool,
started: bool,
root: PathBuf,
layout: Layout,
command: &'static str,
bundle_from: Option<String>,
bundle_to: String,
taken: String,
entries: Vec<Entry>,
}
impl Session {
/// `enabled` is false for `--verify` (a dry run must not create state, the same rule that keeps
/// it away from `--print-config`) and for `--no-backup`.
pub fn new(
layout: &Layout,
root: &Path,
command: &'static str,
bundle_from: Option<String>,
bundle_to: String,
enabled: bool,
) -> Self {
let taken = chrono::Utc::now();
Self {
// Colons are not legal in a Windows path component, so the stamp is the basic ISO 8601
// form. It still sorts lexicographically, which is what the pruning relies on.
dir: layout
.backups_dir()
.join(taken.format("%Y%m%dT%H%M%SZ").to_string()),
enabled,
started: false,
root: root.to_path_buf(),
layout: layout.clone(),
command,
bundle_from,
bundle_to,
taken: taken.to_rfc3339_opts(chrono::SecondsFormat::Secs, true),
entries: Vec::new(),
}
}
/// True once something has actually been copied.
pub fn has_entries(&self) -> bool {
!self.entries.is_empty()
}
/// Copies `source` into this backup, if it exists and backups are enabled.
///
/// A file that does not exist is not an error and not an entry: the caller asks for anything it
/// *may* be about to overwrite, and "there was nothing there" is the common answer on a first
/// install.
pub fn capture(&mut self, source: &Path, reason: Reason) -> Result<()> {
if !self.enabled || !source.exists() {
return Ok(());
}
let rel = self.relative_to_root(source, reason);
let dest = self.dir.join(reason.root_dir()).join(&rel);
if dest.exists() {
// Two stages can name the same file — a patch target that is also a companion path, or
// a re-entrant caller. First copy wins: it is the one taken furthest from any write.
return Ok(());
}
if !self.started {
fs::create_dir_all(&self.dir).with_context(|| {
format!("cannot create the backup directory {}", self.dir.display())
})?;
self.started = true;
}
if let Some(parent) = dest.parent() {
fs::create_dir_all(parent)
.with_context(|| format!("cannot create {}", parent.display()))?;
}
fs::copy(source, &dest).with_context(|| {
format!(
"cannot back up {} before overwriting it. Re-run with --no-backup to proceed \
without a copy",
source.display()
)
})?;
self.entries.push(Entry {
path: format!(
"{}/{}",
reason.root_dir(),
rel.replace(std::path::MAIN_SEPARATOR, "/")
),
source: source.display().to_string(),
sha256: util::sha256_file(source)?,
reason: reason.as_str().to_string(),
});
Ok(())
}
/// Writes `manifest.json` and prunes older backups. Returns the directory when one was written.
///
/// The manifest is written **last**, so a directory carrying one is a complete backup. Pruning
/// only considers directories that have one, for the same reason: a run interrupted mid-copy
/// must not be able to evict a good backup by being newer than it.
pub fn finish(mut self) -> Result<Option<PathBuf>> {
if !self.started {
return Ok(None);
}
self.entries.sort_by(|a, b| a.path.cmp(&b.path));
let manifest = Manifest {
schema: SCHEMA,
taken: self.taken.clone(),
command: self.command.to_string(),
installer: env!("CARGO_PKG_VERSION").to_string(),
bundle_from: self.bundle_from.clone(),
bundle_to: self.bundle_to.clone(),
servuo_root: self.root.display().to_string(),
files: self.entries.clone(),
};
let body = serde_json::to_string_pretty(&manifest)? + "\n";
util::write_atomic(&manifest_path(&self.dir), body.as_bytes())
.context("cannot write the backup manifest")?;
prune(&self.layout, KEEP)?;
Ok(Some(self.dir.clone()))
}
/// The path a captured file takes inside the backup, relative to the root it belongs to.
fn relative_to_root(&self, source: &Path, reason: Reason) -> String {
let rel = match reason.root_dir() {
"servuo" => source.strip_prefix(&self.root).unwrap_or(source),
_ => source
.strip_prefix(&self.layout.state_dir)
.unwrap_or(source),
};
// An absolute path outside the root it was filed under would escape the backup directory
// when joined. Falling back to the file name keeps the copy inside; the manifest still
// records exactly where it came from.
if rel.is_absolute() || rel.as_os_str().is_empty() {
return source
.file_name()
.map(|n| n.to_string_lossy().to_string())
.unwrap_or_else(|| "file".to_string());
}
rel.to_string_lossy().to_string()
}
}
fn manifest_path(dir: &Path) -> PathBuf {
dir.join("manifest.json")
}
/// Every complete backup on this host, newest first.
pub fn list(layout: &Layout) -> Vec<PathBuf> {
let mut dirs: Vec<PathBuf> = match fs::read_dir(layout.backups_dir()) {
Ok(entries) => entries
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| manifest_path(p).is_file())
.collect(),
Err(_) => Vec::new(),
};
// The stamp is fixed-width and zero-padded, so lexicographic order is chronological order.
dirs.sort();
dirs.reverse();
dirs
}
/// Reads one backup's manifest.
pub fn read_manifest(dir: &Path) -> Result<Manifest> {
let body = fs::read_to_string(manifest_path(dir))
.with_context(|| format!("cannot read {}", manifest_path(dir).display()))?;
serde_json::from_str(&body)
.with_context(|| format!("{} is not a backup manifest", manifest_path(dir).display()))
}
/// Removes all but the `keep` newest complete backups.
pub fn prune(layout: &Layout, keep: usize) -> Result<()> {
for old in list(layout).into_iter().skip(keep) {
fs::remove_dir_all(&old)
.with_context(|| format!("cannot remove the old backup {}", old.display()))?;
}
Ok(())
}
/// Deletes every backup. Reached only from `uninstall --purge`, alongside the config, the database
/// and the cached patch set — they are all the same kind of thing: the only offline record of what
/// was here before.
pub fn remove_all(layout: &Layout) -> Result<()> {
let dir = layout.backups_dir();
if dir.exists() {
fs::remove_dir_all(&dir).with_context(|| format!("cannot remove {}", dir.display()))?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::util::TempDir;
fn layout_in(root: &Path) -> Layout {
let mut layout = crate::paths::layout();
layout.state_dir = root.join("state");
layout
}
fn write(path: &Path, body: &str) {
fs::create_dir_all(path.parent().unwrap()).unwrap();
fs::write(path, body).unwrap();
}
#[test]
fn a_run_that_overwrites_nothing_leaves_nothing_behind() {
let tmp = TempDir::new("backup-empty").unwrap();
let layout = layout_in(tmp.path());
let root = tmp.path().join("ServUO");
let mut session = Session::new(&layout, &root, "install", None, "2026.08.05".into(), true);
// Nothing on disk to copy — the common first-install case.
session
.capture(
&root.join("Scripts/Custom/Bridge/BridgeLink.cs"),
Reason::OverlayChange,
)
.unwrap();
assert!(session.finish().unwrap().is_none());
assert!(
!layout.backups_dir().exists(),
"an empty dated directory would be indistinguishable from a failed backup"
);
}
#[test]
fn a_captured_file_is_copied_verbatim_and_recorded() {
let tmp = TempDir::new("backup-capture").unwrap();
let layout = layout_in(tmp.path());
let root = tmp.path().join("ServUO");
let source = root.join("Scripts/Custom/Bridge/BridgeLink.cs");
write(&source, "the operator's own edit\n");
write(&layout.sidecar_config(), "[web]\nauth_token = \"secret\"\n");
let mut session = Session::new(
&layout,
&root,
"update",
Some("2026.08.04".into()),
"2026.08.05".into(),
true,
);
session.capture(&source, Reason::OverlayChange).unwrap();
session
.capture(&layout.sidecar_config(), Reason::SidecarConfig)
.unwrap();
let dir = session.finish().unwrap().expect("a backup was taken");
let copy = dir.join("servuo/Scripts/Custom/Bridge/BridgeLink.cs");
assert_eq!(
fs::read_to_string(&copy).unwrap(),
"the operator's own edit\n"
);
assert!(dir.join("state/sidecar.toml").is_file());
let manifest = read_manifest(&dir).unwrap();
assert_eq!(manifest.command, "update");
assert_eq!(manifest.bundle_from.as_deref(), Some("2026.08.04"));
assert_eq!(manifest.files.len(), 2);
let overlay = manifest
.files
.iter()
.find(|f| f.reason == "overlay-change")
.unwrap();
// The path inside the backup is always `/`-separated, so a manifest written on Windows
// reads the same as one written on Linux.
assert_eq!(overlay.path, "servuo/Scripts/Custom/Bridge/BridgeLink.cs");
assert_eq!(overlay.sha256, util::sha256_file(&source).unwrap());
assert!(overlay.source.contains("BridgeLink.cs"));
}
#[test]
fn the_first_copy_of_a_file_wins() {
// Two stages can name the same path. The earlier capture is the one taken furthest from
// any write, so a later one must not overwrite it with content that has already changed.
let tmp = TempDir::new("backup-twice").unwrap();
let layout = layout_in(tmp.path());
let root = tmp.path().join("ServUO");
let source = root.join("Server/EventSink.cs");
write(&source, "before\n");
let mut session = Session::new(&layout, &root, "install", None, "2026.08.05".into(), true);
session.capture(&source, Reason::PatchTarget).unwrap();
write(&source, "after\n");
session.capture(&source, Reason::PatchTarget).unwrap();
let dir = session.finish().unwrap().unwrap();
assert_eq!(
fs::read_to_string(dir.join("servuo/Server/EventSink.cs")).unwrap(),
"before\n"
);
assert_eq!(read_manifest(&dir).unwrap().files.len(), 1);
}
#[test]
fn disabled_sessions_write_nothing() {
let tmp = TempDir::new("backup-off").unwrap();
let layout = layout_in(tmp.path());
let root = tmp.path().join("ServUO");
let source = root.join("Scripts/Custom/Bridge/BridgeLink.cs");
write(&source, "content\n");
let mut session = Session::new(&layout, &root, "install", None, "2026.08.05".into(), false);
session.capture(&source, Reason::OverlayChange).unwrap();
assert!(session.finish().unwrap().is_none());
assert!(!layout.backups_dir().exists());
}
#[test]
fn pruning_keeps_the_newest_and_ignores_incomplete_directories() {
let tmp = TempDir::new("backup-prune").unwrap();
let layout = layout_in(tmp.path());
for stamp in ["20260801T000000Z", "20260802T000000Z", "20260803T000000Z"] {
write(
&layout.backups_dir().join(stamp).join("manifest.json"),
"{\"schema\":1}",
);
}
// A run interrupted before its manifest was written. It must neither be listed nor be able
// to evict a complete backup by being newer.
write(
&layout.backups_dir().join("20260804T000000Z/servuo/x.cs"),
"half a copy\n",
);
assert_eq!(list(&layout).len(), 3);
prune(&layout, 2).unwrap();
let kept: Vec<String> = list(&layout)
.iter()
.map(|p| p.file_name().unwrap().to_string_lossy().to_string())
.collect();
assert_eq!(kept, vec!["20260803T000000Z", "20260802T000000Z"]);
assert!(
layout.backups_dir().join("20260804T000000Z").exists(),
"an incomplete directory is left for a human to look at, not silently deleted"
);
}
#[test]
fn purge_removes_every_backup() {
let tmp = TempDir::new("backup-purge").unwrap();
let layout = layout_in(tmp.path());
write(
&layout.backups_dir().join("20260801T000000Z/manifest.json"),
"{\"schema\":1}",
);
remove_all(&layout).unwrap();
assert!(!layout.backups_dir().exists());
// Removing what is not there is not an error: `uninstall --purge` runs on hosts that never
// took a backup.
remove_all(&layout).unwrap();
}
}