Files
installer/src/doctor.rs
wtclaude 82900da939
All checks were successful
PR Checks / rust-gates (pull_request) Successful in 1m59s
feat(installer): back up what a run is about to overwrite
PLAN.md §5.3. Before anything is written, every file this run will
replace is copied into <state>/backups/<utc-stamp>/ with a manifest
naming where each came from. --no-backup opts out; --verify takes none.

Scoped by what cannot be fetched again. The sidecar binary and the
overlay files are re-downloadable and hash-named in the bundle, and the
database is a cache with a schema -- link's store.rs creates every table
IF NOT EXISTS over shard state the sweeps repopulate. What a run can
destroy for good is an operator's edits to a deployed .cs file, which
Phase 1 overwrites unconditionally and by design, and sidecar.toml,
whose token the website already holds.

Two deviations from §5.3 as written, both found by building it:

- The trigger is "this run is about to overwrite something", not "an
  update, or an install over an existing record". §5.3 justified the
  latter with "a first install overwrites nothing" -- which is not true
  of a tree deployed by hand per INSTALL.md Appendix A2, a documented
  path. There the first install finds .cs files that differ, plans them
  as Change, and overwrites them with no record anywhere. The direct
  test covers that case and still writes nothing for a genuine first
  install, because there is nothing to copy.
- sidecar.toml joins a backup that is already being taken and is never
  the reason for one. Nothing here rewrites it, so making it a trigger
  would put a dated directory on disk after every no-op update; it is
  copied so a restored set of files comes with the token that matches
  them.

The directory is created lazily and the manifest is written last, so a
directory carrying one is a complete backup -- and pruning only
considers those, so a run interrupted mid-copy cannot evict a good
backup by being newer than it. Three are kept. uninstall keeps them and
names them in its report; --purge removes them, alongside the config,
the database and the cached patch set. doctor reports the newest.

Restoring stays printed rather than done, as the uninstall report is:
the installer cannot know what has changed since, and putting an old
.cs file back over a newer overlay eats work rather than saving it.

Verified live against two scratch ServUO trees built from the real 57.4
files: a clean first install leaving no backups directory at all, an
update after editing a deployed .cs (copy holds the edit, tree gets the
release's file, manifest lists both it and sidecar.toml), a no-op update
taking none, --no-backup and --verify each taking none, a fourth backup
pruning the oldest, doctor's row, uninstall keeping three and listing
them, --purge removing them, and a --patches run capturing the
pre-patch Logging.cs while the two rung-0 patches correctly captured
nothing. fmt, clippy -D warnings and 144 tests on both Linux and
Windows.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-05 05:47:25 -05:00

917 lines
36 KiB
Rust

//! The `doctor` command — diagnose an existing deployment end to end.
//!
//! PLAN.md §5 Phase 4 calls this "the command that makes the whole thing supportable", and the row
//! that carries the phase is the last one: **has a shard actually dialed in?** Everything else can
//! be true — files copied, service running, hashes matching — while the bridge does nothing at all,
//! because ServUO shells out to `dotnet build`, prints the output, ignores the exit code and reloads
//! the previous `Scripts.dll` (§2.1). A clean boot is not evidence. `plugin_connected` is.
//!
//! Three rules shape this module:
//!
//! - **It writes nothing, anywhere.** Not to the ServUO tree, not to `install.json`, not to the
//! sidecar's config. That is why `--print-config` is run only when the config file already
//! exists: that flag *provisions* (it writes the file and mints a token when absent), so calling
//! it on a host that has none would have `doctor` create the very state it is reporting on.
//! - **It asks the thing itself, not the record.** The installed binary answers `--version` and
//! `--print-config`; the service manager answers `is-active`; the sidecar answers `/health`. The
//! record says what `install` *did*, which is a different question from what is true now — and
//! the gap between those two is the whole reason to run this.
//! - **A missing answer is a row, not an exception.** A host with no route to Gitea, a sidecar that
//! is down, a config this process cannot read: each degrades to one honest line and the report
//! still prints. The exit code is what a monitoring script reads — `1` if any row failed — and it
//! is deliberately not raised by a `⚠`, which means "worth knowing", not "broken".
//!
//! The token is never printed here. `--print-config` returns it (it is one document), and this
//! module reads the paths, the bind and the protocol out of that document and drops the rest.
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::Result;
use serde::Deserialize;
use crate::cli::Cli;
use crate::record::{InstallRecord, LinkRecord};
use crate::servuo::ServUoRoot;
use crate::{bundle, net, patch, paths, servuo, sidecar, ui};
/// Both network calls give up quickly. Every row here is context around local state, so a host with
/// no route out must produce its report seconds later rather than appear to hang.
const BUNDLE_TIMEOUT: Duration = Duration::from_secs(15);
const HEALTH_TIMEOUT: Duration = Duration::from_secs(5);
/// The verdict on one row.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Mark {
Ok,
Warn,
Fail,
}
impl Mark {
fn glyph(self) -> &'static str {
match self {
Self::Ok => "",
Self::Warn => "",
Self::Fail => "",
}
}
}
/// One line of the report, plus any detail lines that belong underneath it.
#[derive(Debug, Clone)]
pub struct Row {
pub mark: Mark,
pub label: String,
pub detail: String,
pub notes: Vec<String>,
}
impl Row {
fn new(mark: Mark, label: &str, detail: impl Into<String>) -> Self {
Self {
mark,
label: label.to_string(),
detail: detail.into(),
notes: Vec::new(),
}
}
fn ok(label: &str, detail: impl Into<String>) -> Self {
Self::new(Mark::Ok, label, detail)
}
fn warn(label: &str, detail: impl Into<String>) -> Self {
Self::new(Mark::Warn, label, detail)
}
fn fail(label: &str, detail: impl Into<String>) -> Self {
Self::new(Mark::Fail, label, detail)
}
fn note(mut self, note: impl Into<String>) -> Self {
self.notes.push(note.into());
self
}
fn notes_from(mut self, notes: impl IntoIterator<Item = String>) -> Self {
self.notes.extend(notes);
self
}
}
/// Runs every check and prints the report. The `i32` is the process exit code.
pub fn run(cli: &Cli) -> Result<i32> {
let layout = paths::layout();
let record_path = layout.install_record();
println!(
"\nRunic Gateway installer {} — doctor",
env!("CARGO_PKG_VERSION")
);
let Some(record) = InstallRecord::load(&record_path)? else {
// Not an error in the `anyhow` sense — the command ran fine and the answer is "nothing is
// installed here". Exit 1 all the same, because a monitoring script asking after a
// deployment on this host has had its question answered in the negative.
println!();
ui::warn(&format!(
"No deployment is recorded on this host.\n \
Looked for {}\n \
Run `install` first. If you installed with {} set, set it again for this run.",
record_path.display(),
paths::STATE_DIR_ENV
));
return Ok(1);
};
let mut rows = vec![Row::ok(
"Install record",
format!(
"{} (bundle {}, installer {}, {})",
record_path.display(),
record.bundle.tag,
record.installer.version,
record.updated
),
)];
// ── ServUO and the overlay ───────────────────────────────────────────────
let root = open_root(cli, &record);
rows.push(servuo_row(&record, root.as_ref()));
rows.push(overlay_row(&record, root.as_ref()));
rows.push(patch_row(&record, root.as_ref(), &layout));
// ── The sidecar ──────────────────────────────────────────────────────────
// Asking the installed binary is what makes these rows describe the sidecar that will actually
// answer the website, rather than the one the record believes was installed.
let link = record.link_record();
let live = link.as_ref().and_then(live_config);
let health = live.as_ref().and_then(|doc| health_of(&doc.web.bind));
rows.push(link_row(link.as_ref(), live.as_ref()));
rows.push(service_row(link.as_ref()));
rows.push(reachable_row(live.as_ref(), health.as_ref()));
rows.push(protocol_row(&record, live.as_ref(), health.as_ref()));
rows.push(shard_row(root.as_ref(), health.as_ref()));
// ── The bundle ───────────────────────────────────────────────────────────
rows.push(bundle_row(&record));
// ── Backups ──────────────────────────────────────────────────────────────
rows.push(backup_row(&layout));
// ── Report ───────────────────────────────────────────────────────────────
println!();
for row in &rows {
println!("{} {:<24} {}", row.mark.glyph(), row.label, row.detail);
for note in &row.notes {
println!(" {note}");
}
}
let failed = rows.iter().filter(|r| r.mark == Mark::Fail).count();
let warned = rows.iter().filter(|r| r.mark == Mark::Warn).count();
println!();
match (failed, warned) {
(0, 0) => println!("Everything checks out."),
(0, w) => println!("{w} thing(s) worth knowing about, nothing broken."),
(f, _) => println!(
"{f} check(s) failed. Start with the first ✗ above; \
INSTALL.md's Troubleshooting table is keyed to these symptoms."
),
}
Ok(if failed > 0 { 1 } else { 0 })
}
/// The ServUO root to inspect: `--servuo` if given, else the one the record names.
///
/// [`servuo::open`] rather than `open_stopped`: a running shard is the *expected* state for a
/// diagnosis — it is the only state in which the shard-connected row can be true — and refusing to
/// report on a live host would make this command useless exactly when it is needed.
fn open_root(cli: &Cli, record: &InstallRecord) -> Result<ServUoRoot> {
let path = cli
.servuo
.clone()
.unwrap_or_else(|| record.servuo.path.clone());
servuo::open(Path::new(&path))
}
fn servuo_row(record: &InstallRecord, root: Result<&ServUoRoot, &anyhow::Error>) -> Row {
let root = match root {
Ok(root) => root,
Err(error) => {
return Row::fail("ServUO found", record.servuo.path.clone())
.note(error.to_string().replace('\n', " "))
}
};
let row = Row::ok(
"ServUO found",
format!("{} ({})", root.path.display(), root.version_display()),
);
// A tree that has been upgraded under an install is the single most useful thing this row can
// say: the patch tier was resolved against the version recorded here, not against this one.
match &record.servuo.version {
Some(recorded) if Some(recorded) != root.version.as_ref() => Row::warn(
"ServUO found",
format!("{} ({})", root.path.display(), root.version_display()),
)
.note(format!(
"this tree was {recorded} when Runic Gateway was installed — re-check the patch tier \
below"
)),
_ if !root.is_supported_version() => row.note(format!(
"{} is the only supported version; the base overlay is expected to work anyway",
servuo::SUPPORTED_VERSION
)),
_ => row,
}
}
/// Compares every deployed file against the record.
///
/// The comparison that matters is *which* hash a file differs from (PLAN.md §7.0): differing from
/// what the installer put there means the operator edited it, while agreeing with the record on a
/// host whose bundle has moved on means the overlay upstream is newer. A file recorded as
/// `kept-operator-modified` is theirs by definition, so a further edit there is not a finding.
fn overlay_row(record: &InstallRecord, root: Result<&ServUoRoot, &anyhow::Error>) -> Row {
let Some(overlay) = &record.overlay else {
return Row::fail("Overlay in sync", "no overlay recorded in install.json");
};
let Ok(root) = root else {
return Row::fail(
"Overlay in sync",
format!("{} files recorded, tree unreadable", overlay.files.len()),
);
};
let mut missing = Vec::new();
let mut edited = Vec::new();
let mut operator_owned = 0usize;
for (rel, file) in &overlay.files {
if file.state == "kept-operator-modified" {
operator_owned += 1;
continue;
}
let path = patch::join(&root.path, rel);
match crate::util::sha256_file(&path) {
Ok(actual) if actual == file.on_disk_sha256 => {}
Ok(_) => edited.push(rel.clone()),
Err(_) => missing.push(rel.clone()),
}
}
let total = overlay.files.len();
let owned = if operator_owned > 0 {
format!(", {operator_owned} operator-owned")
} else {
String::new()
};
if missing.is_empty() && edited.is_empty() {
return Row::ok(
"Overlay in sync",
format!("{total} files, all hashes match install.json{owned}"),
);
}
let mark = if missing.is_empty() {
Mark::Warn
} else {
Mark::Fail
};
let mut row = Row::new(
mark,
"Overlay in sync",
format!(
"{total} files{owned}{} missing, {} edited since deployment",
missing.len(),
edited.len()
),
)
.notes_from(missing.iter().map(|f| format!("missing: {f}")))
.notes_from(edited.iter().map(|f| format!("edited: {f}")));
if !missing.is_empty() {
row = row.note("run `update` (or `install`) to put the release's copies back");
}
if !edited.is_empty() {
row = row.note(
"these are code files the overlay owns — an `update` overwrites them without asking",
);
}
row
}
/// Reports the patch tier from the record, then checks the tree still agrees with it.
///
/// The check is not decoration. The tier's whole risk is that its edits sit inside files ServUO
/// itself ships, so a core upgrade, a hand revert, or a restored backup silently removes them —
/// and nothing else in this report would notice. The cached `.patch` (PLAN.md §2.2) is what makes
/// the check possible offline: resolving it against the current file must land on rung 0, because
/// the record says it is already applied.
fn patch_row(
record: &InstallRecord,
root: Result<&ServUoRoot, &anyhow::Error>,
layout: &paths::Layout,
) -> Row {
let records = record.patch_records();
if records.is_empty() {
return Row::warn("Patch tier", "not applied — this is optional").note(
"the two optional features (vendor.sale events, in-game moderation audit) are \
unavailable; INSTALL.md §4",
);
}
let applied: Vec<String> = records
.iter()
.map(|f| {
let rungs: Vec<&str> = f.patches.iter().map(|p| p.rung.as_str()).collect();
format!("{} ({})", f.feature, rungs.join(", "))
})
.collect();
let unsupported: Vec<&str> = records
.iter()
.filter(|f| f.unsupported_servuo)
.map(|f| f.feature.as_str())
.collect();
let mut notes = Vec::new();
let mut gone = 0usize;
match root {
Ok(root) => {
for feature in &records {
for applied_patch in &feature.patches {
match verify_applied(layout, &applied_patch.name, &applied_patch.sha256) {
None => notes.push(format!(
"{}: no cached copy of the patch, so it could not be re-checked",
applied_patch.name
)),
Some(parsed) => {
let target = patch::join(&root.path, &applied_patch.target);
let content = std::fs::read(&target).unwrap_or_default();
let resolution = patch::resolve(&parsed, &content);
if !matches!(resolution, patch::Resolution::AlreadyPresent { .. }) {
gone += 1;
notes.push(format!(
"{} is NO LONGER in {} — the file was replaced, reverted or \
upgraded since it was applied",
applied_patch.name, applied_patch.target
));
}
}
}
}
for companion in &feature.companions {
if !patch::join(&root.path, &companion.path).is_file() {
gone += 1;
notes.push(format!("{} is missing from the tree", companion.path));
}
}
}
}
Err(_) => {
notes.push("the ServUO tree is unreadable, so nothing could be re-checked".into())
}
}
if !unsupported.is_empty() {
// The label follows the install (PLAN.md §2.2.2): whoever inherits this shard must be able
// to see it here, not only in the output of a run they never saw.
notes.push(format!(
"applied on an UNSUPPORTED ServUO ({}): {}",
records
.iter()
.find_map(|f| f.servuo_version.clone())
.unwrap_or_else(|| "unknown".into()),
unsupported.join(", ")
));
}
let detail = format!("{} applied — {}", records.len(), applied.join("; "));
let mark = if gone > 0 {
Mark::Fail
} else if unsupported.is_empty() {
Mark::Ok
} else {
Mark::Warn
};
Row::new(mark, "Patch tier", detail).notes_from(notes)
}
/// Finds the cached `.patch` for a recorded patch and parses it.
///
/// By name first, which is what the tier writes; then by content hash across the cache, so a
/// release that renames a patch file does not silently turn this check off.
fn verify_applied(
layout: &paths::Layout,
name: &str,
sha256: &str,
) -> Option<crate::diff::FilePatch> {
let by_name = layout.patches_dir().join(format!("{name}.patch"));
let candidates: Vec<PathBuf> = if by_name.is_file() {
vec![by_name]
} else {
std::fs::read_dir(layout.patches_dir())
.ok()?
.filter_map(|e| e.ok())
.map(|e| e.path())
.filter(|p| p.extension().is_some_and(|e| e == "patch"))
.collect()
};
for path in candidates {
let Ok(bytes) = std::fs::read(&path) else {
continue;
};
if path.file_stem().is_some_and(|s| s == name)
|| crate::util::sha256_bytes(&bytes) == sha256
{
if let Ok(parsed) = crate::diff::parse(&bytes) {
return parsed.single_file().ok().cloned();
}
}
}
None
}
fn link_row(link: Option<&LinkRecord>, live: Option<&sidecar::ConfigDoc>) -> Row {
let Some(link) = link else {
return Row::fail("uo-link installed", "no sidecar recorded in install.json")
.note("the overlay alone does not reach a website — see INSTALL.md §2");
};
let binary = Path::new(&link.binary.path);
if !binary.is_file() {
return Row::fail(
"uo-link installed",
format!("{} is missing", link.binary.path),
);
}
// The version line comes from the binary, the hash decides whether it is the one that was
// installed. A hand-replaced binary that still reports the right version is exactly the case a
// version string alone would call healthy.
let reported =
sidecar::version_line(binary).unwrap_or_else(|| format!("uo-link {}", link.version));
let row = match crate::util::sha256_file(binary) {
Ok(actual) if actual.eq_ignore_ascii_case(&link.binary.sha256) => {
Row::ok("uo-link installed", reported)
}
Ok(_) => Row::warn("uo-link installed", reported).note(format!(
"{} is not the binary this installer recorded — it was replaced by hand or by another \
tool",
link.binary.path
)),
Err(error) => Row::warn("uo-link installed", reported)
.note(format!("cannot hash {}: {error}", link.binary.path)),
};
// The paths the *binary* resolves, under the same environment the service definition pins (see
// `live_config`) — not the ones the record believes it was told. A sidecar reading a different
// config from the one the installer wrote is a failure mode nothing else here would surface.
match live {
Some(doc) => row.note(format!(
"config {} database {}",
doc.config_path, doc.store.path
)),
None => row.note(format!(
"config {} (not readable by this run) database {}",
link.config_path, link.db_path
)),
}
}
fn service_row(link: Option<&LinkRecord>) -> Row {
let Some(link) = link else {
return Row::fail("Service", "nothing recorded");
};
let Some(service) = &link.service else {
// A deliberate outcome, not a bug: a host with no service manager the installer can drive
// gets the binary, the config, and printed instructions (PLAN.md Phase 2).
return Row::warn("Service", "not registered")
.note("the binary and config are installed but nothing runs them — INSTALL.md §7");
};
let status = crate::service::observe(&service.kind, &service.name);
let account = service
.user
.as_deref()
.map(|u| format!(" as {u}"))
.unwrap_or_default();
let detail = format!("{} {}{account}", service.name, status.detail);
if !status.present {
Row::fail("Service", detail).note("re-run `install` to register it again")
} else if !status.running {
Row::fail("Service", detail)
.note("a service that will not stay up usually cannot read its config — INSTALL.md §7")
} else if !status.enabled {
Row::warn("Service", detail).note("it is running but will not come back after a reboot")
} else {
Row::ok("Service", detail)
}
}
/// Asks the installed binary what it resolves — but only if it has a config to read.
///
/// Two rules are load-bearing here:
///
/// - **Only when the config already exists.** `--print-config` provisions: it writes the file and
/// mints a token when there is none. `doctor` must not write, and a diagnosis that created the
/// very state it was asked to report on would be worse than one that said "no config".
/// - **Under the environment the service runs with.** The systemd unit pins `UOLINK_DB_PATH`
/// wherever the database does not land beside the config (`crate::paths`), so a bare
/// `--print-config` would report the path the binary picks *on its own* — `/etc/runicgateway/`
/// rather than `/var/lib/runicgateway/` — and INSTALL.md §7 promises this row names the file the
/// service actually opens.
///
/// The returned document holds the auth token. Nothing here reads it, and nothing prints it.
fn live_config(link: &LinkRecord) -> Option<sidecar::ConfigDoc> {
let config = Path::new(&link.config_path);
if !config.is_file() {
return None;
}
let db = Path::new(&link.db_path);
let db_env = (config.parent() != db.parent()).then_some(db);
sidecar::print_config(Path::new(&link.binary.path), config, db_env).ok()
}
/// The sidecar's `/health`, which needs no auth and is therefore safe to ask for from here.
///
/// Always over loopback, never over the configured bind: `[web] bind` is regularly `0.0.0.0`, and
/// this check is about whether the process on *this* host is answering.
fn health_of(bind: &str) -> Option<Health> {
let port = bind.rsplit_once(':').map(|(_, p)| p).unwrap_or(bind);
let url = format!("http://127.0.0.1:{port}/health");
let body = net::get_text_within(&url, HEALTH_TIMEOUT).ok()?;
serde_json::from_str(&body).ok()
}
/// The sidecar's `/health` document. Every field is optional so a newer sidecar that drops or
/// renames one still produces a report rather than a parse failure.
#[derive(Debug, Clone, Deserialize)]
pub struct Health {
pub status: Option<String>,
pub protocol: Option<u32>,
pub plugin_connected: Option<bool>,
pub database: Option<String>,
pub uptime: Option<String>,
pub last_event: Option<String>,
}
fn reachable_row(live: Option<&sidecar::ConfigDoc>, health: Option<&Health>) -> Row {
let Some(doc) = live else {
return Row::fail("Sidecar reachable", "could not read the sidecar's config").note(
"either the config file is gone or this process cannot read it — it is deliberately \
readable only by root/Administrator and the service account",
);
};
let port = doc
.web
.bind
.rsplit_once(':')
.map(|(_, p)| p.to_string())
.unwrap_or_else(|| doc.web.bind.clone());
match health {
Some(health) => {
let uptime = health
.uptime
.as_deref()
.map(|u| format!(", up {u}"))
.unwrap_or_default();
let db = health.database.as_deref().unwrap_or("unknown");
Row::ok(
"Sidecar reachable",
format!(
"127.0.0.1:{port} /health {}{uptime}, database {db}",
health.status.as_deref().unwrap_or("ok")
),
)
}
None => Row::fail(
"Sidecar reachable",
format!("nothing answered http://127.0.0.1:{port}/health"),
)
.note("the binary is installed; this is about whether it is running and listening"),
}
}
fn protocol_row(
record: &InstallRecord,
live: Option<&sidecar::ConfigDoc>,
health: Option<&Health>,
) -> Row {
let overlay = record.overlay.as_ref().map(|o| o.protocol);
// `/health` first: that is the number the website is answered with. `--print-config` is the
// same value from the same binary and covers a sidecar that is installed but not running.
let sidecar = health.and_then(|h| h.protocol).or(live.map(|d| d.protocol));
match (sidecar, overlay) {
(Some(s), Some(o)) if s == o => {
Row::ok("Protocol", format!("sidecar {s} = overlay manifest {o}"))
}
(Some(s), Some(o)) => Row::fail("Protocol", format!("sidecar {s} ≠ overlay manifest {o}"))
.note(
"the sidecar rejects a mismatched website with 409 rather than mis-parsing it; \
this pair was never checked together — run `update` to move both to one bundle",
),
(s, o) => Row::warn(
"Protocol",
format!(
"sidecar {}, overlay manifest {}",
s.map(|v| v.to_string()).unwrap_or_else(|| "unknown".into()),
o.map(|v| v.to_string()).unwrap_or_else(|| "unknown".into())
),
),
}
}
/// The row the rest of the report exists to make trustworthy.
///
/// A shard that is not running cannot have dialed in, so that case is a `⚠` with the reason rather
/// than a `✗`: reporting a stopped shard as a failure would train operators to ignore this line,
/// which is the one line worth reading.
fn shard_row(root: Result<&ServUoRoot, &anyhow::Error>, health: Option<&Health>) -> Row {
let running = root.ok().and_then(|r| servuo::find_running(&r.path));
match (health.and_then(|h| h.plugin_connected), running) {
(Some(true), _) => {
let last = health
.and_then(|h| h.last_event.clone())
.map(|e| format!(" (last event {e})"))
.unwrap_or_default();
Row::ok("Shard connected", format!("yes{last}"))
}
(Some(false), Some(shard)) => Row::fail(
"Shard connected",
format!(
"no — the shard is running (pid {}) but has not dialed in",
shard.pid
),
)
.note(
"the classic silent failure: ServUO ignores the script build's exit code and reloads \
the previous Scripts.dll",
)
.note("run `[bridge status` in game, and check [shard] bind against Config/Bridge.cfg"),
(Some(false), None) => Row::warn(
"Shard connected",
"no — the shard process is not running on this host",
)
.note("start ServUO and re-run doctor; nothing reaches the website until it dials in"),
(None, _) => Row::warn(
"Shard connected",
"unknown — the sidecar did not answer /health",
),
}
}
/// Whether this deployment is still the current bundle.
///
/// Offline is a `⚠`, never a `✗`. A shard host with no outbound route to Gitea is a supported way
/// to run this — the operator downloads artifacts elsewhere — and failing a health check over it
/// would report a working deployment as broken.
/// The most recent backup, so "can I go back?" is answerable without knowing the layout.
///
/// Always `✓`, never a failure: having no backup is the correct state on a host that has never
/// overwritten anything, and a shard that is running fine does not become broken because nothing
/// has displaced a file yet.
fn backup_row(layout: &paths::Layout) -> Row {
let backups = crate::backup::list(layout);
let Some(newest) = backups.first() else {
return Row::ok("Backups", "none taken — no run has replaced a file yet");
};
let detail = match crate::backup::read_manifest(newest) {
Ok(manifest) => format!(
"{}{} file(s) replaced by {} to bundle {}",
manifest.taken,
manifest.files.len(),
manifest.command,
manifest.bundle_to
),
// A directory with an unreadable manifest is still a directory of the operator's files, so
// it is reported rather than skipped.
Err(_) => format!("{} — manifest unreadable", newest.display()),
};
Row::ok("Backups", detail).note(format!(
"{} kept in {}",
backups.len(),
layout.backups_dir().display()
))
}
fn bundle_row(record: &InstallRecord) -> Row {
let url = bundle::url_for(None);
let current = match net::get_text_within(&url, BUNDLE_TIMEOUT).and_then(|b| bundle::parse(&b)) {
Ok(bundle) => bundle,
Err(error) => {
return Row::warn(
"Bundle",
format!("{} — could not check for a newer one", record.bundle.tag),
)
.note(error.to_string().replace('\n', " "))
}
};
if current.bundle == record.bundle.tag {
return Row::ok("Bundle", format!("{} — up to date", record.bundle.tag));
}
let mut moves = Vec::new();
if let Some(link) = record.link_record() {
if link.version != current.link.version {
moves.push(format!(
"uo-link {}{}",
link.version, current.link.version
));
}
}
if let Some(overlay) = &record.overlay {
if overlay.version != current.overlay.version {
moves.push(format!(
"overlay {}{}",
overlay.version, current.overlay.version
));
}
}
let detail = if moves.is_empty() {
format!(
"{}{} (no component changed)",
record.bundle.tag, current.bundle
)
} else {
format!(
"{}{}: {}",
record.bundle.tag,
current.bundle,
moves.join(", ")
)
};
Row::warn("Bundle", detail).note("run `update` to move both halves to one checked combination")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::record::{BundleRef, FileRecord, InstallerInfo, OverlayRecord, ServUoRef, SCHEMA};
use std::collections::BTreeMap;
fn record() -> InstallRecord {
InstallRecord {
schema: SCHEMA,
installer: InstallerInfo {
version: "0.1.0".into(),
},
updated: "2026-08-05T10:00:00Z".into(),
bundle: BundleRef {
tag: "2026.08.04".into(),
protocol: 3,
url: "https://example/current.json".into(),
},
servuo: ServUoRef {
path: "/opt/ServUO".into(),
version: Some("57.4".into()),
},
overlay: Some(OverlayRecord {
repo: "RunicGateway/servuo-plugins".into(),
tag: "v0.1.1".into(),
version: "0.1.1".into(),
commit: "3a52abb".into(),
protocol: 3,
files: BTreeMap::new(),
}),
link: None,
patches: Vec::new(),
extra: BTreeMap::new(),
}
}
fn health(protocol: u32, connected: bool) -> Health {
Health {
status: Some("ok".into()),
protocol: Some(protocol),
plugin_connected: Some(connected),
database: Some("ok".into()),
uptime: Some("2m".into()),
last_event: Some("2026-08-05T10:00:00Z".into()),
}
}
#[test]
fn the_documented_health_document_parses() {
// Copied from INSTALL.md §6 — the shape doctor's last three rows are read from.
let body = r#"{"status":"ok","protocol":3,"plugin_connected":true,"database":"ok",
"uptime":"2m","last_event":"2026-08-04T18:22:10.412Z"}"#;
let health: Health = serde_json::from_str(body).unwrap();
assert_eq!(health.protocol, Some(3));
assert_eq!(health.plugin_connected, Some(true));
}
#[test]
fn a_health_document_missing_fields_still_parses() {
// A newer sidecar dropping or renaming a key must degrade to an unknown row, not to a
// doctor that cannot report at all.
let health: Health = serde_json::from_str(r#"{"status":"ok"}"#).unwrap();
assert_eq!(health.protocol, None);
assert_eq!(health.plugin_connected, None);
}
#[test]
fn a_protocol_mismatch_fails_the_row() {
assert_eq!(
protocol_row(&record(), None, Some(&health(3, true))).mark,
Mark::Ok
);
assert_eq!(
protocol_row(&record(), None, Some(&health(4, true))).mark,
Mark::Fail
);
// Nothing to compare is not a failure — an unreachable sidecar is already its own ✗ row,
// and reporting the same outage twice buries the one that names the cause.
assert_eq!(protocol_row(&record(), None, None).mark, Mark::Warn);
}
#[test]
fn a_stopped_shard_is_a_warning_and_a_silent_one_is_a_failure() {
// The distinction the whole row exists for: "you have not started it" and "it is running
// and the bridge is dead" are different problems, and only the second is broken.
let err = anyhow::anyhow!("no tree");
assert_eq!(
shard_row(Err(&err), Some(&health(3, false))).mark,
Mark::Warn
);
assert_eq!(shard_row(Err(&err), Some(&health(3, true))).mark, Mark::Ok);
assert_eq!(shard_row(Err(&err), None).mark, Mark::Warn);
}
#[test]
fn an_empty_patch_tier_is_a_warning_not_a_failure() {
// The tier is optional and most shards will decline it. A ✗ there would make a correct
// install look broken forever.
let layout = paths::layout();
let err = anyhow::anyhow!("no tree");
assert_eq!(patch_row(&record(), Err(&err), &layout).mark, Mark::Warn);
}
#[test]
fn a_missing_overlay_file_fails_and_an_edited_one_warns() {
let dir = crate::util::TempDir::new("rg-test-doctor").unwrap();
let root = ServUoRoot {
path: dir.path().to_path_buf(),
version: Some("57.4".into()),
};
std::fs::create_dir_all(dir.path().join("Scripts")).unwrap();
std::fs::write(dir.path().join("Scripts/A.cs"), b"deployed").unwrap();
let deployed = crate::util::sha256_bytes(b"deployed");
let mut record = record();
let files = &mut record.overlay.as_mut().unwrap().files;
files.insert(
"Scripts/A.cs".into(),
FileRecord {
overlay_sha256: deployed.clone(),
on_disk_sha256: deployed.clone(),
state: "deployed".into(),
},
);
assert_eq!(overlay_row(&record, Ok(&root)).mark, Mark::Ok);
std::fs::write(dir.path().join("Scripts/A.cs"), b"edited by hand").unwrap();
assert_eq!(overlay_row(&record, Ok(&root)).mark, Mark::Warn);
std::fs::remove_file(dir.path().join("Scripts/A.cs")).unwrap();
assert_eq!(overlay_row(&record, Ok(&root)).mark, Mark::Fail);
}
#[test]
fn an_operator_owned_file_is_never_reported_as_drift() {
// Bridge.cfg is *meant* to be edited in place; flagging it every run would teach operators
// to ignore this row.
let dir = crate::util::TempDir::new("rg-test-doctor-owned").unwrap();
let root = ServUoRoot {
path: dir.path().to_path_buf(),
version: Some("57.4".into()),
};
std::fs::create_dir_all(dir.path().join("Config")).unwrap();
std::fs::write(dir.path().join("Config/Bridge.cfg"), b"edited again").unwrap();
let mut record = record();
record.overlay.as_mut().unwrap().files.insert(
"Config/Bridge.cfg".into(),
FileRecord {
overlay_sha256: "aa".into(),
on_disk_sha256: "bb".into(),
state: "kept-operator-modified".into(),
},
);
let row = overlay_row(&record, Ok(&root));
assert_eq!(row.mark, Mark::Ok);
assert!(row.detail.contains("operator-owned"), "{}", row.detail);
}
}