feat(rust): the permission manager — the site owns the whole store (D160-D163, D188-D198)

PLAN_REDESIGNS section 1.

- Every sync reads the store (perm.inventory), reconciles it against the
  site's record and its ledger, and pushes. A change made in the game is
  settled by the server's policy (D161): auto-adopt (default), adopt, or
  revoke. The first read of a server imports everything (D198).
- Groups belong to one server unless an admin shares them (D189), in new
  id-keyed tables; the old ones are copied once at boot and left unread.
  Holders may be a Steam account nobody linked (D188).
- An in-game change affects that server only (D190): a grant that reaches
  further gains an exception, a shared group is split.
- Never judged: a permission the server does not register right now (an
  unloaded plugin is not a revocation), and a pair an event lease holds.
- A new admin API (server view, grant/revoke with everywhere-or-here,
  groups by id, share/split, members, drift answers) and a screen on
  PermissionsManager's flow with a state on every toggle (D162, D163, U-1).
- The announcement voice names a group by id; old name settings still read.

Walked on both rigs against the walk core: import on an existing install,
auto-adopt of a grant and a revoke, a fleet grant's exception, Kits
unloaded without loss, a shared group split, adopt and revoke policies.
Server 420/420, client 58/58, swagger, imports and route manifest current.

Refs #21

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E14m6SuuY6i1vASFeGDBeY

server/permSync.js

@@ -1,22 +1,23 @@
-// ── Keeping a game's permission store equal to what the site authored ─────
+// ── Keeping a game's permission store and the site's record of it equal ───
 //
-// R2's whole mechanism, and it is chapter 4's board pointed the other way: the
-// site is the single producer of a set, it re-sends the whole thing rather than
-// a stream of edits, and the receiver reconciles. What is new is the direction —
-// the module telling the game what the site knows, where every earlier phase
-// asked the game what it knew.
+// R2's whole mechanism, rebuilt in protocol 13 (PLAN_REDESIGNS §1) around one
+// fact: **the site owns every permission and group on the server** — those that
+// were there before it, those an admin makes, and those changed in the game
+// (D160). So a sync is three steps, not one:
 //
-// ── One verb (D32) ────────────────────────────────────────────────────────
+//   1. READ the whole store (`perm.inventory`), in pages, with the plugin that
+//      registered each permission.
+//   2. RECONCILE it against the site's record and what the site last pushed
+//      (`reconcile.js`): a change made in the game becomes the site's own, a
+//      question for a person, or undone — the server's policy says which (D161).
+//      The first read of a server imports everything it finds (D198).
+//   3. PUSH the whole desired set (`perm.sync`, D32) with what the site has
+//      withdrawn, and record what the plugin says landed.
 //
-// A sync sends the whole desired set and the plugin diffs it against the live
-// store. The website never holds a copy of the game's permissions, which is the
-// point: a second source of truth is stale the moment it lands, and the store is
-// the bigger of the two sets.
-//
-// The delta the site DOES compute is the one the game cannot: what this site put
-// there and has since withdrawn (`retirements`). A name in the store that is not
-// in the desired set is either that, or a hand edit — and only the pushed ledger
-// can tell them apart (D31).
+// Step 2 writes to the site's own tables, and a change in one game affects that
+// server only (D190) — which can split a group shared with other servers. So it
+// runs under ONE lock for the whole fleet: two servers' reconciles never split
+// the same shared group at once. Steps 1 and 3 run in parallel across servers.
 //
 // ── When it runs ──────────────────────────────────────────────────────────
 //
@@ -25,25 +26,26 @@
 // yes. A sync therefore happens when:
 //
 //   • an operator changed something (the dirty flag, and the digest behind it)
-//   • the game restarted or wiped (a new boot id or wipe id: the store may have
-//     been emptied, and R2's promise is that a wipe is not a data-loss event)
+//   • the game restarted or wiped (a new boot id or wipe id)
 //   • a permission hook fired in the game that we did not cause (`ingest.js`
-//     marks the server dirty; the authoritative answer is this sync's report)
-//   • the audit interval elapsed — the backstop that finds drift on a quiet
-//     server nobody has touched
+//     marks the server dirty; the inventory then says what changed)
+//   • the audit interval elapsed — the backstop that finds a hand edit on a quiet
+//     server
 //   • the last attempt failed, after a backoff
 //
 // ── What it never does ────────────────────────────────────────────────────
 //
-// It does not remove a grant it did not make (D31), it does not invent a
-// permission the server has not registered (D33), and it does not treat a
-// silent sidecar as a reason to forget anything. A server that is unreachable
-// keeps its retirements and its revocations until it comes back.
+// It never judges a permission the server has not registered right now (a
+// plugin unloaded for a minute is not a revocation), never touches what an event
+// lease holds, never invents a permission the server has not registered (D33),
+// and never treats a silent sidecar as a reason to forget anything.
 
 const core = require('./core')
 
+const apply = require('./model/permissions/permissions.apply')
 const db = require('./model/permissions/permissions.db')
 const model = require('./model/permissions/permissions.model')
+const reconcile = require('./model/permissions/reconcile')
 const servers = require('./model/servers/servers.model')
 const serversDb = require('./model/servers/servers.db')
 const sidecar = require('./sidecarClient')
@@ -54,11 +56,8 @@ const log = core.logger('permissions')
 const TICK_MS = 30 * 1000
 
 /**
- * How long a server may go without a full reconciliation, however quiet it is.
- *
- * The digest comparison is what keeps the loop cheap, and on its own it would
- * also mean a server whose store somebody edited by hand is never asked about
- * again. This is the interval at which the question gets asked anyway.
+ * How long a server may go without a full reconciliation, however quiet it is:
+ * the interval at which a hand edit on a quiet server is found anyway.
  */
 const AUDIT_MS = 15 * 60 * 1000
 
@@ -66,16 +65,26 @@ const AUDIT_MS = 15 * 60 * 1000
 const FAIL_BACKOFF_MS = 2 * 60 * 1000
 
 /**
- * The most rows one sync may carry.
- *
- * Below the sidecar's line cap and below the plugin's operation ceiling, so the
- * refusal happens here — where it can name the server and reach an operator —
- * rather than as a `413` or a `too-large` from two processes away.
+ * The most rows one sync may carry. Below the sidecar's line cap and the
+ * plugin's operation ceiling, so the refusal happens here, where it can name the
+ * server and reach an operator.
  */
 const MAX_ROWS = 15000
 
+/** The policies a server may have (D161). */
+const POLICIES = ['auto-adopt', 'adopt', 'revoke']
+
 let timer = null
 
+/** The fleet-wide lock the reconcile step runs under (see the header). */
+let lock = Promise.resolve()
+
+function withLock(fn) {
+  const run = lock.then(fn, fn)
+  lock = run.catch(() => {})
+  return run
+}
+
 function start() {
   if (timer) return
 
@@ -96,25 +105,26 @@ function stop() {
 /**
  * One pass over every enabled server.
  *
- * The authored set is read ONCE and handed to each server's build: six servers
- * are six different answers derived from the same four tables, and re-reading
- * them per server is six times the queries for identical rows.
+ * The authored set is read once here, for the cheap "does anything need doing"
+ * digest. The reconcile re-reads it under the lock, because another server's
+ * reconcile may have changed it in between.
  */
 async function tick({ force = null } = {}) {
   await db.ensureSyncRows()
 
-  const [rows, state, sync, authored] = await Promise.all([
+  const [rows, state, sync, authored, policyRows] = await Promise.all([
     servers.listForPolling(),
     serversDb.listState(),
     db.listSync(),
     model.readAuthored(),
+    db.listPolicies(),
   ])
 
   const syncById = new Map(sync.map((row) => [row.serverId, row]))
   const stateById = new Map(state.map((row) => [row.serverId, row]))
+  const policies = new Map(policyRows.map((row) => [row.serverId, row.policy]))
 
-  // `allSettled`, for the same reason the board poll uses it: one unreachable
-  // host must not stop the other five being reconciled.
+  // `allSettled`: one unreachable host must not stop the others.
   await Promise.allSettled(
     rows
       .filter((server) => force === null || force === server.id)
@@ -124,6 +134,7 @@ async function tick({ force = null } = {}) {
           sync: syncById.get(server.id) || null,
           state: stateById.get(server.id) || null,
           force: force !== null,
+          policy: policies.get(server.id) || 'auto-adopt',
         }),
       ),
   )
@@ -179,18 +190,99 @@ function age(value) {
   return Number.isFinite(at) ? Date.now() - at : Number.MAX_SAFE_INTEGER
 }
 
-async function syncOne(server, { authored, sync, state, force }) {
-  const desired = model.buildDesired(server.id, authored)
-  const reason = reasonToSync({ desiredHash: desired.hash, sync, state, force })
+/** Record a failed attempt, with the reason on the row and in the log (F7). */
+async function failed(server, { reason, error, desiredHash, sync, bootId, wipeId }) {
+  log.warn('permission sync failed', { server: server.id, reason, error })
+  await db.putSyncResult(server.id, {
+    state: 'failed',
+    desiredHash,
+    syncedHash: sync ? sync.syncedHash : null,
+    bootId,
+    wipeId,
+    report: null,
+    error: String(error).slice(0, 191),
+  })
+}
+
+/**
+ * Read, reconcile and push one server. Returns what happened, for the log and
+ * the tests: null (nothing to do), 'ok', or why it stopped.
+ */
+async function syncOne(server, { authored, sync, state, force, policy = 'auto-adopt' }) {
+  const first = model.buildDesired(server.id, authored)
+  const reason = reasonToSync({ desiredHash: first.hash, sync, state, force })
 
   if (!reason) return null
 
-  const [pushed, revocations] = await Promise.all([
-    db.listPushed(server.id),
-    db.listRevocations(server.id),
-  ])
+  const bootId = state && state.bootId ? state.bootId : null
+  const wipeId = state && state.wipeId ? state.wipeId : null
 
-  const retirements = model.retirements(pushed, desired.rows)
+  // ── 1. Read the whole store ────────────────────────────────────────────
+  const read = await sidecar.readInventory(server)
+
+  if (!read.ok) {
+    await failed(server, { reason, error: read.error, desiredHash: first.hash, sync, bootId, wipeId })
+    return 'inventory'
+  }
+
+  const { inventory } = read
+  const registered = new Set(inventory.permissions.map((row) => model.normaliseName(row.name)).filter(Boolean))
+
+  // The option source, from the same read: what this server registers, and which
+  // plugin registered each name (PLAN_REDESIGNS §0.1).
+  await db.putCatalogue(
+    server.id,
+    inventory.permissions
+      .map((row) => ({ permission: model.normaliseName(row.name), owner: row.owner || null }))
+      .filter((row) => row.permission),
+  )
+
+  const importing = !(sync && sync.importedAt)
+
+  // ── 2. Reconcile, under the fleet lock ─────────────────────────────────
+  const settled = await withLock(async () => {
+    let current = await model.readAuthored()
+    let desired = model.buildDesired(server.id, current)
+    const pushed = await db.listPushed(server.id)
+
+    const classes = reconcile.classify({
+      present: reconcile.presentRows(inventory),
+      pushed,
+      desired: desired.rows,
+      registered,
+      leased: inventory.leased,
+    })
+
+    const planned = reconcile.plan({ classes, policy, importing, sources: desired.sources })
+
+    if (planned.ops.length) {
+      const done = await apply.applyOps(server.id, planned.ops, log)
+      log.info(importing ? 'permissions imported from the game' : 'changes made in the game adopted', {
+        server: server.id,
+        policy,
+        ops: done.length,
+      })
+
+      current = await model.readAuthored()
+      desired = model.buildDesired(server.id, current)
+    }
+
+    for (const row of planned.revocations) {
+      await db.queueRevocation({ serverId: server.id, ...row, requestedBy: null })
+    }
+
+    return { desired, planned, classes }
+  })
+
+  const { desired, planned } = settled
+  const held = reconcile.withHold(desired, planned.hold)
+
+  // ── 3. Push ────────────────────────────────────────────────────────────
+  const [pushed, revocations] = await Promise.all([db.listPushed(server.id), db.listRevocations(server.id)])
+
+  // Retired against the WHOLE desired set: a held row is still wanted, it is
+  // only not being pushed back while a person decides.
+  const retirements = model.retirements(pushed, desired.rows, planned.hold)
   const { styleRetired, sent: styleRetire } = styleRetirements(retirements)
   const retire = [
     ...retirements
@@ -200,84 +292,68 @@ async function syncOne(server, { authored, sync, state, force }) {
     ...revocations.map((row) => ({ kind: row.kind, subject: row.subject, object: row.object })),
   ]
 
-  const bootId = state && state.bootId ? state.bootId : null
-  const wipeId = state && state.wipeId ? state.wipeId : null
-
-  if (desired.rows.length + retire.length > MAX_ROWS) {
-    // Refused here rather than sent: the sidecar would answer `413` and the
-    // plugin would answer `too-large`, and neither of those messages reaches the
-    // person who has to make the set smaller.
-    const error = `the permission set is too large to push (${desired.rows.length + retire.length} rows, limit ${MAX_ROWS})`
-    log.error('permission sync refused', { server: server.id, rows: desired.rows.length })
-    await db.putSyncResult(server.id, {
-      state: 'failed',
+  if (held.rows.length + retire.length > MAX_ROWS) {
+    await failed(server, {
+      reason,
+      error: `the permission set is too large to push (${held.rows.length + retire.length} rows, limit ${MAX_ROWS})`,
       desiredHash: desired.hash,
-      syncedHash: sync ? sync.syncedHash : null,
+      sync,
       bootId,
       wipeId,
-      report: null,
-      error,
     })
-
     return 'too-large'
   }
 
   log.info('syncing permissions', {
     server: server.id,
     reason,
-    rows: desired.rows.length,
+    policy,
+    importing,
+    rows: held.rows.length,
     retire: retire.length,
+    held: planned.hold.size,
   })
 
   const result = await sidecar.permSync(server, {
     setId: desired.hash,
-    groups: withExpect(desired.payload.groups, pushed),
-    grants: desired.payload.grants,
-    managed: desired.payload.managed,
-    credits: desired.payload.credits,
+    groups: withExpect(held.payload.groups, pushed),
+    grants: held.payload.grants,
+    credits: held.payload.credits,
     retire,
   })
 
   if (!result.ok) {
-    // Said in the log as well as on the row (F7): the first walk's restart sync
-    // timed out with nothing in the log at all, while the titles push that failed
-    // beside it did log.
-    log.warn('permission sync failed', { server: server.id, reason, status: result.status })
-    await db.putSyncResult(server.id, {
-      state: 'failed',
-      desiredHash: desired.hash,
-      syncedHash: sync ? sync.syncedHash : null,
-      bootId,
-      wipeId,
-      report: null,
-      error: result.status,
-    })
-
+    await failed(server, { reason, error: result.status, desiredHash: desired.hash, sync, bootId, wipeId })
     return result.status
   }
 
   const report = result.data || {}
 
-  // The plugin refuses a whole sync with `perm.error` — `busy` while an earlier
-  // one is still draining, `too-large` past its own ceiling. Both are answers
-  // rather than transport failures, exactly like a refused link code, so they
-  // arrive as a 200 and are told apart by `kind`.
+  // `perm.error` (`busy`, `too-large`) is an answer, not a transport failure.
   if (report.kind === 'perm.error') {
-    log.warn('permission sync refused by the game', { server: server.id, reason, refused: report.reason || 'unknown' })
-    await db.putSyncResult(server.id, {
-      state: 'failed',
+    await failed(server, {
+      reason,
+      error: `the game refused the sync: ${report.reason || 'unknown'}`,
       desiredHash: desired.hash,
-      syncedHash: sync ? sync.syncedHash : null,
+      sync,
       bootId,
       wipeId,
-      report: null,
-      error: `the game refused the sync: ${report.reason || 'unknown'}`,
     })
-
     return report.reason || 'refused'
   }
 
-  await applyReport(server, { desired, retire, styleRetired, report, bootId, wipeId })
+  await applyReport(server, {
+    desired: held,
+    hash: desired.hash,
+    retire,
+    styleRetired,
+    report,
+    bootId,
+    wipeId,
+    drift: planned.drift,
+  })
+
+  if (importing) await db.markImported(server.id)
 
   return 'ok'
 }
@@ -324,31 +400,20 @@ function styleRetirements(retirements) {
 }
 
 /**
- * Record what the game said it did.
+ * Record what the game said it did, and what waits for a person.
  *
- * Three writes, and the order matters only in that all three are safe to repeat:
- * a sync that crashes here is re-run next tick and reaches the same place, which
- * is the property that lets this loop be the only writer.
+ * Every write is safe to repeat: a sync that crashes here is re-run next tick and
+ * reaches the same place.
  */
-async function applyReport(server, { desired, retire, styleRetired = [], report, bootId, wipeId }) {
+async function applyReport(server, { desired, hash, retire, styleRetired = [], report, bootId, wipeId, drift = [] }) {
   const unresolved = new Set((report.unresolved || []).map(model.normaliseName))
   const pending = new Set(report.pending || [])
-  // Grants the plugin made and then did not find in the store when it read it
-  // back (D85). Before protocol 9 there was no such read-back, and on Oxide every
-  // grant of another plugin's permission landed nowhere while this site recorded
-  // it as pushed (PLAN.md §27.6).
+  // Grants the plugin made and then did not find when it read them back (D85);
+  // since protocol 13 also `group:parent:name` for a parent that would not set.
   const notLanded = new Set((report.notLanded || []).map((entry) => String(entry).toLowerCase()))
 
-  // A grant naming a permission this server has not registered did NOT land —
-  // `GrantUserPermission` no-ops silently for an unregistered name, which is
-  // why the plugin pre-checks and says so. Recording it as pushed would make the
-  // site believe it had given a privilege it had not.
-  //
-  // The same for a member the store could not place: the membership is waiting
-  // on their first connection, and it is not in the game yet.
   // Protocol 12. A style field landed when BetterChat was there to take it and
-  // the report names it neither drift nor failed. With BetterChat absent none
-  // did, and each is sent again with the same `expect` next time (§33.2).
+  // the report names it neither drift nor failed (§33.2).
   const chat = report.chat && typeof report.chat === 'object' ? report.chat : null
   const chatLoaded = Boolean(chat && chat.loaded === true)
   const fieldKey = (group, field) => `${group} ${String(field || '').toLowerCase()}`
@@ -366,19 +431,24 @@ async function applyReport(server, { desired, retire, styleRetired = [], report,
       return !unresolved.has(row.object) && !notLanded.has(`${row.subject}:${row.object}`.toLowerCase())
     }
     if (row.kind === 'member') return !pending.has(`${row.subject}:${row.object}`)
+    if (row.kind === 'group') {
+      // The value landed only if the parent did; a group whose parent would not
+      // set is recorded without a value, so the next sync pushes it again.
+      if ([...notLanded].some((entry) => entry.startsWith(`${row.subject}:parent:`))) {
+        row.value = null
+      }
+    }
     return true
   })
 
   await db.addPushed(server.id, landed)
 
   // Everything retired is gone from the game whether the plugin removed it or
-  // found it already absent, so it stops being something this site put there.
-  // A `chat-group` is not a ledger row; its fields are, below.
+  // found it already absent. A `chat-group` is not a ledger row; its fields are.
   await db.removePushed(server.id, retire.filter((row) => row.kind !== 'chat-group'))
 
-  // A style's fields leave the ledger only once BetterChat has removed the group
-  // — or for `default`, which is never removed — so a style withdrawn while
-  // BetterChat was absent is retired by the first sync that can (D139).
+  // A style's fields leave the ledger only once BetterChat has removed the group,
+  // or for `default`, which is never removed (D139).
   const removed = new Set((chatLoaded && chat.removed) || [])
   await db.removePushed(
     server.id,
@@ -388,51 +458,42 @@ async function applyReport(server, { desired, retire, styleRetired = [], report,
   const revocations = await db.listRevocations(server.id)
   await db.deleteRevocations(revocations.map((row) => row.id))
 
+  // What waits for a person: the reconcile's (D161's `adopt`, and an event's
+  // grant removed in the game), and a style field somebody changed by hand (D138).
   await db.replaceDrift(server.id, [
-    ...(report.foreign || []).map((row) => ({
+    ...drift.map((row) => ({
       kind: String(row.kind || ''),
       subject: String(row.subject || ''),
       object: String(row.object || ''),
+      detail: row.detail === undefined || row.detail === null ? null : String(row.detail).slice(0, 255),
+      direction: row.direction || 'added',
     })),
-    // A style field somebody changed by hand, with what it holds now (D138).
     ...((chatLoaded && chat.drift) || []).map((row) => ({
       kind: 'chat-field',
       subject: String(row.group || ''),
       object: String(row.field || ''),
       detail: row.game === undefined || row.game === null ? null : String(row.game).slice(0, 255),
+      direction: 'changed',
     })),
   ])
 
   await db.putSyncResult(server.id, {
     state: 'ok',
-    desiredHash: desired.hash,
-    syncedHash: desired.hash,
+    desiredHash: hash,
+    syncedHash: hash,
     bootId,
     wipeId,
     report: JSON.stringify(report),
     error: null,
   })
 
-  // The option source, refreshed from the same server that just answered. It is
-  // a second round trip and it is worth it: the form must not offer a name that
-  // stopped being registered when somebody uninstalled a plugin, because a grant
-  // against one is a privilege nobody ever gets and nothing ever reports.
-  const catalogue = await sidecar.permCatalogue(server)
-
-  if (catalogue.ok && catalogue.data && Array.isArray(catalogue.data.permissions)) {
-    await db.putCatalogue(
-      server.id,
-      catalogue.data.permissions.map(model.normaliseName).filter(Boolean),
-    )
-  }
-
   log.info('permissions synced', {
     server: server.id,
     applied: report.applied,
     unresolved: (report.unresolved || []).length,
-    foreign: (report.foreign || []).length,
     pending: (report.pending || []).length,
     notLanded: (report.notLanded || []).length,
+    waiting: drift.length,
     ...(chat
       ? {
           betterChat: chatLoaded,
@@ -452,6 +513,7 @@ module.exports = {
   AUDIT_MS,
   FAIL_BACKOFF_MS,
   MAX_ROWS,
+  POLICIES,
   start,
   stop,
   tick,
@@ -460,4 +522,5 @@ module.exports = {
   applyReport,
   withExpect,
   styleRetirements,
+  withLock,
 }