feat(sidecar): forward the cliloc table, and stop reading refusals for meaning (Phase 2)

`GET /cliloc` — the first protocol-8 family that carries content rather than a
manifest. The shard decompresses its own client's table and cuts it into pages;
this forwards them and keeps none of it, which matters more here than usual: the
payload is five megabytes of EA's strings out of the operator's own client, and
the one copy that should exist is the one the website imports.

Paging is the caller's, deliberately. `?cursor=` echoes back the previous reply's
cursor until one says `more: false`; a sidecar that helpfully assembled the pages
would be holding the whole table in memory to do it. `?lang=` selects the file
and defaults on the shard.

`asset_error_status` replaces phase 1's substring test. That test chose 403 or
400 by looking for the word "disabled" in an operator-facing sentence, so
rewording the message would silently turn a refusal into a bad request. The
overlay now sends a `code`: DISABLED 403, NOT_FOUND 404 (a client without the
file — an operator fact, not a bug), UNREADABLE 422 (a file it has and cannot
decode, where repeating the request cannot help), UNAVAILABLE 503, BAD_REQUEST
400. The substring check survives as a fallback, with a test, because an overlay
and a sidecar are deployed separately and a phase-1 shard must keep its 403.

No `PROTOCOL_VERSION` change: 8 already covers this family (v8.md §14).

Verified against a live shard: 12 pages, 67,496 rows, every page inside the
512 KiB budget (max 524,086 of 524,288) and well under the 1 MiB line cap, the
whole table in 1.4 s. Concurrent callers get 425 while one is served, which is
the flow control working rather than an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

sidecar/src/main.rs

@@ -115,6 +115,14 @@ use tracing_subscriber::EnvFilter;
 ///     to every connected client. The dumb-forwarder property is doing real work here: the sidecar
 ///     does not know what an asset is, and must not learn.
 ///
+/// Phase 2 adds the first family that actually carries content: **`cliloc.table`**, served at
+/// `GET /cliloc`. It pages — the shard cuts at a byte budget and the caller echoes a cursor back —
+/// and the sidecar forwards it without keeping any of it, which matters more here than usual: the
+/// payload is five megabytes of EA's strings out of the operator's own client, and the one copy of
+/// it that should exist is the one the website imports. That phase also gave `assets.error` a
+/// `code`, so the status a refusal maps to stops depending on the wording of a human-facing
+/// sentence (see [`web::asset_error_status`]).
+///
 /// **No store migration**, again: nothing on this plane is an event, so nothing is persisted.
 pub const PROTOCOL_VERSION: u32 = 8;
 

sidecar/src/web.rs

@@ -141,6 +141,11 @@ pub async fn serve(addr: &str, state: AppState) -> anyhow::Result<()> {
         // the files on that host changed since the last import", and a cached answer to that is
         // worse than no answer.
         .route("/assets/sources", get(assets_sources))
+        // The cliloc table (Protocol 8, phase 2): UO's id -> display-string map, read out of the
+        // shard's own client and paged. RPC for the same reason as the source gate, and one more:
+        // it is five megabytes of somebody else's copyrighted strings, which this process has no
+        // business holding a copy of. It forwards them and forgets them.
+        .route("/cliloc", get(cliloc_table))
         .route_layer(middleware::from_fn_with_state(state.clone(), gate));
 
     let app = Router::new()
@@ -1345,18 +1350,7 @@ fn respond_assets(result: Result<Value, RpcError>) -> (StatusCode, Json<Value>)
             if kind == BUSY_KIND {
                 (BUSY_STATUS, Json(value))
             } else if kind == "assets.error" {
-                let reason = value
+                (asset_error_status(&value), Json(value))
-                    .get("reason")
-                    .and_then(|r| r.as_str())
-                    .unwrap_or("request rejected");
-
-                let code = if reason.contains("disabled") {
-                    StatusCode::FORBIDDEN
-                } else {
-                    StatusCode::BAD_REQUEST
-                };
-
-                (code, Json(value))
             } else {
                 (StatusCode::OK, Json(value))
             }
@@ -1372,6 +1366,85 @@ fn respond_assets(result: Result<Value, RpcError>) -> (StatusCode, Json<Value>)
     }
 }
 
+/// The status behind one `assets.error`.
+///
+/// Phase 2 gave the frame a `code`, and the reason is worth stating: phase 1 decided between 403
+/// and 400 by looking for the word "disabled" **in the operator-facing sentence**. That works
+/// until someone improves the wording, at which point a refusal quietly becomes a bad request and
+/// an administrator goes hunting for a bug in a correctly-written call. The sentence is for a
+/// human; the code is for this function.
+///
+/// The substring check survives as a fallback because an overlay is deployed independently of the
+/// sidecar: a phase-1 shard paired with a phase-2 sidecar still sends the codeless frame, and it
+/// must keep getting its 403.
+fn asset_error_status(value: &Value) -> StatusCode {
+    match value.get("code").and_then(|c| c.as_str()) {
+        Some("DISABLED") => return StatusCode::FORBIDDEN,
+        // The shard has no such file. Not the caller's mistake and not a broken shard -- a client
+        // that does not carry what was asked for, which the website reports to its operator.
+        Some("NOT_FOUND") => return StatusCode::NOT_FOUND,
+        // It has the file and cannot decode it: truncated, hand-edited, or not what it claims to
+        // be. 422 rather than 400, because the request was fine and repeating it will not help.
+        Some("UNREADABLE") => return StatusCode::UNPROCESSABLE_ENTITY,
+        // The shard cannot do this right now (it could not start its asset worker, say). Same
+        // status as "no shard connected", because it means the same thing to a caller: come back.
+        Some("UNAVAILABLE") => return StatusCode::SERVICE_UNAVAILABLE,
+        Some("BAD_REQUEST") => return StatusCode::BAD_REQUEST,
+        _ => {}
+    }
+
+    let reason = value
+        .get("reason")
+        .and_then(|r| r.as_str())
+        .unwrap_or("request rejected");
+
+    if reason.contains("disabled") {
+        StatusCode::FORBIDDEN
+    } else {
+        StatusCode::BAD_REQUEST
+    }
+}
+
+/// The cliloc table, as the shard's own client holds it (docs/link/v8.md §9).
+///
+/// Until protocol 8 this table reached the website by hand: the operator installed UOFiddler,
+/// built a converter against its `Ultima.dll`, ran it over their client's compressed `Cliloc.enu`
+/// and copied the result to the web host. The shard could not help, because ServUO's bundled
+/// `Ultima.StringList` cannot read a modern client's file either. Phase 2 put the decompressor in
+/// the overlay, so the shard reads its own client and the operator installs nothing.
+///
+/// **Paged, and the caller drives the paging** -- `?cursor=` echoes back whatever the previous
+/// reply's `cursor` was, until a reply says `more: false`. The pages are cut by byte budget on the
+/// shard (512 KiB against the 1 MiB inbound line cap), so a stock English table arrives in about
+/// eleven of them. The sidecar keeps none of it: it has no opinion about what a cliloc is, and a
+/// cached copy of five megabytes of EA's strings is exactly what this process should not hold.
+///
+/// `?lang=` selects the file; it defaults to `enu` on the shard and the shard refuses anything its
+/// `Ultima.Files` cannot resolve, which is a 404 rather than a 400.
+async fn cliloc_table(
+    State(st): State<AppState>,
+    Query(q): Query<ClilocQuery>,
+) -> impl IntoResponse {
+    let req_id = st.rpc.next_req_id();
+    let mut cmd = json!({"kind": "cliloc.table", "reqId": req_id});
+
+    if let Some(lang) = q.lang.as_deref().filter(|s| !s.is_empty()) {
+        cmd["lang"] = json!(lang);
+    }
+
+    if let Some(cursor) = q.cursor.as_deref().filter(|s| !s.is_empty()) {
+        cmd["cursor"] = json!(cursor);
+    }
+
+    respond_assets(st.rpc.call(&st.shard, cmd, &req_id).await)
+}
+
+#[derive(Deserialize)]
+struct ClilocQuery {
+    lang: Option<String>,
+    cursor: Option<String>,
+}
+
 // ---- websocket ----
 
 async fn ws_upgrade(ws: WebSocketUpgrade, State(state): State<AppState>) -> impl IntoResponse {
@@ -1480,6 +1553,74 @@ mod tests {
         assert_eq!(respond_assets(Ok(value)).0, StatusCode::BAD_REQUEST);
     }
 
+    /// Phase 2's codes, each of which says something a 400 does not. `NOT_FOUND` is a client that
+    /// does not carry the file (an operator fact, not a bug); `UNREADABLE` is a file that is there
+    /// and cannot be decoded, where repeating the request cannot help; `UNAVAILABLE` is the shard
+    /// declining for now.
+    #[test]
+    fn an_asset_error_code_picks_the_status() {
+        let with = |code: &str| {
+            respond_assets(Ok(json!({
+                "kind": "assets.error",
+                "reqId": "r-1",
+                "code": code,
+                "reason": "…"
+            })))
+            .0
+        };
+
+        assert_eq!(with("DISABLED"), StatusCode::FORBIDDEN);
+        assert_eq!(with("NOT_FOUND"), StatusCode::NOT_FOUND);
+        assert_eq!(with("UNREADABLE"), StatusCode::UNPROCESSABLE_ENTITY);
+        assert_eq!(with("UNAVAILABLE"), StatusCode::SERVICE_UNAVAILABLE);
+        assert_eq!(with("BAD_REQUEST"), StatusCode::BAD_REQUEST);
+        // An unknown code is not a reason to invent a status.
+        assert_eq!(with("SOMETHING_NEW"), StatusCode::BAD_REQUEST);
+    }
+
+    /// The overlay and the sidecar ship separately and an operator can run a phase-1 shard against
+    /// a phase-2 sidecar, so the codeless refusal must keep its 403. This is the whole reason the
+    /// substring check survives rather than being deleted with the code it preceded.
+    #[test]
+    fn a_codeless_disabled_refusal_is_still_a_403() {
+        let value = json!({
+            "kind": "assets.error",
+            "reqId": "r-9",
+            "reason": "asset extraction is disabled on this shard"
+        });
+
+        assert_eq!(respond_assets(Ok(value)).0, StatusCode::FORBIDDEN);
+    }
+
+    /// One page of the cliloc table. The assertion that matters is the envelope: `more` and
+    /// `cursor` reach the caller untouched, because the sidecar does not page this — the shard
+    /// cuts the pages and the website drives them, and a sidecar that "helpfully" assembled them
+    /// would be holding the whole table in memory to do it.
+    #[test]
+    fn a_cliloc_page_is_a_200_and_keeps_its_cursor() {
+        let value = json!({
+            "kind": "cliloc.table.ok",
+            "reqId": "r-2",
+            "lang": "enu",
+            "extractorVersion": 1,
+            "total": 67496,
+            "rows": [{"n": 1023721, "f": 0, "t": "quarter staff"}],
+            "more": true,
+            "cursor": "n:1023721",
+            "cut": "budget"
+        });
+
+        let (status, body) = respond_assets(Ok(value));
+
+        assert_eq!(status, StatusCode::OK);
+        assert_eq!(body.0.get("more").and_then(|v| v.as_bool()), Some(true));
+        assert_eq!(
+            body.0.get("cursor").and_then(|v| v.as_str()),
+            Some("n:1023721")
+        );
+        assert_eq!(body.0.get("total").and_then(|v| v.as_i64()), Some(67496));
+    }
+
     /// A source manifest comes back whole. Worth asserting because `respond_assets` sniffs `kind`
     /// and a family whose success kind ends in `.ok` sits one character away from the `.error`
     /// suffix the generic responder matches on -- which is exactly why this plane has its own