Claude
936a922487
Phase 7 found this on the tree family and fixed it there. It was inline in THREE
places: the body catalogue (phase 3), statics and land (phase 5), and the tree.
`expected != null` treats "" as a real fingerprint, so a caller that serialises a
missing value as an empty string has EVERY fetch refused -- with a sentence that
names no catalog at all ("catalog is now 8159778b"), which reads as a shard
fault rather than a caller one.
All three now go through one BridgeAssets.CatalogMismatch. Three copies of a
comparison are three chances for the next family to get it wrong in a way only a
differently-written client would ever reveal.
BridgeLeases keeps its own `expected != null` and is deliberately untouched:
there the value is a world property, where an empty string is a legitimate thing
to expect.
Verified against a live shard on a stock ServUO install, every family asked three
ways -- with a real catalog, with the field absent, and with an empty string:
cliloc.table walk 67,496 rows, 12 pages
body manifest / fetch 1,095 rows; ok all three ways
static + land fetch ok all three ways
static/land carry their OWN catalog art 66a112c1 vs body 323f284f
a cross-family catalog refused 422
tree manifest / fetch 141 files incl. BOTH empty ones, all three ways
empty files carry a VALID gzip member 2 rows gunzip to 0 bytes
a STALE catalog still refused on body, static and tree
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
1306 lines
56 KiB
C#
1306 lines
56 KiB
C#
using System;
|
|
using System.Collections.Generic;
|
|
using System.Drawing;
|
|
using System.Drawing.Imaging;
|
|
using System.Globalization;
|
|
using System.IO;
|
|
using System.Runtime.InteropServices;
|
|
using System.Security.Cryptography;
|
|
using System.Text;
|
|
using System.Threading;
|
|
|
|
using Ultima;
|
|
|
|
namespace Server.Custom.Bridge
|
|
{
|
|
/// <summary>
|
|
/// **The Asset Bridge's transport** (docs/link/v8.md §3, §6, §7 — protocol 8, phase 1).
|
|
///
|
|
/// Everything else on this link answers on the Core thread, reads live world state, and
|
|
/// replies in microseconds. The asset plane cannot: it reads hundreds of megabytes of the
|
|
/// operator's client files and decodes pictures out of them, and doing either on the Core
|
|
/// thread would stop the world for every player on the shard. So this class is the one
|
|
/// genuinely new shape in protocol 8 — a handler that accepts on the Core thread, hands
|
|
/// the work to a **dedicated asset worker**, and returns immediately.
|
|
///
|
|
/// Three rules hold it together, and each of them is answering a specific way this could
|
|
/// go wrong.
|
|
///
|
|
/// **1. Replies, never events.** Every asset frame carries the caller's `reqId`, so
|
|
/// `rpc.rs`'s `try_route` consumes it before `app.rs` can persist it to SQLite and
|
|
/// broadcast it to every WebSocket subscriber. An asset stream on the event path would
|
|
/// grow the sidecar's store without bound and fan megabytes at every connected client,
|
|
/// forever. Nothing here emits an unsolicited frame — if a request has no `reqId` it is
|
|
/// refused rather than answered.
|
|
///
|
|
/// **2. One request outstanding, always.** <see cref="BridgeLink.Emit"/>'s queue is bounded
|
|
/// drop-oldest in **lines, not bytes** — a design that is right for live events and
|
|
/// dangerous for bulk transfer, because 10,000 queued 200 KB replies is 2 GB of shard
|
|
/// memory. The bound that actually holds is flow control, not a bigger queue: this plane
|
|
/// has **one slot**, and a second asset request arriving while one is in flight is
|
|
/// answered `bridge.busy` (which the sidecar already maps to 425) rather than queued.
|
|
/// Queue depth therefore stays at approximately one by construction. A dropped or lost
|
|
/// reply just times out and is re-requested, which is safe because reading a client file
|
|
/// is idempotent and touches no world state.
|
|
///
|
|
/// Note what that costs, deliberately: a status poll shares the slot with a batch, so
|
|
/// polling during a long import gets 425 until the batch lands. That is honest — this
|
|
/// plane really is doing one thing at a time — and the admin surface (phase 8) is where a
|
|
/// separate status lane would have to argue for itself.
|
|
///
|
|
/// **3. Byte budgets, not counts.** Batches are cut by encoded size
|
|
/// (<see cref="BridgeConfig.AssetBatchBytes"/>, 512 KB), not by item count, because the
|
|
/// ceilings this has to live inside are byte ceilings: the sidecar refuses an inbound line
|
|
/// over 1 MiB, and base64 costs 33% on top of whatever the payload measures.
|
|
/// <see cref="PageBuilder"/> is that budget, and every asset family shares it so the
|
|
/// envelope cannot drift apart between them.
|
|
///
|
|
/// **Emitting from off the Core thread is safe here, and it is worth saying why.**
|
|
/// <c>BridgeLink.Emit</c> enqueues onto a <c>ConcurrentQueue</c> and never touches the
|
|
/// socket, so the enqueue itself is fine. The subtle part is
|
|
/// <c>BridgeIdempotency.Observe</c>, which <c>Emit</c> calls while a keyed command is in
|
|
/// flight: it captures a line only when that line's correlation field **exactly equals**
|
|
/// the open command's correlation value, and correlation values come from one monotonic
|
|
/// counter in the sidecar. An asset reply therefore cannot be mistaken for a keyed
|
|
/// command's reply, whatever the interleaving.
|
|
/// </summary>
|
|
public static class BridgeAssets
|
|
{
|
|
/// <summary>
|
|
/// What version of *our derivation* produced these bytes (§7).
|
|
///
|
|
/// The source gate hashes the operator's client files, which answers "did the inputs
|
|
/// change". It cannot answer "did the way we read them change" — and that is the case
|
|
/// that bites, because a corrected frame offset or a fixed hue application changes
|
|
/// every derived byte while every source file stays identical. So this is folded into
|
|
/// stage 1 alongside the hashes, and bumping it makes the whole working set drift,
|
|
/// which is the intended and correct blast radius.
|
|
///
|
|
/// Bump it whenever extraction changes what it produces from unchanged input. It is
|
|
/// the same rule <c>spawnAtlasSource.js</c>'s `PARSER_VERSION` follows, and it applies
|
|
/// here more rather than less: this pipeline derives far more from far less.
|
|
///
|
|
/// **2** — phase 4 (§4.3, §4.9). The catalogue now falls back to
|
|
/// <c>AnimationFrame*.uop</c> for bodies the legacy path has nothing for, which on a
|
|
/// stock client is 235 new sprites and two of them player-character bodies; and the
|
|
/// player-body set no longer carries ghost ids. Every client file is byte-identical
|
|
/// and the answer is different, which is precisely what this number exists to say.
|
|
///
|
|
/// **3** — phase 6 (§4.10, §11.2). A body with no art at action 0 is catalogued at
|
|
/// the first action that has any, and its key names that action. 74 more bodies on a
|
|
/// stock client, no existing key's bytes changed — but a body that was absent is now
|
|
/// a row, which is the same "unchanged input, different answer" this number covers.
|
|
/// </summary>
|
|
public const int EXTRACTOR_VERSION = 3;
|
|
|
|
// ── the one slot (§3.2) ──────────────────────────────────────────────────────────────
|
|
|
|
private static readonly object _sync = new object();
|
|
private static Thread _worker;
|
|
private static readonly AutoResetEvent _wake = new AutoResetEvent(false);
|
|
private static Action _job;
|
|
private static string _inFlight;
|
|
private static DateTime _inFlightSince;
|
|
private static bool _running;
|
|
|
|
private static long _served, _busied, _failed;
|
|
|
|
// ── the hash cache (§6) ──────────────────────────────────────────────────────────────
|
|
|
|
private static readonly Dictionary<string, CachedHash> _hashes =
|
|
new Dictionary<string, CachedHash>(StringComparer.OrdinalIgnoreCase);
|
|
|
|
private static Thread _hasher;
|
|
private static volatile bool _hashing;
|
|
|
|
private sealed class CachedHash
|
|
{
|
|
public long Size;
|
|
public long MTime;
|
|
public string Sha256;
|
|
}
|
|
|
|
// ── imaging (§4.4) ───────────────────────────────────────────────────────────────────
|
|
|
|
private static bool _imagingChecked;
|
|
private static bool _imagingOk;
|
|
private static string _imagingReason;
|
|
|
|
public static void Initialize()
|
|
{
|
|
if (!BridgeConfig.Enabled)
|
|
return;
|
|
|
|
DisableTheLibraryCache();
|
|
|
|
BridgeBoot.RegisterHandler("assets.sources", OnSources);
|
|
BridgeBoot.RegisterHandler("assets.fetch", OnFetch);
|
|
|
|
// Owned here since phase 7, for the same reason `assets.fetch` moved here in phase 5:
|
|
// it is the transport, and more than one family has something to enumerate.
|
|
BridgeBoot.RegisterHandler("assets.manifest", OnManifest);
|
|
}
|
|
|
|
/// <summary>
|
|
/// **Turns <c>Ultima.Files.CacheData</c> off for the life of the process** (phase 5,
|
|
/// §17.10). One line, and it answers two separate problems that both end in a
|
|
/// confident wrong picture or an out-of-memory shard.
|
|
///
|
|
/// **The poisoning.** <c>Art.GetStatic</c> and <c>Art.GetLand</c> memoise into a
|
|
/// <c>Bitmap[0xFFFF]</c> and hand back **the same instance** on every call, while
|
|
/// <c>Hue.ApplyTo</c> repaints a bitmap **in place**. So hueing a static edits the
|
|
/// library's cached copy: measured on this client, hue item 3922 once and every later
|
|
/// request for the *plain* 3922 comes back hued, and a second hue stacks on the first.
|
|
/// Nothing downstream can see it — the row is the right size, the right shape and the
|
|
/// right id. It is §4.5's failure mode arriving through a completely different door.
|
|
///
|
|
/// **The retention.** That array is never trimmed. Decoding this client's 39,189
|
|
/// statics once would leave 74 MB of <c>Bitmap</c> in a static field of a game server,
|
|
/// kept for as long as the process lives, to serve pictures nobody asked for twice.
|
|
///
|
|
/// The obvious alternative — copy each bitmap before hueing — was rejected, and not
|
|
/// only for the retention: <c>new Bitmap(src)</c> **throws** on the
|
|
/// <c>Format16bppArgb1555</c> these decoders produce, so the copy has to name the
|
|
/// source pixel format explicitly, which is a subtlety on the wrong side of a
|
|
/// correctness boundary.
|
|
///
|
|
/// **What it costs is nothing measurable here.** <c>Animations</c> — the whole of the
|
|
/// body catalogue — does not consult this flag at all, and
|
|
/// <see cref="BridgeCatalog"/> and <see cref="BridgeArt"/> each keep their own cache of
|
|
/// *encoded PNG bytes*, which is the thing worth holding: a tenth of the size, already
|
|
/// hashed, and released when it goes idle.
|
|
///
|
|
/// It is a process-global on a library nothing else in this overlay reads, which is why
|
|
/// setting it here rather than saving and restoring it around each decode is safe —
|
|
/// and a save/restore would not be, because the asset worker is a thread.
|
|
/// </summary>
|
|
private static void DisableTheLibraryCache()
|
|
{
|
|
try
|
|
{
|
|
Files.CacheData = false;
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
// A client this library cannot even open. The families report that for themselves,
|
|
// per key, with a reason; it must not stop the plugin booting.
|
|
Console.WriteLine("[Bridge] assets: could not disable the Ultima bitmap cache: {0}",
|
|
e.Message);
|
|
}
|
|
}
|
|
|
|
public static string Status()
|
|
{
|
|
int cached;
|
|
|
|
lock (_hashes)
|
|
{
|
|
cached = _hashes.Count;
|
|
}
|
|
|
|
lock (_sync)
|
|
{
|
|
return String.Format(
|
|
"assets(served={0} busied={1} failed={2} inFlight={3} hashing={4} cached={5})",
|
|
_served, _busied, _failed, _inFlight ?? "-", _hashing, cached);
|
|
}
|
|
}
|
|
|
|
// ── the request plane ────────────────────────────────────────────────────────────────
|
|
|
|
/// <summary>
|
|
/// Stage 1 of §6: what the shard's client files currently are. No pixels, no assets —
|
|
/// just the gate that lets the website decide whether anything needs importing at all,
|
|
/// because the normal case is a restart that changed nothing and it must cost nothing.
|
|
/// </summary>
|
|
private static void OnSources(Dictionary<string, object> o)
|
|
{
|
|
var reqId = BridgeJson.GetString(o, "reqId");
|
|
|
|
if (reqId == null)
|
|
{
|
|
// Rule 1. Without a correlation id this would land on the event path, be persisted
|
|
// to the sidecar's store and broadcast to every subscriber. Refuse instead.
|
|
Fail(null, "BAD_REQUEST", "assets.sources requires a reqId");
|
|
return;
|
|
}
|
|
|
|
// Stage 1 answers for the whole plane, not for the client files alone: since phase 7
|
|
// an operator can serve the shard's own configuration tree while declining to serve
|
|
// their UO client, and `families` is where a website discovers which. Refused only
|
|
// when there is nothing at all to report.
|
|
if (Families().Count == 0)
|
|
{
|
|
Fail(reqId, "DISABLED", "asset extraction is disabled on this shard");
|
|
return;
|
|
}
|
|
|
|
Accept(reqId, "assets.sources", () => ReplySources(reqId));
|
|
}
|
|
|
|
/// <summary>
|
|
/// Claims the single slot and hands the work to the worker, or answers `bridge.busy`.
|
|
/// Runs on the Core thread and does nothing expensive; <paramref name="job"/> runs on
|
|
/// the worker and must touch no world state.
|
|
/// </summary>
|
|
internal static void Accept(string reqId, string kind, Action job)
|
|
{
|
|
lock (_sync)
|
|
{
|
|
if (_inFlight != null)
|
|
{
|
|
_busied++;
|
|
Busy(reqId, kind);
|
|
return;
|
|
}
|
|
|
|
_inFlight = kind;
|
|
_inFlightSince = DateTime.UtcNow;
|
|
_job = job;
|
|
|
|
try
|
|
{
|
|
EnsureWorker();
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
// The slot is claimed and there is now nothing that will ever free it. Give it
|
|
// back here or this plane answers `bridge.busy` for the life of the process.
|
|
_inFlight = null;
|
|
_job = null;
|
|
|
|
Console.WriteLine("[Bridge] cannot start the asset worker: {0}", e.Message);
|
|
Fail(reqId, "UNAVAILABLE", "the shard could not start its asset worker");
|
|
return;
|
|
}
|
|
}
|
|
|
|
_wake.Set();
|
|
}
|
|
|
|
private static void Busy(string reqId, string kind)
|
|
{
|
|
var held = (DateTime.UtcNow - _inFlightSince).TotalSeconds;
|
|
|
|
var sb = BridgeJson.Begin("bridge.busy");
|
|
|
|
sb.Str("reqId", reqId)
|
|
// `busyKind`, never a second `kind` — `Begin` has already written this frame's own,
|
|
// and a JSON object carrying two makes every parser take the last. Protocol 6 shipped
|
|
// that bug once and it made the sidecar answer 200 for a refusal.
|
|
.Str("busyKind", kind)
|
|
.Num("heldForSec", (long)held)
|
|
.Str("reason", "the asset plane serves one request at a time");
|
|
|
|
BridgeLink.Emit(sb.End());
|
|
}
|
|
|
|
// ── assets.fetch, and the families behind it (§5, phase 5) ───────────────────────────
|
|
|
|
/// <summary>
|
|
/// One family's answer to a fetch. Runs on the asset worker, never the Core thread.
|
|
/// </summary>
|
|
internal delegate void FamilyFetch(string reqId, List<string> keys, string catalog, string cursor);
|
|
|
|
/// <summary>
|
|
/// One family's answer to a manifest walk — everything it can serve, no payload.
|
|
/// Runs on the asset worker, never the Core thread. A family with nothing to
|
|
/// enumerate (statics and land are addressed, not listed) registers none.
|
|
/// </summary>
|
|
internal delegate void FamilyManifest(string reqId, string cursor);
|
|
|
|
/// <summary>
|
|
/// What one §5 key family registered: how to serve it, how to list it, and — since
|
|
/// phase 7 — which operator consent it answers to.
|
|
///
|
|
/// The gate is per family rather than per plane because the planes are not one
|
|
/// consent. `body`, `static` and `land` are the operator's UO CLIENT, licensed from
|
|
/// EA and read off their disk; `tree` is the shard's OWN configuration, which they
|
|
/// wrote. An operator can reasonably want the second published and not the first, and
|
|
/// before this the atlas would have been what silently disappeared when they said so.
|
|
/// </summary>
|
|
private sealed class FamilyReader
|
|
{
|
|
public FamilyFetch Fetch;
|
|
public FamilyManifest Manifest;
|
|
public Func<bool> Enabled;
|
|
public string DisabledReason;
|
|
}
|
|
|
|
private static readonly Dictionary<string, FamilyReader> _families =
|
|
new Dictionary<string, FamilyReader>(StringComparer.Ordinal);
|
|
|
|
/// <summary>
|
|
/// Claims one §5 key family for a reader.
|
|
///
|
|
/// Phase 3 gave <c>assets.fetch</c> to the body catalogue outright, which was right
|
|
/// while there was one family and wrong the moment there were three: the command is
|
|
/// the *transport*, and the family is a property of the key. So the shared parts — the
|
|
/// correlation id, the operator's consent, the key-count ceiling, and deciding which
|
|
/// reader a request belongs to — live here once, and a family only ever sees a list of
|
|
/// keys it owns.
|
|
///
|
|
/// Registration is order-independent on purpose: ServUO calls every
|
|
/// <c>Initialize</c> in an order nothing here controls, and this fills a dictionary the
|
|
/// handler does not read until a request arrives.
|
|
/// </summary>
|
|
internal static void RegisterFamily(string name, FamilyFetch fetch)
|
|
{
|
|
RegisterFamily(name, fetch, null, null, null);
|
|
}
|
|
|
|
/// <summary>
|
|
/// The full registration: a fetch reader, an optional manifest reader, and the
|
|
/// consent this family answers to.
|
|
///
|
|
/// <paramref name="enabled"/> null means the asset plane's own gate
|
|
/// (<c>Bridge.AssetsEnabled</c>), which is what every client-file family wants.
|
|
/// A family that reads something else entirely passes its own.
|
|
/// </summary>
|
|
internal static void RegisterFamily(string name, FamilyFetch fetch, FamilyManifest manifest,
|
|
Func<bool> enabled, string disabledReason)
|
|
{
|
|
lock (_families)
|
|
{
|
|
_families[name] = new FamilyReader
|
|
{
|
|
Fetch = fetch,
|
|
Manifest = manifest,
|
|
Enabled = enabled,
|
|
DisabledReason = disabledReason
|
|
};
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// The families this shard can serve **right now**, for §6's stage 1 and for
|
|
/// diagnostics.
|
|
///
|
|
/// Filtered by consent rather than by registration, because that is the question the
|
|
/// website is actually asking: a family it can see in this list is one it can fetch.
|
|
/// Listing a family the operator has switched off would turn one clear refusal at
|
|
/// import time into a per-key refusal on every pass, forever — which is exactly the
|
|
/// failure `families` was added in phase 5 to prevent.
|
|
/// </summary>
|
|
internal static List<string> Families()
|
|
{
|
|
var names = new List<string>();
|
|
|
|
lock (_families)
|
|
{
|
|
foreach (var pair in _families)
|
|
{
|
|
if (EnabledFor(pair.Value))
|
|
names.Add(pair.Key);
|
|
}
|
|
}
|
|
|
|
names.Sort(StringComparer.Ordinal);
|
|
return names;
|
|
}
|
|
|
|
private static bool EnabledFor(FamilyReader reader)
|
|
{
|
|
if (reader == null)
|
|
return false;
|
|
|
|
try
|
|
{
|
|
return reader.Enabled == null ? BridgeConfig.AssetsEnabled : reader.Enabled();
|
|
}
|
|
catch
|
|
{
|
|
// A gate that throws is a gate that has not consented.
|
|
return false;
|
|
}
|
|
}
|
|
|
|
private static FamilyReader FamilyFor(string name)
|
|
{
|
|
lock (_families)
|
|
{
|
|
FamilyReader reader;
|
|
return _families.TryGetValue(name, out reader) ? reader : null;
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// Resolves a named family and answers the request itself when it cannot.
|
|
///
|
|
/// Shared by <c>assets.fetch</c> and <c>assets.manifest</c> so the two cannot drift
|
|
/// apart about what "this shard does not serve that" means — and so the consent check
|
|
/// happens in exactly one place for both.
|
|
/// </summary>
|
|
private static bool Resolve(string reqId, string family, out FamilyReader reader)
|
|
{
|
|
reader = FamilyFor(family);
|
|
|
|
if (reader == null)
|
|
{
|
|
Fail(reqId, "BAD_REQUEST",
|
|
"this shard serves no '" + family + "' asset family (it serves "
|
|
+ String.Join(", ", Families().ToArray()) + ")");
|
|
return false;
|
|
}
|
|
|
|
if (!EnabledFor(reader))
|
|
{
|
|
Fail(reqId, "DISABLED", reader.DisabledReason
|
|
?? "asset extraction is disabled on this shard");
|
|
return false;
|
|
}
|
|
|
|
return true;
|
|
}
|
|
|
|
/// <summary>
|
|
/// The family segment of a §5 key: everything before the first `/`.
|
|
/// </summary>
|
|
internal static string FamilyOfKey(string key)
|
|
{
|
|
if (String.IsNullOrEmpty(key))
|
|
return null;
|
|
|
|
int slash = key.IndexOf('/');
|
|
|
|
return slash <= 0 ? null : key.Substring(0, slash);
|
|
}
|
|
|
|
/// <summary>
|
|
/// §14's `assets.fetch`, for every family.
|
|
///
|
|
/// **The family is derived from the keys and is not a request field.** §5 made the key
|
|
/// the address of an asset, so a request that had to name its family as well would have
|
|
/// two places to be wrong and one of them silent. A batch must be of one family —
|
|
/// mixing them is refused rather than split — because the reply carries a single
|
|
/// `catalog` id, and that id is what stops an operator patching their client mid-import
|
|
/// from stitching one asset set out of two. Two families, two fingerprints, and a reply
|
|
/// that claimed one of them would be lying about the other.
|
|
/// </summary>
|
|
private static void OnFetch(Dictionary<string, object> o)
|
|
{
|
|
var reqId = BridgeJson.GetString(o, "reqId");
|
|
|
|
if (reqId == null)
|
|
{
|
|
Fail(null, "BAD_REQUEST", "assets.fetch requires a reqId");
|
|
return;
|
|
}
|
|
|
|
// The consent check is NOT here any more (phase 7). It cannot be: which consent this
|
|
// request needs is a property of the keys, and the keys have not been read yet. So the
|
|
// shape checks come first and the gate happens in `Resolve`, once the family is known.
|
|
var keys = BridgeJson.GetStringList(o, "keys");
|
|
|
|
if (keys.Count == 0)
|
|
{
|
|
Fail(reqId, "BAD_REQUEST", "assets.fetch requires a non-empty `keys` array");
|
|
return;
|
|
}
|
|
|
|
if (keys.Count > BridgeConfig.AssetFetchKeys)
|
|
{
|
|
Fail(reqId, "BAD_REQUEST",
|
|
"assets.fetch takes at most " + BridgeConfig.AssetFetchKeys
|
|
+ " keys per request (asked for " + keys.Count + ")");
|
|
return;
|
|
}
|
|
|
|
string family = FamilyOfKey(keys[0]);
|
|
|
|
for (int i = 1; i < keys.Count; i++)
|
|
{
|
|
if (String.Equals(FamilyOfKey(keys[i]), family, StringComparison.Ordinal))
|
|
continue;
|
|
|
|
Fail(reqId, "BAD_REQUEST",
|
|
"assets.fetch takes keys of one family per request; this one mixes '"
|
|
+ family + "' with '" + FamilyOfKey(keys[i]) + "'");
|
|
return;
|
|
}
|
|
|
|
FamilyReader reader;
|
|
|
|
if (!Resolve(reqId, family, out reader))
|
|
return;
|
|
|
|
if (reader.Fetch == null)
|
|
{
|
|
Fail(reqId, "BAD_REQUEST",
|
|
"the '" + family + "' family cannot be fetched by key on this shard");
|
|
return;
|
|
}
|
|
|
|
var catalog = BridgeJson.GetString(o, "catalog");
|
|
var cursor = BridgeJson.GetString(o, "cursor");
|
|
FamilyFetch fetch = reader.Fetch;
|
|
|
|
Accept(reqId, "assets.fetch", () => fetch(reqId, keys, catalog, cursor));
|
|
}
|
|
|
|
/// <summary>
|
|
/// §14's `assets.manifest`, for every family that has one.
|
|
///
|
|
/// Phase 3 gave this command to the body catalogue outright and phase 5 learned, for
|
|
/// `assets.fetch`, that the command is the transport and the family is a property of
|
|
/// the key. Phase 7 is where the same lesson lands one level up: the tree family
|
|
/// enumerates its files exactly the way the catalogue enumerates its bodies, and
|
|
/// nothing about the envelope, the cursor or the consent differs between them.
|
|
///
|
|
/// **`family` still defaults to `body`.** A phase-3 website asks without naming one
|
|
/// and must keep getting the catalogue it asked for.
|
|
/// </summary>
|
|
private static void OnManifest(Dictionary<string, object> o)
|
|
{
|
|
var reqId = BridgeJson.GetString(o, "reqId");
|
|
|
|
if (reqId == null)
|
|
{
|
|
Fail(null, "BAD_REQUEST", "assets.manifest requires a reqId");
|
|
return;
|
|
}
|
|
|
|
var family = BridgeJson.GetString(o, "family") ?? "body";
|
|
|
|
FamilyReader reader;
|
|
|
|
if (!Resolve(reqId, family, out reader))
|
|
return;
|
|
|
|
if (reader.Manifest == null)
|
|
{
|
|
// Named rather than defaulted: statics and land are ADDRESSED (§11.1) rather than
|
|
// listed, and a website that asked for a list of 49,152 item graphics has made a
|
|
// mistake it needs told about rather than an empty page it will read as "none".
|
|
Fail(reqId, "BAD_REQUEST",
|
|
"the '" + family + "' family is fetched by key and has no manifest");
|
|
return;
|
|
}
|
|
|
|
var cursor = BridgeJson.GetString(o, "cursor");
|
|
FamilyManifest manifest = reader.Manifest;
|
|
|
|
Accept(reqId, "assets.manifest", () => manifest(reqId, cursor));
|
|
}
|
|
|
|
/// <summary>
|
|
/// ARGB1555 to a PNG with a transparent background.
|
|
///
|
|
/// <c>Frame</c> writes 16-bit ARGB1555: a pixel the sprite does not cover is left as
|
|
/// zero and a pixel it does cover carries the top bit set. Saving that format straight
|
|
/// to PNG asks GDI+ to make the conversion, and what it does with a one-bit alpha
|
|
/// channel varies by platform — on Mono it is a different implementation entirely. A
|
|
/// sprite that came back with a black rectangle behind it would look fine in a test
|
|
/// that only checked the bytes decoded, and wrong on every page that showed it.
|
|
///
|
|
/// So the expansion is done here, explicitly: alpha bit clear becomes fully
|
|
/// transparent, and each 5-bit channel is widened to 8 bits by repeating its high bits
|
|
/// (<c>(c << 3) | (c >> 2)</c>) rather than by shifting alone, which would
|
|
/// cap white at 248 and tint the whole catalogue.
|
|
/// </summary>
|
|
internal static byte[] BitmapToPng(Bitmap source)
|
|
{
|
|
var rect = new Rectangle(0, 0, source.Width, source.Height);
|
|
|
|
if (source.PixelFormat != PixelFormat.Format16bppArgb1555)
|
|
{
|
|
// Not what this library has ever produced. Save it rather than reinterpret it:
|
|
// guessing at an unknown layout is how a catalogue fills with confident nonsense.
|
|
using (var ms = new MemoryStream())
|
|
{
|
|
source.Save(ms, ImageFormat.Png);
|
|
return ms.ToArray();
|
|
}
|
|
}
|
|
|
|
using (var target = new Bitmap(source.Width, source.Height, PixelFormat.Format32bppArgb))
|
|
{
|
|
BitmapData src = source.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format16bppArgb1555);
|
|
BitmapData dst = null;
|
|
|
|
try
|
|
{
|
|
dst = target.LockBits(rect, ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
|
|
|
|
var line = new short[source.Width];
|
|
var outLine = new int[source.Width];
|
|
|
|
for (int y = 0; y < source.Height; y++)
|
|
{
|
|
Marshal.Copy(new IntPtr(src.Scan0.ToInt64() + ((long)y * src.Stride)),
|
|
line, 0, source.Width);
|
|
|
|
for (int x = 0; x < source.Width; x++)
|
|
{
|
|
int p = line[x] & 0xFFFF;
|
|
|
|
if ((p & 0x8000) == 0)
|
|
{
|
|
outLine[x] = 0;
|
|
continue;
|
|
}
|
|
|
|
int r = (p >> 10) & 0x1F;
|
|
int g = (p >> 5) & 0x1F;
|
|
int b = p & 0x1F;
|
|
|
|
outLine[x] = unchecked((int)0xFF000000)
|
|
| (((r << 3) | (r >> 2)) << 16)
|
|
| (((g << 3) | (g >> 2)) << 8)
|
|
| ((b << 3) | (b >> 2));
|
|
}
|
|
|
|
Marshal.Copy(outLine, 0, new IntPtr(dst.Scan0.ToInt64() + ((long)y * dst.Stride)),
|
|
source.Width);
|
|
}
|
|
}
|
|
finally
|
|
{
|
|
if (dst != null)
|
|
target.UnlockBits(dst);
|
|
|
|
source.UnlockBits(src);
|
|
}
|
|
|
|
using (var ms = new MemoryStream())
|
|
{
|
|
target.Save(ms, ImageFormat.Png);
|
|
return ms.ToArray();
|
|
}
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// Does a caller's asserted catalog id disagree with what this shard holds?
|
|
///
|
|
/// **An absent fingerprint and an empty one mean the same thing**, and that is the
|
|
/// whole reason this is a function rather than an inline `expected != null`. A caller
|
|
/// with nothing to assert sends the field absent or empty depending on how its own
|
|
/// client serialises a missing value, and treating `""` as a real id refuses **every**
|
|
/// fetch it makes — with a sentence naming no catalog at all ("catalog is now
|
|
/// 8159778b"), which reads as a shard fault rather than a caller one.
|
|
///
|
|
/// Phase 7 found this on the tree family, where a probe passed an empty string by
|
|
/// accident. It was inline in three places by then; it is one function now, because
|
|
/// three copies of a comparison are three chances for the next family to get it wrong
|
|
/// in a way only a differently-written client would ever reveal.
|
|
///
|
|
/// Note this is deliberately NOT the shape `BridgeLeases` uses for its own `expected`:
|
|
/// there the value is a world property, where an empty string is a legitimate thing to
|
|
/// expect and `!= null` is correct.
|
|
/// </summary>
|
|
internal static bool CatalogMismatch(string expected, string actual)
|
|
{
|
|
return !String.IsNullOrEmpty(expected) && !String.Equals(expected, actual, StringComparison.Ordinal);
|
|
}
|
|
|
|
/// <summary>
|
|
/// SHA-256, lowercase hex. Shared because the hash in a manifest row, the hash in a
|
|
/// fetch row and the hash the website stores must be one function.
|
|
/// </summary>
|
|
internal static string Sha256Hex(byte[] bytes)
|
|
{
|
|
using (var sha = SHA256.Create())
|
|
{
|
|
byte[] digest = sha.ComputeHash(bytes);
|
|
var sb = new StringBuilder(digest.Length * 2);
|
|
|
|
foreach (byte b in digest)
|
|
sb.Append(b.ToString("x2", CultureInfo.InvariantCulture));
|
|
|
|
return sb.ToString();
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// The asset plane's one refusal frame, shared by every family on it.
|
|
///
|
|
/// <paramref name="code"/> is what the sidecar maps to a status, and it exists because
|
|
/// the alternative it replaced — matching on the words in <paramref name="reason"/> —
|
|
/// makes an operator-facing sentence load-bearing. Rewording "disabled" would silently
|
|
/// turn a 403 into a 400. The codes are `DISABLED` (the operator switched this plane
|
|
/// off), `NOT_FOUND` (the shard has no such file), `UNREADABLE` (it has it and cannot
|
|
/// decode it), `UNAVAILABLE` (the shard cannot do this right now) and `BAD_REQUEST`
|
|
/// (the default, and the caller's fault).
|
|
/// </summary>
|
|
internal static void Fail(string reqId, string code, string reason)
|
|
{
|
|
var sb = BridgeJson.Begin("assets.error");
|
|
|
|
if (reqId != null)
|
|
sb.Str("reqId", reqId);
|
|
|
|
sb.Str("code", code)
|
|
.Str("reason", reason);
|
|
|
|
BridgeLink.Emit(sb.End());
|
|
}
|
|
|
|
// ── the worker ───────────────────────────────────────────────────────────────────────
|
|
|
|
/// <summary>
|
|
/// Started on first use rather than at boot, so a shard that never imports an asset
|
|
/// never carries the thread. Caller must hold <see cref="_sync"/>.
|
|
/// </summary>
|
|
private static void EnsureWorker()
|
|
{
|
|
if (_worker != null)
|
|
return;
|
|
|
|
_running = true;
|
|
|
|
_worker = new Thread(WorkLoop)
|
|
{
|
|
Name = "BridgeAssets",
|
|
IsBackground = true
|
|
};
|
|
|
|
_worker.Start();
|
|
}
|
|
|
|
private static void WorkLoop()
|
|
{
|
|
while (_running)
|
|
{
|
|
_wake.WaitOne(1000);
|
|
|
|
Action job;
|
|
|
|
lock (_sync)
|
|
{
|
|
job = _job;
|
|
_job = null;
|
|
}
|
|
|
|
if (job == null)
|
|
continue;
|
|
|
|
try
|
|
{
|
|
job();
|
|
Interlocked.Increment(ref _served);
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
// A handler that throws must still free the slot, or this plane is wedged for
|
|
// the life of the process and every later request answers `bridge.busy`.
|
|
Interlocked.Increment(ref _failed);
|
|
Console.WriteLine("[Bridge] asset worker: {0}: {1}", e.GetType().Name, e.Message);
|
|
}
|
|
finally
|
|
{
|
|
lock (_sync)
|
|
{
|
|
_inFlight = null;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// ── assets.sources ───────────────────────────────────────────────────────────────────
|
|
|
|
/// <summary>
|
|
/// The client files whose bytes decide everything downstream.
|
|
///
|
|
/// Resolved through <c>Ultima.Files</c> rather than by joining a configured directory,
|
|
/// because that is what the decoders themselves do — a file this reports is a file
|
|
/// they would actually open.
|
|
/// </summary>
|
|
private static readonly string[] SourceFiles =
|
|
{
|
|
"cliloc.enu",
|
|
"artlegacymul.uop", "art.mul", "artidx.mul",
|
|
"anim.idx", "anim.mul",
|
|
"anim2.idx", "anim2.mul",
|
|
"anim3.idx", "anim3.mul",
|
|
"anim4.idx", "anim4.mul",
|
|
"anim5.idx", "anim5.mul",
|
|
"animationframe1.uop", "animationframe2.uop", "animationframe3.uop",
|
|
"animationframe4.uop", "animationframe6.uop",
|
|
"body.def", "bodyconv.def",
|
|
"hues.mul",
|
|
"verdata.mul"
|
|
};
|
|
|
|
private static void ReplySources(string reqId)
|
|
{
|
|
var sb = BridgeJson.Begin("assets.sources.ok");
|
|
|
|
sb.Str("reqId", reqId)
|
|
.Num("extractorVersion", EXTRACTOR_VERSION)
|
|
// Which of the two consents this shard has given (phase 7). Without it a website
|
|
// whose operator switched client-file extraction off would read an empty `files`
|
|
// array as "your client has no cliloc.enu" — a sentence that sends them looking at
|
|
// their client install for a setting that lives on their shard.
|
|
.Bool("assetsEnabled", BridgeConfig.AssetsEnabled)
|
|
.Bool("treeEnabled", BridgeConfig.TreeEnabled);
|
|
|
|
WriteImaging(sb);
|
|
|
|
// §4.6: whichever of art.mul / artLegacyMUL.uop `FileIndex` would actually open. An
|
|
// operator who added custom graphics to art.mul while the UOP is present is getting
|
|
// nothing, silently, and this is the only place that can tell them so.
|
|
string artData = BridgeAssetValidator.ArtDataPath();
|
|
|
|
sb.Str("artDataFile", artData == null ? null : Path.GetFileName(artData));
|
|
|
|
// Which §5 families this shard can be asked for. Additive, so the protocol stays 8: a
|
|
// consumer that does not read it behaves exactly as it did. One that does can tell an
|
|
// older overlay (bodies only) from this one without discovering it as a refused fetch
|
|
// halfway through a warm pass.
|
|
var families = Families();
|
|
|
|
sb.Append(",\"families\":[");
|
|
|
|
for (int i = 0; i < families.Count; i++)
|
|
{
|
|
if (i > 0)
|
|
sb.Append(',');
|
|
|
|
BridgeJson.Text(sb, families[i]);
|
|
}
|
|
|
|
sb.Append(']');
|
|
|
|
var page = new PageBuilder(sb, "files", BridgeConfig.AssetBatchBytes);
|
|
bool anyMissingHash = false;
|
|
|
|
// The client files are the asset plane's own subject, so they are listed under the
|
|
// asset plane's own consent. A tree-only shard answers this call — that is how its
|
|
// website learns the `tree` family exists — and reports no client files at all,
|
|
// which is the truthful answer to "what may I read here".
|
|
for (int i = 0; BridgeConfig.AssetsEnabled && i < SourceFiles.Length; i++)
|
|
{
|
|
string name = SourceFiles[i];
|
|
string path = ResolvePath(name);
|
|
|
|
if (path == null)
|
|
continue;
|
|
|
|
var item = new StringBuilder(256);
|
|
|
|
item.Append("{\"name\":");
|
|
BridgeJson.Text(item, name);
|
|
item.Append(",\"path\":");
|
|
BridgeJson.Text(item, path);
|
|
|
|
long size = 0, mtime = 0;
|
|
|
|
try
|
|
{
|
|
var info = new FileInfo(path);
|
|
size = info.Length;
|
|
mtime = ToUnixMs(info.LastWriteTimeUtc);
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
item.Append(",\"unreadable\":");
|
|
BridgeJson.Text(item, e.GetType().Name);
|
|
}
|
|
|
|
item.Append(",\"size\":").Append(size.ToString(CultureInfo.InvariantCulture));
|
|
item.Append(",\"mtime\":").Append(mtime.ToString(CultureInfo.InvariantCulture));
|
|
|
|
string hash = CachedHashFor(path, size, mtime);
|
|
|
|
if (hash == null)
|
|
anyMissingHash = true;
|
|
|
|
item.Append(",\"sha256\":");
|
|
BridgeJson.Text(item, hash);
|
|
|
|
// The one diagnostic §4.6 asks for: art.mul is present, and unread.
|
|
if (artData != null
|
|
&& (name == "art.mul" || name == "artidx.mul")
|
|
&& !artData.EndsWith(".mul", StringComparison.OrdinalIgnoreCase))
|
|
{
|
|
item.Append(",\"shadowedBy\":");
|
|
BridgeJson.Text(item, Path.GetFileName(artData));
|
|
}
|
|
|
|
item.Append('}');
|
|
|
|
if (!page.TryAdd(item.ToString(), name))
|
|
break;
|
|
}
|
|
|
|
page.Close();
|
|
|
|
// §6's gate is (size, mtime) first and a content hash only when those differ, because
|
|
// anim.mul and art.mul are 195 MB and 148 MB and a full hash on every status poll
|
|
// would make the admin panel feel broken. It would also blow the sidecar's 10 s reply
|
|
// timeout outright on the first call. So a hash that is not cached is reported `null`
|
|
// and computed in the background: this reply is always fast, and the next poll — after
|
|
// `hashing` goes false — carries the answer.
|
|
if (anyMissingHash)
|
|
StartHashing();
|
|
|
|
sb.Bool("hashing", _hashing);
|
|
sb.Bool("complete", !anyMissingHash);
|
|
|
|
BridgeLink.Emit(sb.End());
|
|
}
|
|
|
|
private static string ResolvePath(string name)
|
|
{
|
|
try
|
|
{
|
|
string path = Files.GetFilePath(name);
|
|
|
|
if (path != null)
|
|
return path;
|
|
}
|
|
catch
|
|
{
|
|
// Ultima's lookup reads the registry on Windows; a host where that throws still
|
|
// has the directories ServUO itself booted from, which is what the fallback uses.
|
|
}
|
|
|
|
// `Ultima.Files` has a fixed table of file names that predates UOP animations, so it
|
|
// answers null for every `AnimationFrame*.uop` however present they are (§4.3). Phase
|
|
// 4 added those to this list, so the fallback is what makes their size, mtime and hash
|
|
// reachable at all.
|
|
return BridgeUop.FindClientFile(name);
|
|
}
|
|
|
|
private static long ToUnixMs(DateTime utc)
|
|
{
|
|
return (long)(utc - new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalMilliseconds;
|
|
}
|
|
|
|
// ── imaging (§4.4) ───────────────────────────────────────────────────────────────────
|
|
|
|
/// <summary>
|
|
/// Whether this host can turn a record into a picture at all.
|
|
///
|
|
/// ServUO targets net48, so a Linux shard runs it under Mono, where
|
|
/// <c>System.Drawing</c> is a thin layer over **libgdiplus** — and §4.2 put
|
|
/// <c>System.Drawing</c> in the *decode* path, not merely the encode: <c>Frame</c>
|
|
/// writes ARGB1555 through a <c>LockBits</c> pointer. Without that library a Linux
|
|
/// shard cannot read a sprite at all, while clilocs and the atlas are unaffected
|
|
/// because neither touches pixels.
|
|
///
|
|
/// It must never present as a stack trace or a 500. It is a named, actionable outcome
|
|
/// in the same family as the cliloc reader's `COMPRESSED`, and it is reported here —
|
|
/// on the *source gate*, the first call any import makes — so an operator learns it
|
|
/// while setting the shard up rather than from an empty bestiary weeks later.
|
|
/// </summary>
|
|
private static void WriteImaging(StringBuilder sb)
|
|
{
|
|
CheckImaging();
|
|
|
|
sb.Append(",\"imaging\":{\"ok\":").Append(_imagingOk ? "true" : "false");
|
|
|
|
if (!_imagingOk)
|
|
{
|
|
sb.Append(",\"code\":\"NO_IMAGING\",\"reason\":");
|
|
BridgeJson.Text(sb,
|
|
"This shard host cannot render images — Mono's System.Drawing needs libgdiplus. "
|
|
+ "Install it (apt-get install libgdiplus) and re-run the import. Cliloc and atlas "
|
|
+ "import are unaffected. (" + _imagingReason + ")");
|
|
}
|
|
|
|
sb.Append('}');
|
|
}
|
|
|
|
/// <summary>
|
|
/// Whether this host can produce a picture, for the families that produce pictures.
|
|
///
|
|
/// <see cref="WriteImaging"/> reports this on the source gate so an operator learns it
|
|
/// while setting the shard up. The catalogue needs the same answer as a *decision* —
|
|
/// it must refuse rather than throw a <c>DllNotFoundException</c> out of the middle of
|
|
/// a decode loop — so the check itself is shared and this is its one accessor.
|
|
/// </summary>
|
|
internal static bool ImagingOk(out string reason)
|
|
{
|
|
CheckImaging();
|
|
reason = _imagingReason;
|
|
return _imagingOk;
|
|
}
|
|
|
|
private static void CheckImaging()
|
|
{
|
|
if (_imagingChecked)
|
|
return;
|
|
|
|
_imagingChecked = true;
|
|
|
|
try
|
|
{
|
|
TouchImaging();
|
|
_imagingOk = true;
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
// On a host with no libgdiplus this is a TypeInitializationException wrapping a
|
|
// DllNotFoundException, and it can surface as the method failing to JIT rather
|
|
// than as a throw from inside it — which is why the construction lives in its own
|
|
// method, so the failure is contained here instead of taking this class's
|
|
// static initialisation with it.
|
|
_imagingOk = false;
|
|
_imagingReason = e.GetType().Name + ": " + e.Message;
|
|
}
|
|
}
|
|
|
|
private static void TouchImaging()
|
|
{
|
|
using (var bmp = new System.Drawing.Bitmap(1, 1))
|
|
{
|
|
bmp.SetPixel(0, 0, System.Drawing.Color.Black);
|
|
}
|
|
}
|
|
|
|
// ── the hash cache (§6) ──────────────────────────────────────────────────────────────
|
|
|
|
private static string CachedHashFor(string path, long size, long mtime)
|
|
{
|
|
lock (_hashes)
|
|
{
|
|
CachedHash cached;
|
|
|
|
if (_hashes.TryGetValue(path, out cached)
|
|
&& cached.Size == size
|
|
&& cached.MTime == mtime)
|
|
{
|
|
return cached.Sha256;
|
|
}
|
|
}
|
|
|
|
return null;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Rehashes whatever the cache is missing, on its own thread.
|
|
///
|
|
/// Deliberately **not** a job on the asset worker: hashing 343 MB takes seconds to
|
|
/// tens of seconds, and holding the single slot for that long would answer every
|
|
/// status poll `bridge.busy` for the whole pass — which is exactly the moment an
|
|
/// operator is watching the panel. It emits nothing and correlates with nothing; it
|
|
/// only fills the cache that the next `assets.sources` reads.
|
|
/// </summary>
|
|
private static void StartHashing()
|
|
{
|
|
lock (_sync)
|
|
{
|
|
if (_hashing)
|
|
return;
|
|
|
|
_hashing = true;
|
|
|
|
_hasher = new Thread(HashLoop)
|
|
{
|
|
Name = "BridgeAssetHash",
|
|
IsBackground = true
|
|
};
|
|
|
|
_hasher.Start();
|
|
}
|
|
}
|
|
|
|
private static void HashLoop()
|
|
{
|
|
try
|
|
{
|
|
for (int i = 0; i < SourceFiles.Length; i++)
|
|
{
|
|
string path = ResolvePath(SourceFiles[i]);
|
|
|
|
if (path == null)
|
|
continue;
|
|
|
|
long size, mtime;
|
|
|
|
try
|
|
{
|
|
var info = new FileInfo(path);
|
|
size = info.Length;
|
|
mtime = ToUnixMs(info.LastWriteTimeUtc);
|
|
}
|
|
catch
|
|
{
|
|
continue;
|
|
}
|
|
|
|
if (CachedHashFor(path, size, mtime) != null)
|
|
continue;
|
|
|
|
string hash = HashFile(path);
|
|
|
|
if (hash == null)
|
|
continue;
|
|
|
|
lock (_hashes)
|
|
{
|
|
_hashes[path] = new CachedHash { Size = size, MTime = mtime, Sha256 = hash };
|
|
}
|
|
}
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
Console.WriteLine("[Bridge] asset hash pass: {0}: {1}", e.GetType().Name, e.Message);
|
|
}
|
|
finally
|
|
{
|
|
// Under _sync, matching StartHashing: cleared outside it, two passes could both
|
|
// pass the guard and hash the same 343 MB twice.
|
|
lock (_sync)
|
|
{
|
|
_hashing = false;
|
|
}
|
|
}
|
|
}
|
|
|
|
private static string HashFile(string path)
|
|
{
|
|
try
|
|
{
|
|
using (var sha = SHA256.Create())
|
|
using (var stream = new FileStream(
|
|
path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 1 << 20))
|
|
{
|
|
var buffer = new byte[1 << 20];
|
|
int read;
|
|
|
|
while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
|
|
sha.TransformBlock(buffer, 0, read, null, 0);
|
|
|
|
sha.TransformFinalBlock(buffer, 0, 0);
|
|
|
|
return Hex(sha.Hash);
|
|
}
|
|
}
|
|
catch (Exception e)
|
|
{
|
|
Console.WriteLine("[Bridge] cannot hash {0}: {1}", path, e.Message);
|
|
return null;
|
|
}
|
|
}
|
|
|
|
private static string Hex(byte[] bytes)
|
|
{
|
|
var sb = new StringBuilder(bytes.Length * 2);
|
|
|
|
for (int i = 0; i < bytes.Length; i++)
|
|
sb.Append(bytes[i].ToString("x2", CultureInfo.InvariantCulture));
|
|
|
|
return sb.ToString();
|
|
}
|
|
|
|
// ── the paging envelope (§3.3) ───────────────────────────────────────────────────────
|
|
|
|
/// <summary>
|
|
/// **One envelope for every asset family**, defined here in phase 1 so that clilocs
|
|
/// (phase 2), the body catalogue (3), statics and land (5), deep animation keys (6)
|
|
/// and the ServUO tree files (7) all page the same way. They are otherwise five
|
|
/// chances to invent five slightly different shapes, and the website would have to
|
|
/// learn each one.
|
|
///
|
|
/// The envelope a reply closes with:
|
|
///
|
|
/// <code>
|
|
/// "items": [ … ],
|
|
/// "more": true, // ask again with this cursor
|
|
/// "cursor": "s:4104", // opaque to everyone but the shard; absent when more:false
|
|
/// "cut": "budget" // budget | end | limit — WHY this page stopped
|
|
/// </code>
|
|
///
|
|
/// **The budget is bytes, and it is UTF-8 bytes.** Not item count, because the ceiling
|
|
/// this lives inside is the sidecar's inbound line cap; and not chars, because a
|
|
/// cliloc row is real text and a `StringBuilder`'s Length would undercount every
|
|
/// non-ASCII character in it.
|
|
///
|
|
/// `cut` exists because "the page is short" has three different meanings and the
|
|
/// website must not have to guess which: the source ran out (`end`), the byte budget
|
|
/// was spent (`budget`), or the family stopped at its own limit (`limit`). Only the
|
|
/// first means the import is finished.
|
|
///
|
|
/// **The first item is always admitted**, even if it alone exceeds the budget.
|
|
/// Otherwise an oversized item would make its family unable to make any progress at
|
|
/// all — it would be skipped for the budget on every page, forever. That is safe
|
|
/// precisely because the budget is set to half the sidecar's line cap
|
|
/// (<see cref="BridgeConfig.AssetBatchBytes"/>), so one such item still fits the wire.
|
|
/// </summary>
|
|
public sealed class PageBuilder
|
|
{
|
|
private readonly StringBuilder _sb;
|
|
private readonly int _budget;
|
|
private int _bytes;
|
|
private int _count;
|
|
private string _cursor;
|
|
private string _cut = "end";
|
|
|
|
/// <summary>
|
|
/// Room kept back for the fields the envelope must still be able to write after
|
|
/// the last item — `more`, `cursor`, `cut` and the closing brace. Without it a
|
|
/// page could fill the budget exactly and then overrun it closing itself.
|
|
/// </summary>
|
|
private const int Reserve = 256;
|
|
|
|
public PageBuilder(StringBuilder sb, string arrayName, int budget)
|
|
{
|
|
_sb = sb;
|
|
_budget = budget;
|
|
|
|
sb.Append(",\"").Append(arrayName).Append("\":[");
|
|
|
|
// The prefix is already written, and it counts: the cap the sidecar enforces is
|
|
// on the whole line, not on the array.
|
|
_bytes = Encoding.UTF8.GetByteCount(sb.ToString());
|
|
}
|
|
|
|
public int Count { get { return _count; } }
|
|
|
|
/// <summary>
|
|
/// Adds one already-serialised item. <paramref name="cursorAfter"/> is where the
|
|
/// family should resume if this turns out to be the last item on the page.
|
|
/// Returns false when the budget is spent — the caller stops, and `more` is true.
|
|
/// </summary>
|
|
public bool TryAdd(string item, string cursorAfter)
|
|
{
|
|
if (item == null)
|
|
return true;
|
|
|
|
int cost = Encoding.UTF8.GetByteCount(item) + (_count > 0 ? 1 : 0);
|
|
|
|
if (_count > 0 && _bytes + cost + Reserve > _budget)
|
|
{
|
|
_cut = "budget";
|
|
return false;
|
|
}
|
|
|
|
if (_count > 0)
|
|
_sb.Append(',');
|
|
|
|
_sb.Append(item);
|
|
|
|
_bytes += cost;
|
|
_count++;
|
|
_cursor = cursorAfter;
|
|
|
|
return true;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Stops the page for a reason of the family's own — a per-request limit, say —
|
|
/// rather than because the budget ran out.
|
|
/// </summary>
|
|
public void Cut(string why)
|
|
{
|
|
_cut = why;
|
|
}
|
|
|
|
public void Close()
|
|
{
|
|
bool more = _cut != "end";
|
|
|
|
_sb.Append(']');
|
|
_sb.Bool("more", more);
|
|
_sb.Str("cut", _cut);
|
|
|
|
if (more && _cursor != null)
|
|
_sb.Str("cursor", _cursor);
|
|
}
|
|
}
|
|
}
|
|
}
|