servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeCatalog.cs

using System;
using System.Collections.Generic;
using System.Drawing;
using System.Globalization;
using System.IO;
using System.Text;

using Ultima;

namespace Server.Custom.Bridge
{
    /// <summary>
    ///     **The body catalogue** (docs/link/v8.md §4.8, §5, §6, §11 — protocol 8, phase 3).
    ///
    ///     One thumbnail per creature body: the working set that makes a bestiary, a marketplace
    ///     listing and a character sheet render. Everything deeper — every action, every frame —
    ///     is the same addressing scheme at a deeper key and is **not served** (§11.2, phase 6:
    ///     the site shows still pictures, so frames wait for a consumer that wants them). This is
    ///     the set that is worth importing before anything asks for it, because on this machine's
    ///     client it is **1,096 sprites at about a kilobyte each** — 787 out of the legacy
    ///     `anim*.mul` files, 235 more out of `AnimationFrame*.uop` since phase 4 (§4.3, §4.9),
    ///     and 74 more since phase 6, which have no art at action 0 and real art deeper.
    ///
    ///     ── **One picture per body, at the first action that has one** ──
    ///
    ///     A key carries the action it came from — `body/820/a23` for a horse whose action 0 is
    ///     empty — so the catalogue is still exactly one row per body, and the row says which
    ///     picture it is. What it never does is decode an action the walk did not choose: a fetch
    ///     for `body/820/a0` is `unsupported`, not a second attempt, because the slots past a
    ///     body's band belong to the next body and every check passes on them (§4.10).
    ///
    ///     Two request kinds, which are §6's two stages for assets rather than for sources:
    ///
    ///     <c>assets.manifest</c> — `[{ key, sha256, bytes, width, height }]`, no pixels. The
    ///     website diffs it against what it already holds and asks only for what changed. That is
    ///     the whole difference between an Update and a re-download.
    ///
    ///     <c>assets.fetch</c> — the pixels, for an explicit list of keys.
    ///
    ///     ── **Why the manifest builds the pictures it refuses to send** ──
    ///
    ///     A manifest row carries a hash of the bytes, and the only way to hash bytes is to have
    ///     them. So the scan decodes, encodes to PNG and hashes, then sends the row and **keeps
    ///     the bytes** — a megabyte for the whole catalogue, against re-decoding all 787 sprites a
    ///     second time when the fetch arrives moments later.
    ///
    ///     ── **Why the manifest pages on TIME rather than on bytes** ──
    ///
    ///     Every other family on this plane pages because its rows are large. This one's rows are
    ///     ninety bytes and the whole catalogue is one page by the byte budget — but producing
    ///     that page means decoding 787 animations, and the sidecar gives a reply ten seconds
    ///     (§3.3). So the scan carries a **wall-clock budget** as well
    ///     (<see cref="BridgeConfig.AssetScanMs"/>) and cuts the page `limit` when it is spent,
    ///     resuming from its cursor on the next call. The byte budget is still enforced, because
    ///     the day a family's rows grow is not the day to discover only one of the two bounds was
    ///     real.
    ///
    ///     ── **Why `catalog` is derived from the sources and not minted per build** ──
    ///
    ///     A manifest walk and the fetch that follows it must be talking about the same client
    ///     files, or the website stitches one catalogue out of two. The obvious answer is a fresh
    ///     id per build, and it is wrong: this cache is released when it goes idle, so a rebuild
    ///     halfway through a slow import would change the id and force a restart although nothing
    ///     about the client moved. So the id is a hash of what actually decides the bytes — every
    ///     anim file's size and mtime, both direction settings and
    ///     <see cref="BridgeAssets.EXTRACTOR_VERSION"/>. It is stable across a rebuild and it
    ///     changes exactly when an operator patches their client.
    ///
    ///     ── **The never-sweep rule, and the 357** ──
    ///
    ///     Nothing here asks a file type for an index it does not own, and nothing here trusts the
    ///     library's own success. <see cref="BridgeAssetValidator.ResolveAnimation"/> takes
    ///     `BodyConverter.Convert`'s answer and reports nothing if it leads nowhere (sweeping
    ///     instead puts a giant spider on the gargoyle page, decoding cleanly); and every body is
    ///     put through <see cref="BridgeAssetValidator.CheckEntry"/> and
    ///     <c>RecordReader.AnimationSane</c> **before** it is decoded, because a body whose index
    ///     entry reads `length 0` gets a bitmap back anyway — the previously-decoded creature's,
    ///     from the library's reused stream buffer. That is 357 of the 1,144 bodies the library
    ///     claims on a stock client, and importing them would have written 357 duplicate
    ///     portraits whose subject depended on the order this walk happened to run in.
    ///
    ///     ── **The UOP fallback, and why it cannot reintroduce that** ──
    ///
    ///     Phase 4 added <see cref="BridgeUop"/> beneath the legacy reader: a body the vendored
    ///     path has nothing for is looked for in the UOP packages before it is reported absent.
    ///     That is where two of the six player-character bodies live — `Bodyconv.def` sends
    ///     gargoyles 666 and 667 to `anim5`, at an index past the end of `anim5.idx` — and 233
    ///     other bodies besides. It cannot produce a wrong picture the way a legacy sweep would,
    ///     because a UOP entry is addressed by the hash of a name that contains the body id and
    ///     the payload then declares that id again, which
    ///     <see cref="BridgeUop.Group.TryOpen"/> checks.
    /// </summary>
    public static class BridgeCatalog
    {
        /// <summary>The only family this phase serves. §5's key scheme covers the rest.</summary>
        private const string Family = "body";

        /// <summary>Bodies are addressable to 2047; the sweep behind §4.8 covered exactly this.</summary>
        private const int MaxBody = 2047;

        /// <summary>
        ///     The action a thumbnail comes from when the body has one, which is nearly always.
        ///     Everything deeper than a first frame is deferred — see §11.2.
        /// </summary>
        private const int PreferredAction = 0;

        /// <summary>
        ///     How far the fallback looks for a body with no art at <see cref="PreferredAction"/>.
        ///
        ///     35 because that is the largest band any file type gives a body, so an action beyond
        ///     it is not something the client's own layout can name. The legacy arm is bounded
        ///     tighter still and per body, by <see cref="BridgeAssetValidator.ActionCount"/> —
        ///     this is only the scan's outer stop, and it is the UOP arm's real one, where an
        ///     action is a named entry rather than an offset.
        /// </summary>
        private const int MaxAction = 35;

        public static void Initialize()
        {
            if (!BridgeConfig.Enabled)
                return;

            // Both commands are shared plumbing: `assets.fetch` since phase 5 and
            // `assets.manifest` since phase 7 (§5, §10). BridgeAssets owns the correlation id, the
            // operator's consent, the key ceiling and the family decision; what is registered here
            // is only this family's two readers, and each is called on the asset worker with work
            // it owns.
            BridgeAssets.RegisterFamily(Family, ReplyFetch, ReplyManifest, null, null);
        }

        // ── the cache ────────────────────────────────────────────────────────────────────────

        private sealed class Sprite
        {
            public string Key;
            public int Body;

            /// <summary>
            ///     Which action this body's thumbnail came from — <see cref="PreferredAction"/>
            ///     for all but 74 bodies on this client, and on the wire because the key names it
            ///     (§5, §11.2). A consumer that assumes `a0` would build a dead URL for a horse.
            /// </summary>
            public int Action;

            public int Direction;
            public int FileType;
            public string Sha256;
            public byte[] Png;
            public int Width;
            public int Height;

            /// <summary>
            ///     Which reader produced it: `legacy` for ServUO's vendored <c>Animations</c> over
            ///     <c>anim*.mul</c>, `uop` for phase 4's own reader over
            ///     <c>AnimationFrame*.uop</c> (§4.3, §4.9). On the wire so that an operator
            ///     looking at a wrong picture can tell which half of the extractor to doubt, and
            ///     so the acceptance walk can prove the fallback fired at all.
            /// </summary>
            public string Source;
        }

        private sealed class Catalog
        {
            public string Id;

            /// <summary>
            ///     Keyed by **body**, not by asset key, since phase 6: a body's key now carries
            ///     the action its picture came from, so the key cannot be spelled until the body
            ///     has been resolved. A fetch arrives holding a key and has to reach the same
            ///     sprite, which it does by parsing the body out of it and comparing.
            /// </summary>
            public readonly Dictionary<int, Sprite> ByBody = new Dictionary<int, Sprite>();

            public readonly List<Sprite> Order = new List<Sprite>();

            /// <summary>The next body the scan has yet to look at.</summary>
            public int Next = 1;

            public bool Complete;
            public DateTime LastUsed;
        }

        private static readonly object _sync = new object();
        private static Catalog _catalog;

        private static readonly TimeSpan IdleFor = TimeSpan.FromMinutes(5);

        // ── assets.manifest ──────────────────────────────────────────────────────────────────

        /// <summary>
        ///     Worker thread. Scans forward from the cursor until the byte budget or the time
        ///     budget is spent, hashing what it decodes and keeping the bytes for the fetch.
        /// </summary>
        private static void ReplyManifest(string reqId, string cursor)
        {
            string imagingReason;

            if (!BridgeAssets.ImagingOk(out imagingReason))
            {
                // Never a stack trace and never a 500: on a Linux host without libgdiplus this is
                // the expected outcome, and it is actionable in one line (§4.4).
                BridgeAssets.Fail(reqId, "UNAVAILABLE",
                    "this shard host cannot render images - Mono's System.Drawing needs "
                    + "libgdiplus. Install it (apt-get install libgdiplus) and re-run the import. "
                    + "Cliloc and atlas import are unaffected. (" + imagingReason + ")");
                return;
            }

            string id = SourceId();

            Catalog catalog;

            lock (_sync)
            {
                if (_catalog == null || _catalog.Id != id)
                    _catalog = new Catalog { Id = id };

                catalog = _catalog;
                catalog.LastUsed = DateTime.UtcNow;
            }

            int from = ParseBodyCursor(cursor);

            var sb = BridgeJson.Begin("assets.manifest.ok");

            sb.Str("reqId", reqId)
              .Str("family", Family)
              // What the website compares across pages, and across the fetch that follows. A
              // change means the operator patched their client mid-import and the half already
              // read describes files that no longer exist.
              .Str("catalog", catalog.Id)
              .Num("extractorVersion", BridgeAssets.EXTRACTOR_VERSION)
              .Num("maxBody", MaxBody)
              .Num("from", from);

            WritePlayerBodies(sb);

            var page = new BridgeAssets.PageBuilder(sb, "rows", BridgeConfig.AssetBatchBytes);

            int scanned = 0;
            int last = from - 1;
            bool timedOut = false;
            bool budgetCut = false;

            var deadline = DateTime.UtcNow.AddMilliseconds(BridgeConfig.AssetScanMs);

            using (var readers = new Readers())
            {
                int body = from;

                for (; body <= MaxBody; body++)
                {
                    // Checked before the body rather than after it, so the budget bounds the reply
                    // rather than the reply plus one more decode. One sprite is milliseconds; the
                    // ceiling this lives under is ten seconds and the cost of overshooting it is
                    // the whole page, retried.
                    if (body > from && DateTime.UtcNow >= deadline)
                    {
                        timedOut = true;
                        break;
                    }

                    scanned++;
                    last = body;

                    Sprite sprite = Resolve(catalog, readers, body);

                    if (sprite == null)
                        continue;

                    var item = new StringBuilder(128);

                    item.Append("{\"key\":");
                    BridgeJson.Text(item, sprite.Key);
                    item.Append(",\"sha256\":\"").Append(sprite.Sha256).Append('"');
                    item.Append(",\"bytes\":").Append(sprite.Png.Length.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture));
                    item.Append(",\"source\":\"").Append(sprite.Source).Append('"');
                    item.Append('}');

                    if (!page.TryAdd(item.ToString(), "b:" + body.ToString(CultureInfo.InvariantCulture)))
                    {
                        // The budget stopped this page BEFORE this body's row went on it, so the
                        // next page must resume AT this body rather than after it. Getting this
                        // one line wrong drops exactly one creature from the catalogue per page,
                        // which nothing downstream could ever notice.
                        budgetCut = true;
                        last = body - 1;
                        scanned--;
                        break;
                    }
                }
            }

            if (timedOut)
                page.Cut("limit");

            // The walk reached the end of the addressable range without either budget stopping it.
            // Derived from the two flags rather than from the row count, because a page that ends
            // exactly on a boundary is indistinguishable from a finished one by count alone —
            // §3.4's whole argument for `cut` existing.
            bool finished = !timedOut && !budgetCut && last >= MaxBody;

            int held;

            lock (_sync)
            {
                if (_catalog == catalog)
                {
                    catalog.Next = Math.Max(catalog.Next, last + 1);
                    catalog.LastUsed = DateTime.UtcNow;

                    if (finished)
                        catalog.Complete = true;
                }

                held = catalog.Order.Count;
            }

            page.Close();

            // Past Close(), which is normally the mistake BridgeCliloc's `from` comment warns
            // about — but these three are not knowable until the scan has run, and they cost
            // about fifty bytes against PageBuilder's 256-byte reserve, of which Close() itself
            // spends around forty. Anything larger than this belongs before the page opens.
            sb.Num("scanned", scanned)
              .Num("held", held)
              .Bool("complete", finished);

            BridgeLink.Emit(sb.End());

            Sweep();
        }

        // ── assets.fetch ─────────────────────────────────────────────────────────────────────

        /// <summary>
        ///     The `body` family's half of <c>assets.fetch</c>. The correlation id, the operator's
        ///     consent, the key ceiling and the family decision have already been made by
        ///     <see cref="BridgeAssets.RegisterFamily"/>'s caller; every key here is this family's.
        /// </summary>
        private static void ReplyFetch(string reqId, List<string> keys, string expected, string cursor)
        {
            string imagingReason;

            if (!BridgeAssets.ImagingOk(out imagingReason))
            {
                BridgeAssets.Fail(reqId, "UNAVAILABLE",
                    "this shard host cannot render images - Mono's System.Drawing needs "
                    + "libgdiplus. (" + imagingReason + ")");
                return;
            }

            string id = SourceId();

            if (expected != null && expected != id)
            {
                // The client files moved between the manifest and this fetch. Refusing is the only
                // honest answer: the keys were chosen against a catalogue that no longer describes
                // what is on disk, and serving them would mix two clients in one import with no
                // error anywhere.
                BridgeAssets.Fail(reqId, "UNREADABLE",
                    "the shard's client files changed since that manifest was read (catalog "
                    + expected + " is now " + id + "); start the import again");
                return;
            }

            Catalog catalog;

            lock (_sync)
            {
                if (_catalog == null || _catalog.Id != id)
                    _catalog = new Catalog { Id = id };

                catalog = _catalog;
                catalog.LastUsed = DateTime.UtcNow;
            }

            int from = ParseKeyCursor(cursor);

            var sb = BridgeJson.Begin("assets.fetch.ok");

            sb.Str("reqId", reqId)
              .Str("family", Family)
              .Str("catalog", catalog.Id)
              .Num("extractorVersion", BridgeAssets.EXTRACTOR_VERSION)
              .Num("asked", keys.Count)
              .Num("from", from);

            var page = new BridgeAssets.PageBuilder(sb, "rows", BridgeConfig.AssetBatchBytes);

            int i = from;

            using (var readers = new Readers())
            {
                for (; i < keys.Count; i++)
                {
                    var item = Render(catalog, readers, keys[i]);

                    if (!page.TryAdd(item, "k:" + (i + 1).ToString(CultureInfo.InvariantCulture)))
                        break;
                }
            }

            page.Close();

            sb.Num("sent", page.Count);

            BridgeLink.Emit(sb.End());

            Sweep();
        }

        /// <summary>
        ///     One key to one row, with the bytes.
        ///
        ///     A key this shard cannot serve is a **row**, not a failed request: the website asked
        ///     for a list, and one key naming a body whose art this client does not carry must not
        ///     cost the other four hundred. `status` distinguishes the two ways that happens —
        ///     `absent` (this client has no art at that key, the expected answer for two thirds of
        ///     the player bodies) and `unsupported` (a key shape this phase does not serve, which
        ///     is a website bug rather than a client gap).
        /// </summary>
        private static string Render(Catalog catalog, Readers readers, string key)
        {
            int body, action;

            if (!TryParseKey(key, out body, out action))
            {
                var bad = new StringBuilder(96);
                bad.Append("{\"key\":");
                BridgeJson.Text(bad, key);
                bad.Append(",\"status\":\"unsupported\"}");
                return bad.ToString();
            }

            Sprite sprite = Resolve(catalog, readers, body);

            var item = new StringBuilder(2048);

            item.Append("{\"key\":");
            BridgeJson.Text(item, key);

            if (sprite == null)
            {
                item.Append(",\"status\":\"absent\"}");
                return item.ToString();
            }

            if (sprite.Action != action)
            {
                // The body has a picture, but not at the action this key names. Two ways to get
                // here and both are the caller's: an old manifest that catalogued this body at
                // `a0` before the client was patched, or a key someone built by assuming the
                // action. Neither is served — decoding the asked-for action instead would be
                // §4.10's wrong picture, arrived at politely.
                item.Append(",\"status\":\"unsupported\"");
                item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
                item.Append('}');
                return item.ToString();
            }

            item.Append(",\"status\":\"ok\"");
            item.Append(",\"sha256\":\"").Append(sprite.Sha256).Append('"');
            item.Append(",\"bytes\":").Append(sprite.Png.Length.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture));
            item.Append(",\"source\":\"").Append(sprite.Source).Append('"');
            item.Append(",\"png\":\"").Append(Convert.ToBase64String(sprite.Png)).Append("\"}");

            return item.ToString();
        }

        // ── decode ───────────────────────────────────────────────────────────────────────────

        /// <summary>
        ///     The catalogue entry for one body, decoded and hashed on first sight and cached
        ///     after. Returns null when this client has no art for it — which is an ordinary
        ///     answer for well over half of the addressable range, not a failure.
        /// </summary>
        private static Sprite Resolve(Catalog catalog, Readers readers, int body)
        {
            lock (_sync)
            {
                Sprite cached;

                if (catalog.ByBody.TryGetValue(body, out cached))
                    return cached;
            }

            int direction = IsPlayerBody(body)
                ? BridgeConfig.AssetPlayerDirection
                : BridgeConfig.AssetCreatureDirection;

            Sprite sprite = ResolveAny(readers, body, direction);

            if (sprite == null)
                return null;

            lock (_sync)
            {
                if (!catalog.ByBody.ContainsKey(body))
                {
                    catalog.ByBody[body] = sprite;
                    catalog.Order.Add(sprite);
                }

                return catalog.ByBody[body];
            }
        }

        /// <summary>
        ///     One body's thumbnail: action 0 if it has one, otherwise the first action that does.
        ///
        ///     ── **Why there is a fallback at all** ──
        ///
        ///     Through phase 5 a body with no art at action 0 was simply absent, and on this
        ///     client **74 bodies are in exactly that state while carrying real art deeper** — 66
        ///     of them UOP, 8 legacy. Body 820's first drawn action is 23, and it is a horse.
        ///     They rendered as text on the bestiary for want of looking one action further.
        ///
        ///     ── **Why the key says which action it is** ──
        ///
        ///     The fallback's picture is `body/820/a23`, not `body/820/a0`. Naming it `a0` would
        ///     have been fewer changes downstream and a key that lies about its content, which is
        ///     the failure this protocol keeps meeting from other directions (§4.5, §4.8, §11.1).
        ///
        ///     ── **Why the ceiling is not a detail** ──
        ///
        ///     Scanning actions is the one thing that can walk off the end of a body's slots, and
        ///     the slots immediately after a body's are the **next body's**. Measured in phase 6:
        ///     643 of 795 legacy bodies return a fully validated, correctly-sized picture one
        ///     action past their band, and 452 of those are byte-identical to body+1's action 0.
        ///     <see cref="BridgeAssetValidator.ResolveAnimation"/> refuses past the ceiling, so
        ///     this walk cannot produce one — see §4.10.
        /// </summary>
        private static Sprite ResolveAny(Readers readers, int body, int direction)
        {
            int actions, fileType;
            string reason;

            // The legacy ceiling. A body the legacy path cannot place at all still gets the UOP
            // arm below, where an action is a named entry rather than an offset into a band.
            if (!BridgeAssetValidator.ActionCount(body, out actions, out fileType, out reason))
                actions = 0;

            for (int action = PreferredAction; action < MaxAction; action++)
            {
                // Legacy first, always. The vendored decoder is what 787 of this client's bodies
                // come out of, it is what phase 3 measured, and the UOP packages hold a different
                // and mostly disjoint set (measured: of the 244 bodies they carry, 8 also have
                // legacy art). So this is a fallback rather than a choice, and no body changes
                // reader while a client sits still.
                Sprite sprite = action < actions
                    ? ResolveLegacy(Key(body, action), readers, body, action, direction)
                    : null;

                if (sprite == null)
                    sprite = ResolveUop(Key(body, action), readers, body, action, direction);

                if (sprite != null)
                    return sprite;
            }

            return null;
        }

        /// <summary>
        ///     ServUO's vendored <c>Animations</c> over <c>anim*.mul</c>, behind §4.5's validator.
        /// </summary>
        private static Sprite ResolveLegacy(string key, Readers readers, int body, int action, int direction)
        {
            int fileType, at;
            string reason;

            if (!BridgeAssetValidator.ResolveAnimation(body, action, direction,
                    out fileType, out at, out reason))
                return null;

            FileIndex index = readers.Index(fileType);

            if (index == null)
                return null;

            if (BridgeAssetValidator.CheckEntry(index, at, readers.MulLength(fileType),
                    readers.VerdataLength, out reason) != BridgeAssetValidator.Verdict.Ok)
            {
                // The `length 0` case lands here, and it is the 357. The library would hand back
                // the previously-decoded body's bitmap for every one of them.
                return null;
            }

            var reader = readers.Reader(fileType);

            if (reader == null)
                return null;

            // `maxFrames: 1` because that is what `firstFrame: true` decodes. Checking frames
            // nobody reads would invent refusals, and a checker that refuses real art is worse
            // than no checker at all.
            if (!reader.AnimationSane(index, at, 1, out reason))
                return null;

            try
            {
                return Decode(key, body, action, direction, fileType);
            }
            catch (Exception e)
            {
                Console.WriteLine("[Bridge] catalogue: body {0} action {1}: {2}: {3}",
                    body, action, e.GetType().Name, e.Message);
                return null;
            }
        }

        /// <summary>
        ///     Phase 4's own reader over <c>AnimationFrame*.uop</c> (§4.3, §4.9), for the bodies
        ///     the legacy path has nothing for.
        ///
        ///     On this machine's client that is **235 bodies** the catalogue could not reach
        ///     before, including the two gargoyle player bodies — `Bodyconv.def` sends 666 and 667
        ///     to `anim5`, at an index past the end of `anim5.idx`, and the art has been in
        ///     `AnimationFrame3.uop` all along.
        ///
        ///     Nothing here can produce §4.8's failure. A legacy index is addressed by position,
        ///     so a wrong lookup is another creature's picture; a UOP entry is addressed by the
        ///     hash of a name carrying the body id, and the payload repeats that id in its own
        ///     header for <see cref="BridgeUop.Group.TryOpen"/> to check. A miss is a miss.
        /// </summary>
        private static Sprite ResolveUop(string key, Readers readers, int body, int action, int direction)
        {
            ulong hash = BridgeUop.HashOf(body, action);

            byte[] payload = null;
            string reason = null;

            foreach (int n in BridgeUop.Packages)
            {
                BridgeUop.Package package = readers.Package(n);

                if (package == null || !package.Has(hash))
                    continue;

                if (!package.TryRead(hash, out payload, out reason))
                {
                    Console.WriteLine("[Bridge] catalogue: body {0} action {1} in {2}: {3}",
                        body, action, BridgeUop.PackageName(n), reason);
                    return null;
                }

                break;
            }

            if (payload == null)
                return null;

            BridgeUop.Group group;

            if (!BridgeUop.Group.TryOpen(payload, body, out group, out reason))
            {
                Console.WriteLine("[Bridge] catalogue: body {0} action {1} uop: {2}",
                    body, action, reason);
                return null;
            }

            int frame = group.DirectionAt(direction);

            if (frame < 0)
                return null;

            BridgeUop.Pixels pixels;
            bool empty;

            if (!group.TryDecode(frame, out pixels, out empty, out reason))
            {
                // A 0x0 frame is no art rather than damage — the vendored decoder returns early on
                // exactly the same condition — so it is absent, silently. Anything else is a
                // record this reader refused, and that is worth a line.
                if (!empty)
                    Console.WriteLine("[Bridge] catalogue: body {0} action {1} uop: {2}",
                        body, action, reason);

                return null;
            }

            byte[] png = BridgePng.FromArgb1555(pixels.Argb1555, pixels.Width, pixels.Height);

            if (png == null)
                return null;

            return new Sprite
            {
                Key = key,
                Body = body,
                Action = action,
                Direction = direction,
                FileType = 0,
                Png = png,
                Width = pixels.Width,
                Height = pixels.Height,
                Sha256 = BridgeAssets.Sha256Hex(png),
                Source = "uop"
            };
        }

        private static Sprite Decode(string key, int body, int action, int direction, int fileType)
        {
            int hue = 0;

            // `preserveHue: false` — the catalogue is the creature's own art, and a body-level hue
            // from Body.def belongs to a specific mob rather than to the species. §5's key scheme
            // is where a hued variant is expressed (`static/3922/h33`), not here.
            Frame[] frames = Animations.GetAnimation(body, action, direction, ref hue, false, true);

            if (frames == null || frames.Length == 0 || frames[0] == null)
                return null;

            Bitmap bitmap = frames[0].Bitmap;

            if (bitmap == null || bitmap.Width <= 0 || bitmap.Height <= 0)
                return null;

            byte[] png = BridgeAssets.BitmapToPng(bitmap);

            if (png == null)
                return null;

            return new Sprite
            {
                Key = key,
                Body = body,
                Action = action,
                Direction = direction,
                FileType = fileType,
                Png = png,
                Width = bitmap.Width,
                Height = bitmap.Height,
                Sha256 = BridgeAssets.Sha256Hex(png),
                Source = "legacy"
            };
        }

        // ── player bodies (§5.2) ─────────────────────────────────────────────────────────────

        /// <summary>
        ///     Asked of the shard, never hardcoded.
        ///
        ///     Every registered race carries its living male and female body ids, and a shard that
        ///     calls `RegisterRace` adds ids no table of ours could contain — which is the whole
        ///     argument against a hardcoded list, and it was never hypothetical: stock ServUO's
        ///     own `RaceDefinitions.cs` passes the gargoyle's ghost bodies in the OPPOSITE order
        ///     to the other two races.
        ///
        ///     This set is the whole of what §5.1 gives direction 0 — head-on, facing the viewer,
        ///     because a character is a portrait and should look at you. Everything else takes
        ///     direction 1, the front three-quarter, because head-on is the least legible view of
        ///     a four-legged creature: a wolf seen from the front is a dark blob.
        ///
        ///     **Ghost bodies are deliberately not in it** (§5.2, decided 2026-09-10 in phase 4).
        ///     A race declares four ids and two of them are its ghosts, and no UO client has art
        ///     for any of them: 402/403 and 694/695 read `lookup -1` in `anim.idx`, 607/608 read
        ///     `length 0` — the §4.8 shape, where the library hands back the previously-decoded
        ///     body's picture — and none of the six is in any `AnimationFrame*.uop`, which phase 4
        ///     established by claiming all 10,724 entries of the five packages with the one name
        ///     scheme. Listing them only advertised keys that cannot exist. A shard whose client
        ///     does ship ghost art still gets it: the body is catalogued like any other, at
        ///     direction 1 rather than 0.
        /// </summary>
        /// <summary>
        ///     Cached for the life of the process: `RegisterRace` runs at Configure time, before
        ///     anything on this plane can be asked a question, so the set cannot change under a
        ///     running shard. Rebuilding it per body would enumerate every race 2,047 times per
        ///     scan to answer a question whose answer is six integers.
        /// </summary>
        private static HashSet<int> _playerBodies;

        private static HashSet<int> PlayerBodies()
        {
            var cached = _playerBodies;

            if (cached != null)
                return cached;

            var set = new HashSet<int>();

            try
            {
                foreach (var race in Race.AllRaces)
                {
                    if (race == null)
                        continue;

                    set.Add(race.MaleBody);
                    set.Add(race.FemaleBody);
                }
            }
            catch (Exception e)
            {
                Console.WriteLine("[Bridge] catalogue: cannot enumerate races: {0}", e.Message);
            }

            set.Remove(0);

            _playerBodies = set;

            return set;
        }

        private static bool IsPlayerBody(int body)
        {
            return PlayerBodies().Contains(body);
        }

        private static void WritePlayerBodies(StringBuilder sb)
        {
            var bodies = new List<int>(PlayerBodies());
            bodies.Sort();

            sb.Append(",\"playerBodies\":[");

            for (int i = 0; i < bodies.Count; i++)
            {
                if (i > 0)
                    sb.Append(',');

                sb.Append(bodies[i].ToString(CultureInfo.InvariantCulture));
            }

            sb.Append(']');
        }

        // ── keys, cursors and the source id ──────────────────────────────────────────────────

        private static string Key(int body, int action)
        {
            return "body/" + body.ToString(CultureInfo.InvariantCulture)
                   + "/a" + action.ToString(CultureInfo.InvariantCulture);
        }

        /// <summary>
        ///     `body/&lt;id&gt;/a&lt;n&gt;`, and nothing else in this phase. A deeper key
        ///     (`body/400/a2/f3`) is well-formed under §5 and simply not served, so it comes back
        ///     `unsupported` rather than being silently read as its own first frame.
        ///
        ///     The action is parsed rather than required to be zero — 74 of this client's bodies
        ///     are catalogued at a different one (§11.2) — but a parsed action is not an accepted
        ///     one. <see cref="Render"/> serves a key only when it is the key the catalogue itself
        ///     chose for that body, which is what keeps §4.10's ceiling from being reachable
        ///     through a request: nothing the website can ask makes this decode an action the
        ///     catalogue did not already pick.
        /// </summary>
        private static bool TryParseKey(string key, out int body, out int action)
        {
            body = 0;
            action = -1;

            if (key == null)
                return false;

            string[] parts = key.Split('/');

            if (parts.Length != 3 || parts[0] != "body")
                return false;

            if (!Int32.TryParse(parts[1], NumberStyles.None, CultureInfo.InvariantCulture, out body))
                return false;

            if (body < 1 || body > MaxBody)
                return false;

            if (parts[2].Length < 2 || parts[2][0] != 'a')
                return false;

            if (!Int32.TryParse(parts[2].Substring(1), NumberStyles.None,
                    CultureInfo.InvariantCulture, out action))
                return false;

            return action >= 0 && action < MaxAction;
        }

        private static int ParseBodyCursor(string cursor)
        {
            if (cursor == null)
                return 1;

            int value;

            if (cursor.StartsWith("b:", StringComparison.Ordinal)
                && Int32.TryParse(cursor.Substring(2), NumberStyles.None, CultureInfo.InvariantCulture, out value))
                return Math.Max(1, value + 1);

            return 1;
        }

        private static int ParseKeyCursor(string cursor)
        {
            if (cursor == null)
                return 0;

            int value;

            if (cursor.StartsWith("k:", StringComparison.Ordinal)
                && Int32.TryParse(cursor.Substring(2), NumberStyles.None, CultureInfo.InvariantCulture, out value))
                return Math.Max(0, value);

            return 0;
        }

        /// <summary>
        ///     Everything that decides the bytes, hashed into one short id.
        ///
        ///     Deliberately (size, mtime) rather than content: §6 makes exactly the same choice
        ///     for the source gate, and for the same reason — the anim files are 195 MB and
        ///     hashing them on every page of a walk would turn a manifest into a minute.
        ///     `assets.sources` is where an operator gets content hashes, computed off the request
        ///     path; this is a "did it move while I was reading" check, which (size, mtime)
        ///     answers.
        /// </summary>
        private static string SourceId()
        {
            var sb = new StringBuilder(256);

            sb.Append(BridgeAssets.EXTRACTOR_VERSION)
              .Append(':').Append(BridgeConfig.AssetPlayerDirection)
              .Append(':').Append(BridgeConfig.AssetCreatureDirection);

            var paths = new List<string>();

            for (int fileType = 1; fileType <= 5; fileType++)
                paths.Add(BridgeAssetValidator.AnimDataPath(fileType));

            // Since phase 4 the catalogue's bytes depend on the UOP packages too — 235 of its
            // bodies come out of them — so patching one has to change the catalogue id, exactly as
            // patching an anim*.mul does. Leaving them out would let an operator replace a
            // gargoyle and have an Update find nothing to do.
            foreach (int n in BridgeUop.Packages)
                paths.Add(BridgeUop.PackagePath(n));

            foreach (string path in paths)
            {
                sb.Append('|');

                if (path == null)
                    continue;

                try
                {
                    var info = new FileInfo(path);

                    if (!info.Exists)
                        continue;

                    sb.Append(info.Length).Append(',').Append(info.LastWriteTimeUtc.Ticks);
                }
                catch
                {
                    // An unreadable file is itself a state, and one that must not change from page
                    // to page without being noticed. Leaving the slot empty does that.
                }
            }

            return BridgeAssets.Sha256Hex(Encoding.UTF8.GetBytes(sb.ToString())).Substring(0, 16);
        }

        // ── shared plumbing ──────────────────────────────────────────────────────────────────

        /// <summary>
        ///     The five anim files' index and record readers — and, since phase 4, the five UOP
        ///     packages beside them — opened for one reply and closed with it. Holding them across
        ///     replies would keep handles on the operator's client files for as long as the cache
        ///     lives, for no gain: opening them is microseconds and a page decodes hundreds of
        ///     sprites through them.
        /// </summary>
        private sealed class Readers : IDisposable
        {
            private readonly FileIndex[] _index = new FileIndex[6];
            private readonly BridgeAssetValidator.RecordReader[] _reader =
                new BridgeAssetValidator.RecordReader[6];
            private readonly long[] _length = new long[6];
            private readonly bool[] _open = new bool[6];

            private readonly Dictionary<int, BridgeUop.Package> _packages =
                new Dictionary<int, BridgeUop.Package>();

            public readonly long VerdataLength;

            public Readers()
            {
                VerdataLength = BridgeAssetValidator.MulLength(VerdataPath());
            }

            private static string VerdataPath()
            {
                try
                {
                    return Files.GetFilePath("verdata.mul");
                }
                catch
                {
                    return null;
                }
            }

            private void Ensure(int fileType)
            {
                if (fileType < 1 || fileType > 5 || _open[fileType])
                    return;

                _open[fileType] = true;

                string path = BridgeAssetValidator.AnimDataPath(fileType);

                if (path == null)
                    return;

                try
                {
                    _index[fileType] = BridgeAssetValidator.OpenAnimIndex(fileType);
                    _length[fileType] = BridgeAssetValidator.MulLength(path);
                    _reader[fileType] = new BridgeAssetValidator.RecordReader(path, VerdataPath());
                }
                catch (Exception e)
                {
                    Console.WriteLine("[Bridge] catalogue: anim file type {0}: {1}",
                        fileType, e.Message);
                }
            }

            public FileIndex Index(int fileType)
            {
                Ensure(fileType);
                return fileType >= 1 && fileType <= 5 ? _index[fileType] : null;
            }

            public long MulLength(int fileType)
            {
                Ensure(fileType);
                return fileType >= 1 && fileType <= 5 ? _length[fileType] : 0;
            }

            public BridgeAssetValidator.RecordReader Reader(int fileType)
            {
                Ensure(fileType);
                return fileType >= 1 && fileType <= 5 ? _reader[fileType] : null;
            }

            /// <summary>
            ///     One <c>AnimationFrame*.uop</c>, opened on first use. A package this client does
            ///     not ship is a null that is cached as one: the miss must not be re-resolved and
            ///     re-opened once per body across a 2,047-body walk.
            /// </summary>
            public BridgeUop.Package Package(int n)
            {
                BridgeUop.Package package;

                if (_packages.TryGetValue(n, out package))
                    return package;

                package = BridgeUop.Package.Open(BridgeUop.PackagePath(n));

                _packages[n] = package;

                return package;
            }

            public void Dispose()
            {
                foreach (var package in _packages.Values)
                {
                    if (package == null)
                        continue;

                    try
                    {
                        package.Dispose();
                    }
                    catch
                    {
                        // Closing a read-only handle. Nothing useful is left to do.
                    }
                }

                for (int i = 1; i <= 5; i++)
                {
                    if (_reader[i] == null)
                        continue;

                    try
                    {
                        _reader[i].Dispose();
                    }
                    catch
                    {
                        // Closing a read-only handle. Nothing useful is left to do.
                    }
                }
            }
        }

        /// <summary>
        ///     Lets a megabyte of the operator's client art go once nothing has asked for it in
        ///     five minutes. A rebuild costs one scan and, because <see cref="SourceId"/> is
        ///     derived from the files rather than minted per build, it produces the same catalogue
        ///     id — so an import that spans the drop does not have to start over.
        /// </summary>
        private static void Sweep()
        {
            lock (_sync)
            {
                if (_catalog == null)
                    return;

                if (DateTime.UtcNow - _catalog.LastUsed > IdleFor)
                    _catalog = null;
            }
        }

        public static string Status()
        {
            lock (_sync)
            {
                if (_catalog == null)
                    return "catalog(empty)";

                return String.Format("catalog(id={0} held={1} next={2} complete={3})",
                    _catalog.Id, _catalog.Order.Count, _catalog.Next, _catalog.Complete);
            }
        }
    }
}