diff --git a/.gitea/workflows/build-image.yml b/.gitea/workflows/build-image.yml index 56d31cd..b8c2478 100644 --- a/.gitea/workflows/build-image.yml +++ b/.gitea/workflows/build-image.yml @@ -90,7 +90,7 @@ jobs: echo "${{ secrets.REGISTRY_TOKEN }}" \ | docker login "${REGISTRY}" -u "${{ secrets.REGISTRY_USER }}" --password-stdin - - name: Build and push + - name: Build # Two tags from one build: `latest` for the compose default, `sha-<7>` so # a deploy can be pinned or rolled back to an exact commit. run: | @@ -99,6 +99,61 @@ jobs: -t "${IMAGE}:latest" \ -t "${IMAGE}:${TAG}" \ . + + - name: No layer may exceed the registry's request limit + # Gitea is behind Cloudflare, which refuses a request body over 100 MB on + # every plan below Enterprise, and `docker push` uploads each layer as one + # monolithic PUT. An oversized layer is therefore rejected at the EDGE: + # Gitea never sees it, the log says only `413 Payload Too Large` against a + # blob digest, and the image is not published at all. That is what happened + # on the first merge after phase 12, and the Dockerfile's three-layer + # node_modules split is what fixed it. + # + # A split is a margin, not a guarantee, so this counts the layers before + # the push rather than letting the next fat dependency rediscover the 413. + # 90 MB, not 100: the cap is on the whole request, and the blob is not the + # only thing in it. + # + # Measured by re-compressing what `docker save` writes, because the daemon + # exposes uncompressed sizes only and the limit applies to the compressed + # blob. gzip is what the push uses, so the numbers agree to within a per + # cent; both archive layouts are handled, since a layer is already gzipped + # in one of them and plain in the other. + run: | + set -euo pipefail + LIMIT_MB=90 + + docker save "${IMAGE}:${TAG}" -o /tmp/image.tar + mkdir -p /tmp/layers + tar -xf /tmp/image.tar -C /tmp/layers + + WORST_MB=0 + WORST_FILE="" + # Only files big enough to matter; everything else is metadata. + while IFS= read -r f; do + if [ "$(head -c 2 "$f" | od -An -tx1 | tr -d ' \n')" = "1f8b" ]; then + SIZE=$(stat -c %s "$f") # already compressed + else + SIZE=$(gzip -c "$f" | wc -c) # compress it the way the push will + fi + MB=$(( SIZE / 1048576 )) + if [ "$MB" -gt "$WORST_MB" ]; then + WORST_MB=$MB + WORST_FILE=$f + fi + done < <(find /tmp/layers -type f -size +8M) + + rm -rf /tmp/image.tar /tmp/layers + + echo "Largest layer: ${WORST_MB} MB compressed (limit ${LIMIT_MB} MB)" + if [ "$WORST_MB" -gt "$LIMIT_MB" ]; then + echo "::error::A layer is ${WORST_MB} MB compressed (${WORST_FILE}). Cloudflare rejects a request body over 100 MB, so this push would fail with 413 Payload Too Large and publish nothing. Split the layer in the Dockerfile — see the COPY block that splits node_modules." + exit 1 + fi + + - name: Push + run: | + set -euo pipefail docker push "${IMAGE}:latest" docker push "${IMAGE}:${TAG}" diff --git a/DEPLOY.md b/DEPLOY.md index e3e12fe..583d2b7 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -313,6 +313,15 @@ write `/opt/runicgateway.com`. already been built and pushed by the time it would run, so the manual update below works throughout, and the queued job goes as soon as the runner registers. +**One way the automatic deploy can fail before it starts.** The registry is behind Cloudflare, which +refuses a request body over 100 MB, and `docker push` uploads each image layer as a single request — +so a layer that grows past that is rejected at the edge with `413 Payload Too Large`, publishing +nothing. `needs: build` then keeps the deploy from running at all, which means the container you are +already serving is left alone; the site is simply not updated. The workflow checks layer sizes before +it pushes and fails with a message naming the layer, so this should announce itself rather than +arriving as a `413`. Either way it is fixed in the `Dockerfile` (see the `COPY` block that splits +`node_modules`) and nothing needs doing on the host. + **To update by hand instead** — always available, and what you do if the runner is down: ```bash @@ -378,6 +387,7 @@ docker compose ps | A new logo or site name has not appeared | The mount needs a `docker compose restart site`, not just a file edit — [§5](#5-branding-without-a-rebuild) | | Search finds the old site name | Same restart; the boot rewrite re-indexes | | The site is stock despite files in `brand/` | Check the mount actually landed: `docker compose exec site ls /app/brand` | +| A merge did not deploy, and the build job is red | If it failed on the layer check or on a `413`, a layer grew past Cloudflare's 100 MB request limit — [§7](#7-updating-and-the-automatic-deploy). The running container is untouched; the fix is in the Dockerfile, not on the host | Everything the container writes goes to stdout, so `docker compose logs` is the whole log. The reverse proxy's access log is the only traffic data that exists — there are no analytics anywhere on diff --git a/Dockerfile b/Dockerfile index f221feb..888aca4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -42,6 +42,15 @@ RUN npm run build # modules exactly as they were resolved and built once. RUN npm prune --omit=dev +# Set the two largest packages aside so the runtime stage can copy them as their +# own layers. See the COPY block below for why a single node_modules layer could +# not be pushed at all. Moving them rather than copying them twice is what keeps +# the three layers disjoint: whatever is left in node_modules is exactly the +# remainder, and a dependency added later lands in it automatically. +RUN mkdir -p /split \ + && mv node_modules/@pagefind /split/ \ + && mv node_modules/@img /split/ + # --------------------------------------------------------------------------------------- # Stage 2 — runtime # --------------------------------------------------------------------------------------- @@ -67,7 +76,28 @@ ENV BRAND_DEFAULT_DIR=/app/brand-default # says, records that in dist/.brand-applied.json, and re-indexes dist/client/pagefind # so search finds the mounted site name. A read-only dist would make §7's promise # — recolour and rename by copying a file — fail at boot with a permission error. +# node_modules arrives in THREE layers, not one, and the reason is the registry +# rather than anything about the site. +# +# Gitea sits behind Cloudflare, which refuses a request body over 100 MB on every +# plan below Enterprise, and `docker push` uploads each layer as one monolithic +# PUT. A single `COPY node_modules` measured **108.8 MB compressed** — nine over — +# so the first merge to `main` after phase 12 failed with `413 Payload Too Large` +# on that one blob, from the edge, with Gitea never seeing the request. Nothing +# was published, and `needs: build` meant nothing was deployed either. +# +# `@pagefind` (the search index binaries) and `@img` (sharp's libvips) are the two +# packages that make it fat and both are needed at RUN time — the boot rewrite +# re-indexes the site and re-derives the brand images — so the fix is where they +# land, not whether they ship. Split, they measure 54.7 + 50.6 + 12.1 MB, the +# largest with about 45 MB of headroom. +# +# That headroom is why the workflow counts layers before it pushes: this is a +# margin, not a guarantee, and a dependency that grows past it would otherwise +# come back as the same unreadable 413. See `.gitea/workflows/build-image.yml`. COPY --from=build --chown=node:node /build/node_modules ./node_modules +COPY --from=build --chown=node:node /split/@pagefind ./node_modules/@pagefind +COPY --from=build --chown=node:node /split/@img ./node_modules/@img COPY --from=build --chown=node:node /build/dist ./dist COPY --from=build --chown=node:node /build/brand-default ./brand-default COPY --from=build --chown=node:node /build/scripts ./scripts diff --git a/PLAN.md b/PLAN.md index 5926722..f973f9d 100644 --- a/PLAN.md +++ b/PLAN.md @@ -251,6 +251,7 @@ somewhere other than the thing it decided. The count of record is **fifty-seven* | D47–D50 | §6, "How phase 10 polished it" | Search reaches the marketing pages, the CSP is a real response header from the container, `robots.txt` allows everything and names the sitemap, two blocks of structured data and no more | | D51–D53 | §6, "How phase 11 validated it" | The chrome and the head follow the brand mount while the consent sentence does not, the documentation half gets phase 10's skip-link fix, and no twelfth check | | D54–D57 | §6, "How phase 12 delivered it" | A merge deploys (amending D6), the proxy is documented by its requirements rather than by an example, the container trusts the forwarded headers with nothing to configure, and the operator note is its own file while the two policy files are pointers | +| D58 | §6, "How phase 12 delivered it" | `node_modules` ships in three layers because Cloudflare refuses a request body over 100 MB, and the workflow counts layers before it pushes | --- @@ -521,7 +522,8 @@ The last phase, and the one that turns a repository into a deployment: a two-sta pull-only `docker-compose.yml` carrying both bind mounts, `.env.example`, the publishing workflow, `CONTRIBUTING.md` with the AI-disclosure requirement, the community-health files this repository was the only one in the organisation to lack, and `DEPLOY.md`. Four decisions, **D54–D57**, taking the -count of record to **fifty-seven**. +count of record to **fifty-seven** — and a fifth, **D58**, added when the merge that shipped the +phase could not publish its own image. It also found a defect that would have made the closed beta impossible, and it is the only phase that could have found it. Everything before this ran the site the way a developer runs it: one @@ -634,6 +636,40 @@ surface to attack. variables the code reads, `.env.example` declares and the README tabulates — and the org lead held the line. Eleven checks stand. +#### D58 — the image ships in three layers, because of the registry rather than the site + +The merge that landed phase 12 could not publish the image it had just built. `docker push` answered +**`413 Payload Too Large`** on one blob and stopped; nothing reached the registry, and `needs: build` +meant nothing reached the host either. The site was merged and undeployed, and the log said only that +a digest was too large. + +**The limit is Cloudflare's, not Gitea's.** `gitea.whitlocktech.com` is proxied, and Cloudflare +refuses a request body over **100 MB** on every plan below Enterprise — a plan limit, not a setting. +`docker push` uploads each layer as a single monolithic `PUT`, so the ceiling applies per layer, and +the rejection happens at the edge with Gitea never seeing the request. It is invisible from the +Gitea side and unfixable from it. + +Measured on the merge commit, one layer was over and only just: **`COPY node_modules` at 108.8 MB +compressed**, against a 188.6 MB image whose next largest layer was the 47.6 MB Node base. Two +packages account for it, `@pagefind` (the search binaries) and `@img` (sharp's libvips), and both are +needed at **run** time — the boot rewrite re-indexes the site and re-derives the brand images — so +what could move was where they land, not whether they ship. + +The build stage now moves those two aside after `npm prune`, and the runtime stage copies them as +their own layers: **46.8 + 50.5 + 11.5 MB** in place of 108.8, largest layer 50.5, and the image +**exactly the same total size**, because the same bytes are simply divided differently. Moving rather +than copying twice is what keeps the three disjoint — whatever remains in `node_modules` is the +remainder by construction, so a dependency added later needs no maintenance here. + +**The workflow now counts layers before it pushes**, because a split is a margin and not a +guarantee: `docker save`, re-compress anything over 8 MB the way the push would, and fail at **90 MB** +— not 100, since the blob is not the only thing in the request — naming the layer and what would +have happened. It was tested in both directions, against the fixed image and the broken one, and the +number it reports for the broken layer (108 MB) agrees with what the registry recorded. + +This is a workflow step and not a twelfth check script: it needs a built image rather than a source +tree, which is the one thing the eleven never have. D53 holds. + --- ## 7. Branding is bind-mounted data @@ -1616,10 +1652,15 @@ a mechanism rather than diligence: Phases 5 and 6 are deliberately adjacent and early: the beta cannot start without `/privacy`, and the closed test is the nearest real deadline. -**All twelve are built, as of 2026-08-25.** What is left is not a phase: point the DNS record at the -host (§14, N1), start the `rgcom` runner, and — when the demo VM exists (§15) and the Play track is -open — put two URLs into the mounted `brand.json`. None of those is a code change, which was the -point. +**All twelve are built, as of 2026-08-25.** The `rgcom` runner was registered on the host the next +day, in host mode as §7 of `DEPLOY.md` requires. What is left is not a phase: point the DNS record at +the host (§14, N1), and — when the demo VM exists (§15) and the Play track is open — put two URLs +into the mounted `brand.json`. Neither is a code change, which was the point. + +**One thing did need a code change.** The merge that landed phase 12 built its image and then could +not publish it: Cloudflare rejected the largest layer with `413 Payload Too Large`, so the registry +stayed empty and the deploy never ran. See **D58** — the layer is split, and the workflow now counts +layers before it pushes. ---