Merge pull request 'Wiki upgrade: rich-text editing, categories, drafts, links/backlinks, tags, search, revisions' (#3) from wiki-upgrade into main

Reviewed-on: UOM/website#3
2026-06-27 21:22:44 +00:00
parent efbafe7203 effd606d4d
commit 20b432098a
1 changed files with 366 additions and 0 deletions
--- a/website/WIKI_UPGRADE.md
+++ b/website/WIKI_UPGRADE.md
@@ -0,0 +1,366 @@
+# UOMysticmoon Website — Wiki Upgrade Spec
+
+> Branch: **`wiki-upgrade`**. This document is the contract for upgrading the CMS
+> wiki from a flat single-table page store into a feature-complete wiki.
+> It follows the project workflow: **design (this doc) → build in phases → verify**.
+>
+> Companion to [`BACKEND_DESIGN.md`](BACKEND_DESIGN.md); reuses its stack, auth,
+> logging, and Docker decisions unchanged.
+
+---
+
+## 1. Goal & scope
+
+Turn the wiki into something that behaves like a typical wiki, while staying inside
+the existing Node/Express + MariaDB + React/Vite architecture and the **staff-only**
+auth model (admin/editor — no new roles, no public contributions).
+
+**In scope**
+
+| Feature | Summary |
+|---|---|
+| Rich-text editing | TipTap (ProseMirror) WYSIWYG in the admin; outputs HTML |
+| Sanitization | Server-side allowlist on save **and** client-side on render (fixes today's stored-XSS gap) |
+| Categories / sections | First-class `wiki_categories` table; replaces hardcoded frontend blurbs |
+| Drafts & publish | `published` + `published_at`, mirroring the `posts` pattern |
+| Tags | Many-to-many tags with filtering |
+| Internal links | `[[slug]]`-style links authored in the editor; red-link detection |
+| Backlinks | "Linked from" list, maintained on save |
+| Inline images | Reuse/generalize the existing multer upload for in-body images |
+| Search | MariaDB `FULLTEXT` over title + body |
+| Revision history | Per-save snapshots with view / diff / restore |
+
+**Out of scope (this branch)**
+
+- Public/player editing or suggestion workflow, moderation/review queues.
+- New roles or per-page ACLs (all staff with a login can edit all pages).
+- Real-time collaborative editing, comments/discussion pages, file attachments
+  other than images, page templates/transclusion, multilingual pages.
+
+**Decisions locked from planning**
+
+- Editor: **TipTap**, storing **HTML** (not Markdown, not JSON).
+- Search: **MariaDB FULLTEXT** (no new infrastructure).
+- Revision history and search are **included** (recommended additions beyond the
+  minimum requested set).
+- Authoring is **admin + editor** (`isLoggedIn`); no anonymous edits.
+
+---
+
+## 2. Current state (baseline being replaced)
+
+| Layer | Today | File |
+|---|---|---|
+| Schema | flat `wiki_pages(slug,title,body,updated_by,timestamps)` | [server/db/schema.sql:31](server/db/schema.sql) |
+| Model | thin CRUD by slug | [server/src/model/wiki/wiki.db.js](server/src/model/wiki/wiki.db.js), [wiki.model.js](server/src/model/wiki/wiki.model.js) |
+| Public API | `GET /public/wiki`, `GET /public/wiki/:slug` | [public.controller.js:53](server/src/router/v1/public/public.controller.js) |
+| Admin API | `GET/POST/PUT/DELETE /admin/wiki[...]` | [admin.controller.js:163](server/src/router/v1/admin/admin.controller.js), [admin.routes.js:68](server/src/router/v1/admin/admin.routes.js) |
+| Public UI | card grid (hardcoded blurbs + Roman numerals), article w/ auto-TOC | [Wiki.jsx](client/src/routes/wiki/Wiki.jsx), [WikiArticle.jsx](client/src/routes/wiki/WikiArticle.jsx) |
+| Admin UI | raw-HTML `<textarea>` modal | [WikiAdmin.jsx](client/src/routes/admin/views/WikiAdmin.jsx), [WikiEditor.jsx](client/src/routes/admin/views/WikiEditor.jsx) |
+| API client | `api.wiki`, `api.admin.*Wiki` | [client/src/api/client.js:52](client/src/api/client.js) |
+
+**Known issues this upgrade resolves**
+
+- **Stored XSS**: body is raw HTML rendered with `dangerouslySetInnerHTML` and never
+  sanitized ([WikiArticle.jsx:91](client/src/routes/wiki/WikiArticle.jsx)).
+- Category blurbs and ordering are **faked in the component** ([Wiki.jsx:11](client/src/routes/wiki/Wiki.jsx)), not data.
+- No drafts (every save is instantly public), no history, no search, no tags, no links.
+
+---
+
+## 3. Data model
+
+`utf8mb4`, InnoDB throughout. All changes are **additive and idempotent** so
+`ensureSchema()` upgrades existing databases on boot with no data loss. New columns
+are nullable or have safe defaults; **existing pages default to `published = 1`** so
+nothing disappears on deploy.
+
+### 3.1 `wiki_categories` (new)
+
+| col | type | notes |
+|---|---|---|
+| id | INT PK AI | |
+| slug | VARCHAR(120) UNIQUE NOT NULL | e.g. `guides` |
+| title | VARCHAR(200) NOT NULL | |
+| description | VARCHAR(400) NULL | card teaser on the wiki index |
+| sort_order | INT NOT NULL DEFAULT 0 | manual ordering |
+| created_at / updated_at | DATETIME | standard stamps |
+
+### 3.2 `wiki_pages` (altered)
+
+Add to the existing table:
+
+| col | type | notes |
+|---|---|---|
+| category_id | INT NULL FK→wiki_categories(id) ON DELETE SET NULL | |
+| excerpt | VARCHAR(400) NULL | card/search teaser (replaces hardcoded blurbs) |
+| published | TINYINT(1) NOT NULL DEFAULT 1 | draft/publish toggle |
+| published_at | DATETIME NULL | set on first publish |
+| sort_order | INT NOT NULL DEFAULT 0 | ordering within a category |
+| FULLTEXT idx_wiki_search (title, body) | | search |
+
+### 3.3 `wiki_tags` + `wiki_page_tags` (new)
+
+```
+wiki_tags( id PK, slug VARCHAR(120) UNIQUE, label VARCHAR(120) )
+wiki_page_tags( page_id FK→wiki_pages ON DELETE CASCADE,
+                tag_id  FK→wiki_tags  ON DELETE CASCADE,
+                PRIMARY KEY(page_id, tag_id) )
+```
+
+### 3.4 `wiki_links` (new) — backlinks index
+
+Rebuilt for a page on every save by parsing its body for internal links.
+
+| col | type | notes |
+|---|---|---|
+| source_page_id | INT FK→wiki_pages ON DELETE CASCADE | |
+| target_slug | VARCHAR(120) NOT NULL | may point at a not-yet-created page (red link) |
+| INDEX idx_wiki_links_target (target_slug) | | backlink lookups |
+
+Backlinks for page X = `SELECT source pages WHERE target_slug = X.slug AND source is published`.
+
+### 3.5 `wiki_revisions` (new) — history
+
+| col | type | notes |
+|---|---|---|
+| id | INT PK AI | |
+| page_id | INT FK→wiki_pages ON DELETE CASCADE | |
+| title / body / excerpt | snapshot of content at save time | |
+| category_id | INT NULL | snapshot |
+| editor_id | INT NULL FK→users(id) | who saved |
+| change_note | VARCHAR(280) NULL | optional summary |
+| created_at | DATETIME DEFAULT CURRENT_TIMESTAMP | |
+
+A revision is written **inside the same transaction** as each page create/update.
+
+### 3.6 Seed changes
+
+Rework [seed.js](server/db/seed.js): the current 8 hardcoded pages become **categories**
+(title + the blurb currently living in the frontend), each seeded idempotently via a new
+`seedDefaultCategory`. Existing seeded pages are migrated/attached where applicable.
+`seedDefault` for pages stays `INSERT IGNORE` so reseeding is safe.
+
+---
+
+## 4. Backend changes
+
+Keep the `model` (entity) / `db` (SQL) split and the route grouping by access level.
+
+### 4.1 Models (`server/src/model/wiki/`)
+
+- `wiki.db.js` — add SQL for: category CRUD; page list with `category`, `published`,
+  `q` (FULLTEXT) filters and ordering; tag upsert + attach/detach; `wiki_links` rebuild;
+  revision insert/list/get; backlink query.
+- `wiki.model.js` — orchestration. On **create/update** (single transaction):
+  1. sanitize `body` with the allowlist (§6),
+  2. upsert the page,
+  3. insert a `wiki_revisions` snapshot,
+  4. parse body for internal links → rebuild `wiki_links` for the page,
+  5. sync tags.
+- A small `wiki.links.js` helper: parse internal links out of the saved HTML
+  (anchors written by the editor as `href="/wiki/<slug>"` / a `data-wiki-slug` attr),
+  return the set of target slugs.
+
+### 4.2 Public API (`/api/v1/public`)
+
+| Method | Path | Notes |
+|---|---|---|
+| GET | `/wiki/categories` | ordered categories with page counts |
+| GET | `/wiki?category=&tag=&q=` | **published only**; list/filter/search summaries |
+| GET | `/wiki/:slug` | page + category + tags + backlinks (published only) |
+
+Still passes through the `siteMode` maintenance gate like other public content.
+
+### 4.3 Admin API (`/api/v1/admin`, behind `isLoggedIn` + `noindex`)
+
+| Method | Path | Purpose |
+|---|---|---|
+| GET | `/wiki` | all pages incl. drafts (filters: category, tag, q, status) |
+| GET | `/wiki/:slug` | one page incl. draft, tags, category |
+| POST | `/wiki` | create (slug, title, body, excerpt, category_id, tags, published) |
+| PUT | `/wiki/:slug` | update (allows slug rename — see §7) |
+| PATCH | `/wiki/:slug/publish` | `{published}` toggle, stamps `published_at` |
+| DELETE | `/wiki/:slug` | delete (cascades revisions/links/tags) |
+| GET | `/wiki/:slug/revisions` | list snapshots |
+| GET | `/wiki/:slug/revisions/:id` | one snapshot (for diff/preview) |
+| POST | `/wiki/:slug/revisions/:id/restore` | restore (writes a new revision) |
+| GET/POST/PUT/DELETE | `/wiki/categories[...]` | category CRUD + reorder |
+| GET/POST | `/wiki/tags` | list/create tags |
+| POST | `/uploads` | generalized image upload (see §4.4) → `{url}` |
+
+Validation via `express-validator` (slug regex `^[a-z0-9-]+$`, title required, etc.),
+centralized error handler unchanged. **Every write logs to `activity_log`**
+(`wiki.create`, `wiki.update`, `wiki.publish`, `wiki.delete`, `wiki.revision.restore`,
+`wiki.category.*`) following the existing convention.
+
+### 4.4 Image uploads
+
+Generalize the existing screenshot upload (multer config in [admin.routes.js:17](server/src/router/v1/admin/admin.routes.js))
+into a shared `POST /admin/uploads` returning `{ url: "/uploads/<file>" }`, reused by both
+the post editor and the wiki editor. Same size/mime limits. No new storage —
+served from the existing `uploads/` volume.
+
+---
+
+## 5. Frontend changes
+
+### 5.1 Admin
+
+- **`WikiEditor.jsx`** — replace the raw-HTML `<textarea>` with a **TipTap** editor:
+  bold/italic/headings (H2 for TOC)/lists/quote/code, link tool, **image insert**
+  (uploads via `/admin/uploads`), and an **internal-link picker** (`[[`-triggered
+  autocomplete over existing slugs; flags red links). Adds: category dropdown, tag
+  input (create-on-type), excerpt field, **Save draft / Publish** actions, and a
+  **History** tab (revision list → preview → diff → restore).
+- **`WikiAdmin.jsx`** — list gains status (draft/published), category column, and
+  filters; plus a **Categories** manager (CRUD + drag-to-reorder).
+
+### 5.2 Public
+
+- **`Wiki.jsx`** — fully data-driven: categories + real excerpts from the API
+  (delete the hardcoded `BLURBS`/`ROMAN` constants), a **search box**, optional
+  tag filter.
+- **`WikiArticle.jsx`** — keep auto-TOC; add category breadcrumb, tag chips, a
+  **"Linked from"** backlinks section, "last updated by", and **render via DOMPurify**
+  (`dangerouslySetInnerHTML` only after sanitize).
+
+### 5.3 API client & routes
+
+- Extend [client/src/api/client.js](client/src/api/client.js) with the new public/admin
+  wiki calls (categories, search params, revisions, tags, uploads).
+- Add a public search/category route if needed; admin categories view registered in
+  [App.jsx](client/src/App.jsx) under `/admin/wiki` (sub-tab, no new top-level route required).
+
+### 5.4 Dependencies (new)
+
+- **client**: `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-link`,
+  `@tiptap/extension-image` (+ a small diff lib for history, e.g. `diff`); `dompurify`.
+- **server**: `sanitize-html`.
+
+(The client currently ships only React + react-router, so this is the first feature
+dependency addition — keep the bundle lean, import only the extensions used.)
+
+---
+
+## 6. Security
+
+- **Two-layer sanitization.** Server sanitizes on save with a strict `sanitize-html`
+  allowlist (headings, p, lists, blockquote, code/pre, a[href], img[src,alt],
+  strong/em, hr, table basics); strips scripts, event handlers, `javascript:` URLs,
+  styles. Client re-sanitizes with DOMPurify before render. The stored value is already
+  clean, so even direct DB edits or future API clients can't inject script.
+- **Upload safety** unchanged from posts: mime allowlist (png/jpe/gif/webp/avif),
+  8 MB cap, random filenames, served as static files (no execution).
+- **Authorization**: all mutating wiki/category/tag/upload routes stay behind
+  `isLoggedIn` (admin or editor). Public routes are read-only and published-only.
+- **No secrets/logging changes**; reuse existing rate-limit, helmet/CSP, noindex.
+  CSP `img-src` already covers `/uploads`.
+
+---
+
+## 7. Migration & backward compatibility
+
+- Schema migration is additive; run by `ensureSchema()` on boot and shipped in
+  `schema.sql` for fresh containers. Use `ALTER TABLE ... ADD COLUMN IF NOT EXISTS`
+  / `ADD INDEX` guarded for idempotency (MariaDB 11 supports `IF NOT EXISTS`).
+- Existing pages: `published` backfills to `1`, `published_at` to `updated_at`,
+  `category_id` left NULL (surface as "Uncategorized" until assigned).
+- **Slug rename** (new capability): on `PUT` slug change, update the page slug and
+  best-effort rewrite known internal links pointing at the old slug; old slug is not
+  auto-redirected (acceptable for a staff-curated wiki) — note in release notes.
+- Public API response shape is **extended, not broken**: existing fields
+  (`slug`, `title`, `body`, `updated_at`) remain; new fields are additive, so the
+  current frontend keeps working between phases.
+
+---
+
+## 8. Implementation process (phased)
+
+Each phase is a self-contained, shippable unit: build → run locally → verify in the
+browser preview → commit on `wiki-upgrade`. Open a PR into `main` at the end (or per
+phase if preferred). Do not merge a phase that hasn't been verified.
+
+### Phase 0 — Branch & scaffolding ✅ (this doc)
+- `wiki-upgrade` branch created; this spec committed.
+
+### Phase 1 — Foundation & safety (highest value) ✅
+- Schema: add `wiki_categories`, alter `wiki_pages` (category_id, excerpt, published,
+  published_at, sort_order, FULLTEXT), update `seed.js`.
+- Server: server-side sanitization on save; drafts/publish endpoints; categories CRUD;
+  public list filtered to published + categories endpoint.
+- Client: data-driven `Wiki.jsx` (remove hardcoded blurbs); DOMPurify render in
+  `WikiArticle.jsx`; draft/publish + category in the (still-textarea) admin editor.
+- **Exit check**: existing pages still render; XSS payload in body is neutralized;
+  draft pages hidden from the public list/article.
+- **Verified** (2026-06-27): schema migration ran clean on MariaDB 11; XSS payload
+  (`<script>`, `onerror=`, `javascript:`) stripped server-side; drafts return 404 on
+  the public API and are absent from the public list while visible in admin; public
+  index is data-driven (categories + sections); article shows category breadcrumb;
+  client builds and server boots with no errors.
+
+### Phase 2 — Authoring UX ✅
+- TipTap editor replaces the textarea; generalized `/admin/uploads`; inline images.
+- **Exit check**: create/edit a page with headings, a list, a link, and an inline
+  image; verify it renders sanitized on the public page.
+- **Verified** (2026-06-27): `/admin/uploads` returns `{url}` and the file serves as
+  an image; a page authored with H2/H3, lists, a link, and an uploaded inline image
+  round-trips through the WYSIWYG and renders sanitized publicly (link `rel` forced,
+  `<script>` stripped); a toolbar edit (insert divider) saved and persisted. TipTap
+  is code-split into its own chunk (lazy-loaded), keeping it off the public bundle.
+
+### Phase 3 — Connectivity ✅
+- Internal `[[slug]]` links + red-link detection; `wiki_links` rebuild on save;
+  backlinks on the article; tags + tag/category filtering.
+- **Exit check**: link page A→B, confirm B shows A under "Linked from"; tag filter works.
+- **Verified** (2026-06-27): internal links authored via an in-editor page picker
+  (links to `/wiki/<slug>`); A→B made B list A under "Linked from"; a link to a
+  non-existent page renders as a red link; removing the link on save cleared the
+  backlink (link index rebuilt). Tags upsert on save, filter via `?tag=` (chips +
+  flat index view), list with published counts, and orphan tags are auto-pruned.
+- Implementation note: links are plain anchors to `/wiki/<slug>` (the WYSIWYG fits
+  this better than `[[ ]]` syntax); the sanitizer also allows `data-wiki-slug`.
+
+### Phase 4 — Discovery & trust ✅
+- FULLTEXT search (public search box + admin filter); revision history list /
+  diff / restore.
+- **Exit check**: search returns expected pages; edit a page twice, diff the
+  revisions, restore an older one, confirm a new revision is recorded.
+- **Verified** (2026-06-27): `?q=` natural-language search matches on both body
+  (`recipes`→crafting) and title (`monsters`); the public search box and admin
+  filter both work. A page edited twice produced 3 revisions; the History modal
+  shows a word-level diff (added vs removed) of an old revision against current;
+  restoring reverted the page and appended a "Restored from revision #N" entry.
+
+### Verification (every phase)
+Use the preview workflow, not manual hand-off: start the dev server, exercise the
+public wiki and the admin editor, check console/network for errors, and capture a
+screenshot of the changed surface. Confirm `npm run` lint/build passes for the client
+and the server boots cleanly with `ensureSchema()` applying the migration.
+
+---
+
+## 9. File-change map (reference)
+
+| Area | Files |
+|---|---|
+| Schema/seed | `server/db/schema.sql`, `server/db/seed.js`, `server/src/utils/db.js` (ensureSchema) |
+| Models | `server/src/model/wiki/wiki.db.js`, `wiki.model.js`, **new** `wiki.links.js` |
+| API | `server/src/router/v1/public/public.{routes,controller}.js`, `server/src/router/v1/admin/admin.{routes,controller}.js` |
+| Sanitize | **new** `server/src/utils/sanitizeHtml.js` |
+| Client API | `client/src/api/client.js` |
+| Public UI | `client/src/routes/wiki/Wiki.jsx`, `WikiArticle.jsx` |
+| Admin UI | `client/src/routes/admin/views/WikiAdmin.jsx`, `WikiEditor.jsx`, **new** category manager + revisions view |
+| Deps | `client/package.json`, `server/package.json` |
+
+---
+
+## 10. Open questions / assumptions
+
+1. **Slug redirects**: assumed not needed on rename (staff wiki). Revisit if pages get
+   external inbound links.
+2. **Search ranking**: FULLTEXT natural-language mode assumed; can switch to BOOLEAN
+   mode if operators are wanted later.
+3. **Diff granularity**: line/word diff of the HTML source is assumed sufficient for
+   revision compare; a rendered visual diff is a later nice-to-have.
+4. **Editor scope**: tables and embeds beyond images are deferred unless requested.