Deployment — Vercel + Render
How the Meta Museum system maps onto hosted infrastructure.
| Piece | Host | Config |
|---|---|---|
| Next.js web app | Vercel | `vercel.json` |
| Postgres (system of record) | Neon or Vercel Postgres | `DATABASE_URL` |
| Validation service (pySHACL) | Render | `render.yaml` |
| Reconciliation service (+ Redis) | Render | `render.yaml` |
| AG2 worker (optional, disabled by default) | Render | `render.yaml` |
| Background workers (outbox / publish queue) | Vercel Cron, opt-in drains | `vercel.json`, `/api/cron/*`, `/workers` |
| Solr / GraphDB (Era C search + graph) | Off until projection readiness says `enable` | skipped unless `OUTBOX_PROJECT_TO_*` is explicitly enabled |
A read-only public demo needs only Vercel + Neon. The Render validation and reconciliation services are now deployed for readiness evidence; the workers add async projection/publishing.
Run `pnpm projection:readiness` before enabling Solr/GraphDB target flags. The
portable path remains correct until record volume, search/graph p95, or active
discovery workflow counts cross the documented thresholds.
<!-- BEGIN:PROJECT_STATS -->
<!-- Generated by `pnpm docs:stats`; do not edit by hand. -->
| Generated project stats | Current value |
|---|---|
| Next.js | `16.2.9` |
| React | `19.2.4` |
| App page files | root homepage + `33` non-root page files (`34` total) |
| API route handlers | `146` `app/api` route handlers |
<!-- END:PROJECT_STATS -->
---
Part A — Next.js on Vercel
One-time setup
- Import the repo in Vercel. Framework auto-detects as Next.js.
- `vercel.json` already overrides the build:
- Provision Neon Postgres (or Vercel Postgres) and copy the pooled connection string.
- Set the env vars below (Project → Settings → Environment Variables), then deploy.
- `buildCommand: next build` — critical. The repo's `pnpm build` runs `session:closeout:check` first, which fails 24h after the last close-out and would break every deploy. `next build` skips that guard. Local `pnpm build` is unchanged and still enforces the guard.
Vercel environment variables
| Var | Required | Value |
|---|---|---|
| `DATABASE_URL` | ✅ | Neon pooled connection string |
| `METAMUSEUM_STORAGE_MODE` | ✅ | `postgres` |
| `AUTH_SECRET` | ✅ | `openssl rand -base64 32` |
| `BASE_URL` | ✅ | shareable production URL, for example `https://www.metamuseum.org` |
| `METAMUSEUM_PUBLIC_READ_BASE_URL` | ✅ | exactly the same URL as `BASE_URL` |
| `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET` | ⛔ optional | enables GitHub sign-in; omit for public read-only |
| `VALIDATION_SERVICE_URL` | ✅ for launch readiness | `https://metamuseum-validation.onrender.com/validate` (Part B) |
| `RECONCILIATION_SERVICE_URL` | ✅ for launch readiness | `https://metamuseum-reconciliation.onrender.com` (Part B / ops probes) |
| `METAMUSEUM_EXTERNAL_SERVICE_PROBE_TIMEOUT_MS` | ⛔ optional | default `45000`; allows Render cold starts in preflight |
| `METAMUSEUM_VALIDATION_SERVICE_CAPACITY` | ✅ if strict validation is public on Render | `paid` after upgrading the validation service |
| `METAMUSEUM_VALIDATION_SERVICE_MAX_HEALTH_MS` | ⛔ optional | default `5000`; user-facing strict-validation health budget |
| `VALIDATION_TIMEOUT_MS` | ⛔ optional | default `15000`; runtime timeout for `/api/validate` proxy calls |
| `METAMUSEUM_AG2_BRIDGE_ENABLED` | ⛔ optional | leave unset (bridge stays off) |
| `CRON_SECRET` | ✅ for Vercel Cron | long random value; Vercel sends it as `Authorization: Bearer ...` |
| `METAMUSEUM_OUTBOX_CRON_ENABLED` | ⛔ optional | `1` only when Solr/GraphDB projection is enabled |
| `METAMUSEUM_PUBLISH_QUEUE_CRON_ENABLED` | ⛔ optional | `1` only when wiki/publication queue processing is enabled |
Notes:
storage registry in `src/utils/storage.ts`, which routes those documents to
Postgres in production. `tests/quality/route-storage-write-guard.test.ts`
walks `app/api` imports and fails when a production route can reach raw
filesystem JSON writes or a JSON target not listed in
`MANAGED_STORAGE_DOCUMENT_FILES`.
must stay identical in production. Next.js resolves Open Graph image URLs
against the public-read metadata base, and `pnpm launch:preflight:production`
now fails if the two variables diverge or if the public-read base is missing.
After every domain change, redeploy, run `pnpm smoke:crawler-preview`, and
refresh the homepage in Facebook Sharing Debugger plus the equivalent
LinkedIn/X crawler preview tools.
optional, but their mode is no longer implicit. When service URLs are unset,
deployment preflight records the local fallback path: non-strict validation
falls back to the local Linked Art profile checker, strict `/api/validate`
still needs a running validation service, and the app uses in-process
deterministic reconciliation. When either URL is set, preflight probes
`/health` with `METAMUSEUM_EXTERNAL_SERVICE_PROBE_TIMEOUT_MS` and records
status plus duration for launch evidence. Health probes intentionally tolerate
cold starts, but user-facing strict `/api/validate` has a separate runtime
timeout and production preflight fails an `onrender.com` validator unless
`METAMUSEUM_VALIDATION_SERVICE_CAPACITY` proves paid/no-sleep capacity and
the `/health` duration stays within `METAMUSEUM_VALIDATION_SERVICE_MAX_HEALTH_MS`.
- Storage self-heals on a fresh DB. Core stores (`records.ts`) catch a missing Postgres document and seed an empty `[]`, so an empty Neon DB won't crash the app.
- Runtime is Node, not Edge for API routes (required by `pg` + Auth.js v5) — this is the App Router default; no action needed.
- Read-only FS guard: request-path JSON writes must go through the managed
- Social preview base URL: `BASE_URL` and `METAMUSEUM_PUBLIC_READ_BASE_URL`
- External service mode: validation and reconciliation Render services are
---
Part B — Python services on Render
`render.yaml` is a Blueprint defining the deployed FastAPI validation and
reconciliation services plus the Redis cache used by reconciliation.
Current deployed endpoints:
- Validation health: `https://metamuseum-validation.onrender.com/health`
- Validation API: `https://metamuseum-validation.onrender.com/validate`
- Reconciliation health: `https://metamuseum-reconciliation.onrender.com/health`
- Reconciliation API base: `https://metamuseum-reconciliation.onrender.com`
Setup
- In Render: New → Blueprint, point at this repo. It reads `render.yaml`.
- Each service: own `rootDir`, `pip install -r requirements.txt`, and
`uvicorn main:APP --host 0.0.0.0 --port $PORT` (bypasses the hardcoded
`127.0.0.1` in each `__main__`). Health check: `/health`.
- Redis (`metamuseum-reconcile-cache`) is wired into the reconciliation
service via `RECONCILIATION_REDIS_URL` automatically.
- AG2 worker is optional — delete that block from `render.yaml` unless you
plan to enable the bridge.
Wire services back to Vercel
Set these URLs on Vercel so deployment preflight captures live Render health
evidence:
is upgraded off sleeping/free-tier capacity for user-facing strict validation
when operator tooling or smoke evidence should use the deployed reconciliation service
- `VALIDATION_SERVICE_URL = https://metamuseum-validation.onrender.com/validate`
- `METAMUSEUM_VALIDATION_SERVICE_CAPACITY = paid` after the validation service
- `RECONCILIATION_SERVICE_URL = https://metamuseum-reconciliation.onrender.com`
The reconciliation service is invoked by operator scripts/ops tooling rather
than the Next runtime; point those at `https://metamuseum-reconciliation.onrender.com`.
Free-tier caveat: Render free web services sleep after inactivity and
cold-start in ~30–60s. Fine for a demo; upgrade validation to paid/no-sleep
capacity before exposing strict `/api/validate` to production users, then set
`METAMUSEUM_VALIDATION_SERVICE_CAPACITY=paid` and rerun preflight.
---
Part C — Background workers
Vercel cannot run persistent Node processes, so the app uses secured Vercel
Cron routes for bounded drain cycles:
| Worker | Script | Needed when |
|---|---|---|
| Outbox projector | `pnpm outbox:projector` | Solr/GraphDB projection is enabled (off in this deploy) |
| Publish queue worker | `pnpm publish:queue:worker` | Wiki/publication publishing is used |
Configured cron routes:
- `/api/cron/outbox` every 5 minutes.
- `/api/cron/publish` every 10 minutes.
Both routes return `cache-control: no-store`. The schedules may be deployed
safely while the workers are disabled: disabled drains return `200` no-op JSON
instead of `503`, so Vercel Observability does not count intentional idle
schedules as production errors. Once a worker drain is enabled, that route
requires `Authorization: Bearer $CRON_SECRET` before any work can run:
Tune with `METAMUSEUM_OUTBOX_CRON_LIMIT` and optional
`METAMUSEUM_OUTBOX_CRON_WORKER_ID`. Projection still requires the usual
`OUTBOX_PROJECT_TO_SOLR` / `OUTBOX_PROJECT_TO_GRAPHDB` target env vars; the
cron route will not claim events until at least one target is explicitly `1`.
Direct projector scripts use the same target gates and skip Solr/GraphDB when
those vars are unset.
batch. Tune with `METAMUSEUM_PUBLISH_QUEUE_CRON_MAX_ITEMS`. The default
queue document is `storage/publish-queue.json`, which is a managed storage
document and therefore Postgres-backed in Vercel `postgres` mode.
- `METAMUSEUM_OUTBOX_CRON_ENABLED=1` drains one bounded outbox projector batch.
- `METAMUSEUM_PUBLISH_QUEUE_CRON_ENABLED=1` processes a bounded publish queue
Scale enablement guard:
`/api/cron/outbox` is deployed, `CRON_SECRET` is set, and
`METAMUSEUM_OUTBOX_CRON_ENABLED=1` is ready to drain at the configured
cadence. First run `pnpm projection:readiness`; production preflight fails if
projection targets are enabled without that scheduler.
`/api/cron/publish` is deployed, `CRON_SECRET` is set, and
`METAMUSEUM_PUBLISH_QUEUE_CRON_ENABLED=1` is ready. Production preflight
fails if live publication endpoints or bot tokens are configured without the
publish queue scheduler.
- Keep `OUTBOX_PROJECT_TO_SOLR` and `OUTBOX_PROJECT_TO_GRAPHDB` off until
- Keep live MediaWiki/Wikibase API URLs and bot tokens out of production until
Operator status:
next drain time, and disabled projection/publish mode.
review-goals, launch, long-term, ActivityStreams, pilot, DR, and
disabled-system drill artifacts.
agent checks. Use it after worker env changes so scheduled-but-disabled
features do not look as if they are progressing.
`pnpm staging:disabled-drill:check` weekly (`20 10 1`) while AG2,
Solr/GraphDB projection, and publication drains are intentionally off; run
the same command manually after staging env changes. It writes
`artifacts/staging-disabled-systems/latest.json`, uploads timestamped run
artifacts, confirms disabled paths do not make external network calls,
mock-exercises the first enabled AG2, Solr/GraphDB projection, and
publication webhook branches, and keeps latent worker/queue regressions
visible before real enablement.
- `/workers` renders worker lag, backlog, next Vercel Cron wakeup, effective
- `/readiness` renders a compact editor-gated status view over the latest
- `/api/workers/status` exposes the same no-store JSON snapshot for probes or
- `.github/workflows/staging-disabled-systems-drill.yml` runs
For heavier projection loads, keep these cron routes disabled and run the same
scripts as a Render background worker instead:
- `pnpm outbox:projector:once`
- `pnpm publish:queue:worker:once`
---
Quick path to a live demo
- Neon DB → copy pooled `DATABASE_URL`.
- Vercel import → set `DATABASE_URL`, `METAMUSEUM_STORAGE_MODE=postgres`,
`AUTH_SECRET`, `BASE_URL`, `METAMUSEUM_PUBLIC_READ_BASE_URL` → deploy.
- Render Blueprint services are live → set `VALIDATION_SERVICE_URL` and
`RECONCILIATION_SERVICE_URL` on Vercel → redeploy.
- Run `BASE_URL=https://<shareable-url> METAMUSEUM_PUBLIC_READ_BASE_URL=https://<shareable-url> VALIDATION_SERVICE_URL=https://metamuseum-validation.onrender.com/validate METAMUSEUM_VALIDATION_SERVICE_CAPACITY=paid RECONCILIATION_SERVICE_URL=https://metamuseum-reconciliation.onrender.com pnpm launch:preflight:production`; it will capture configured Render service `/health` status/duration and fail if strict validation is still on sleeping capacity or outside the user-facing health budget. Then run `BASE_URL=https://<shareable-url> pnpm smoke:crawler-preview` and re-scrape social previews in crawler debug tools.
- Set `CRON_SECRET`; leave worker enable flags off until projection/publishing is needed.