{"id":"ops/era-c-exit-gate-evidence","relativePath":"ops/era-c-exit-gate-evidence.md","title":"Era C Exit-Gate Evidence Pack","markdown":"# Era C Exit-Gate Evidence Pack\n\nThis runbook turns the four Era C exit checks into automated, dated pass/fail evidence.\n\n## Scope\n\nThe evidence pack evaluates:\n\n1. SOTA §20.4 p95 SLOs over a rolling 30-day window: cached Record, cold Record, keyword+facet search, whitelisted SPARQL, and IIIF tile serving.\n2. Public-read uptime >= 99.9% over 30 days.\n3. >= 3 declared external consumers of `/api/activity` within the adoption window.\n4. SOTA §26 KPI thresholds.\n\n## Artifact schema\n\nCanonical schema:\n\n- `docs/schemas/era-c-exit-gate-evidence.schema.json`\n\nLatest and historical artifacts:\n\n- `artifacts/exit-gate/era-c-exit-gate-latest.json`\n- `artifacts/exit-gate/runs/era-c-exit-gate-.json`\n- `artifacts/exit-gate/trend-index.json`\n\nThe trend index stores each check's pass/fail state plus compact `failedChecks` reasons so agents can identify the next blocking gate without opening every historical run artifact.\n\nSupporting performance trend input:\n\n- `artifacts/performance/k6-slo-summary.json`\n- `artifacts/performance/k6-slo-trend.json`\n\nNew performance samples must include all five p95 metrics. Legacy three-metric samples remain in history, but they fail the all-SLO gate until replaced by 30 days of complete samples.\nExit-gate reports include `missingMetricsInWindow` plus per-sample `metricDetails` so partial k6 summaries are distinguishable from actual p95 threshold breaches.\n\n## Policy and inputs\n\nPolicy thresholds:\n\n- `config/era-c-exit-gate-policy.json`\n - Public-read uptime evidence must be recent: `uptime.maxSnapshotAgeHours` defaults to `48`, so stale or undated snapshots cannot satisfy the 99.9% gate.\n - SOTA §26 KPI evidence must be recent: `kpis26.maxSnapshotAgeHours` defaults to `48`.\n - AI query cost must come from real usage telemetry when `kpis26.requireAiQueryCostTelemetry` is `true`; fallback default cost values do not satisfy the KPI gate.\n - AI query rows written by `/api/ai/query` include `costUsd`, `costCurrency`, `costSource`, and usage counts. The deterministic local planner records `costUsd: 0` with `costSource: \"deterministic-local-planner\"` because it makes no paid model call.\n\nMetric inputs:\n\n- Uptime snapshot: `monitoring/public-read-uptime.json`\n - Exit-gate reports include the uptime `source` (`prometheus`, `probe`, or `unavailable`) and source `notes` so missing 30-day proof is actionable from the artifact alone.\n- Uptime history: `monitoring/public-read-uptime-history.json`\n- KPI snapshot: `monitoring/kpis.json`\n - Exit-gate reports include KPI `sources`, snapshot `notes`, and per-failed-metric details (`metric`, `source`, `reason`) so SOTA §26 blockers can be triaged from the evidence artifact.\n - Optional aggregate KPI input: `monitoring/kpi-evidence.json` supplies production-like record-enrichment and reconciliation review counts before `pnpm monitoring:telemetry:sync`; see `docs/ops/kpi-evidence.md`.\n - `aiQueryCostUsd` is calculated from `storage/ai-query-log.json`; run at least one representative `/api/ai/query` request in the target environment before telemetry sync so the source is no longer the fallback default.\n- Activity adoption telemetry: `storage/activity-consumers.json`\n - Declared consumer rows must preserve `class: \"declared\"` and `declaredId`; the exit-gate normalizer intentionally ignores derived fingerprints for adoption credit.\n - Proof command: `pnpm activity:adoption:probe` records one partner-owned declared consumer and writes `artifacts/activity-adoption/activity-adoption-proof-latest.json`.\n - Matrix proof command: `pnpm activity:adoption:matrix` records/checks three partner-owned declared consumers and writes `artifacts/activity-adoption/activity-adoption-matrix-latest.json`.\n\nActivity adoption counts only declared external consumers (`x-linked-art-consumer-id`). Derived fingerprints remain useful diagnostics, but they do not satisfy the external Linked Art systems exit gate.\nSee `docs/ops/activity-adoption-proof.md` for partner hand-off instructions and placeholder-ID safeguards.\n\n## Commands\n\nRefresh telemetry snapshots first:\n\n```bash\npnpm monitoring:telemetry:sync\n```\n\nGenerate evidence report (always writes artifact):\n\n```bash\npnpm era-c:exit-gate:evidence\n```\n\nEnforce as a hard gate (non-zero exit on any failed check):\n\n```bash\npnpm era-c:exit-gate:check\n```\n\n## Nightly automation\n\nWorkflow:\n\n- `.github/workflows/era-c-exit-gate-evidence.yml`\n\nNightly job does:\n\n1. Run `pnpm k6:slo` against the deployed target when `METAMUSEUM_EVIDENCE_BASE_URL` and `METAMUSEUM_EVIDENCE_IIIF_TILE_URL` are configured; otherwise run local `pnpm k6:slo:ci` as a fallback.\n2. Seed one deployed `/api/ai/query` request when `METAMUSEUM_EVIDENCE_BASE_URL` is configured so cost telemetry can flow into SOTA §26 KPI evidence.\n3. Probe the declared `/api/activity` adoption matrix when `METAMUSEUM_ACTIVITY_CONSUMER_IDS` is configured; fall back to one declared consumer when only `METAMUSEUM_ACTIVITY_CONSUMER_ID` is configured.\n4. Run `pnpm era-c:exit-gate:evidence` (auto-runs telemetry sync).\n5. Upload exit-gate, performance, activity-adoption, uptime, and KPI artifacts.\n\nThe workflow always uploads artifacts so failed checks still produce dated proof.\n\nDeployed-target workflow variables:\n\n- `METAMUSEUM_EVIDENCE_BASE_URL` — deployed app URL used as `BASE_URL` for k6, AI query seeding, and adoption proof.\n- `METAMUSEUM_EVIDENCE_IIIF_TILE_URL` — CDN-backed IIIF tile used as `IIIF_TILE_URL` for the tile-serving p95 check.\n- `METAMUSEUM_EVIDENCE_SPARQL_URL` — whitelisted SPARQL endpoint used as `SPARQL_URL` by `pnpm k6:slo`.\n- `METAMUSEUM_EVIDENCE_SPARQL_QUERY` — optional whitelisted SPARQL query override for `pnpm k6:slo`.\n- `METAMUSEUM_EVIDENCE_AI_QUERY` — optional representative query for the deployed `/api/ai/query` telemetry seed.\n- `METAMUSEUM_ACTIVITY_CONSUMER_IDS` — comma-separated partner-owned declared consumer IDs for the full adoption matrix proof.\n- `METAMUSEUM_ACTIVITY_CONSUMER_ID` — partner-owned declared consumer ID for the single-consumer adoption proof fallback.\n\n## Telemetry source configuration\n\nPrimary uptime source (recommended):\n\n- `METAMUSEUM_UPTIME_PROMETHEUS_URL`\n- `METAMUSEUM_UPTIME_PROMQL_AVAILABILITY`\n- `METAMUSEUM_UPTIME_PROMQL_SAMPLE_COUNT`\n\nGitHub Actions wiring:\n\n- secret: `METAMUSEUM_UPTIME_PROMETHEUS_URL`\n- repo/org var: `METAMUSEUM_UPTIME_PROMQL_AVAILABILITY`\n- repo/org var: `METAMUSEUM_UPTIME_PROMQL_SAMPLE_COUNT`\n\nFallback uptime source:\n\n- `METAMUSEUM_PUBLIC_READ_BASE_URL`\n- `METAMUSEUM_UPTIME_PROBE_PATHS`\n- `METAMUSEUM_UPTIME_PROBE_TIMEOUT_MS`\n\nWindow controls:\n\n- `METAMUSEUM_UPTIME_WINDOW_DAYS` (default `30`)\n- `METAMUSEUM_UPTIME_HISTORY_RETENTION_DAYS` (default `45`)\n","sections":[{"level":2,"heading":"Scope","anchor":"scope"},{"level":2,"heading":"Artifact schema","anchor":"artifact-schema"},{"level":2,"heading":"Policy and inputs","anchor":"policy-and-inputs"},{"level":2,"heading":"Commands","anchor":"commands"},{"level":2,"heading":"Nightly automation","anchor":"nightly-automation"},{"level":2,"heading":"Telemetry source configuration","anchor":"telemetry-source-configuration"}],"html":"

Era C Exit-Gate Evidence Pack

\n

This runbook turns the four Era C exit checks into automated, dated pass/fail evidence.

\n

Scope

\n

The evidence pack evaluates:

\n
  1. SOTA §20.4 p95 SLOs over a rolling 30-day window: cached Record, cold Record, keyword+facet search, whitelisted SPARQL, and IIIF tile serving.
\n
  1. Public-read uptime >= 99.9% over 30 days.
\n
  1. >= 3 declared external consumers of `/api/activity` within the adoption window.
\n
  1. SOTA §26 KPI thresholds.
\n

Artifact schema

\n

Canonical schema:

\n\n

Latest and historical artifacts:

\n\n

The trend index stores each check's pass/fail state plus compact `failedChecks` reasons so agents can identify the next blocking gate without opening every historical run artifact.

\n

Supporting performance trend input:

\n\n

New performance samples must include all five p95 metrics. Legacy three-metric samples remain in history, but they fail the all-SLO gate until replaced by 30 days of complete samples.

\n

Exit-gate reports include `missingMetricsInWindow` plus per-sample `metricDetails` so partial k6 summaries are distinguishable from actual p95 threshold breaches.

\n

Policy and inputs

\n

Policy thresholds:

\n\n

Metric inputs:

\n\n

Activity adoption counts only declared external consumers (`x-linked-art-consumer-id`). Derived fingerprints remain useful diagnostics, but they do not satisfy the external Linked Art systems exit gate.

\n

See `docs/ops/activity-adoption-proof.md` for partner hand-off instructions and placeholder-ID safeguards.

\n

Commands

\n

Refresh telemetry snapshots first:

\n
\npnpm monitoring:telemetry:sync\n
\n

Generate evidence report (always writes artifact):

\n
\npnpm era-c:exit-gate:evidence\n
\n

Enforce as a hard gate (non-zero exit on any failed check):

\n
\npnpm era-c:exit-gate:check\n
\n

Nightly automation

\n

Workflow:

\n\n

Nightly job does:

\n
  1. Run `pnpm k6:slo` against the deployed target when `METAMUSEUM_EVIDENCE_BASE_URL` and `METAMUSEUM_EVIDENCE_IIIF_TILE_URL` are configured; otherwise run local `pnpm k6:slo:ci` as a fallback.
\n
  1. Seed one deployed `/api/ai/query` request when `METAMUSEUM_EVIDENCE_BASE_URL` is configured so cost telemetry can flow into SOTA §26 KPI evidence.
\n
  1. Probe the declared `/api/activity` adoption matrix when `METAMUSEUM_ACTIVITY_CONSUMER_IDS` is configured; fall back to one declared consumer when only `METAMUSEUM_ACTIVITY_CONSUMER_ID` is configured.
\n
  1. Run `pnpm era-c:exit-gate:evidence` (auto-runs telemetry sync).
\n
  1. Upload exit-gate, performance, activity-adoption, uptime, and KPI artifacts.
\n

The workflow always uploads artifacts so failed checks still produce dated proof.

\n

Deployed-target workflow variables:

\n\n

Telemetry source configuration

\n

Primary uptime source (recommended):

\n\n

GitHub Actions wiring:

\n\n

Fallback uptime source:

\n\n

Window controls:

\n","updatedAt":"2018-10-20T01:46:40.000Z","checksum":"656b9c7f85c62b4efa1d927e02f8a1c7185098614cf43e63293d9e8f0c01677c","checksumPrefix":"656b9c7f85c6","anchorCount":6,"lineCount":133,"rawUrl":"/api/docs/content?path=ops%2Fera-c-exit-gate-evidence.md","htmlUrl":"/docs?doc=ops%2Fera-c-exit-gate-evidence.md","apiUrl":"/api/docs/content?path=ops%2Fera-c-exit-gate-evidence.md"}