This runbook defines the optional aggregate KPI evidence file consumed by `pnpm monitoring:telemetry:sync`.
\n`monitoring/kpis.json` stays auto-generated. When `monitoring/kpi-evidence.json` is present, telemetry sync uses it for production-like SOTA §26 inputs that cannot be inferred reliably from local seed records:
\nIf the evidence file is missing or invalid, telemetry sync falls back to record-state heuristics and does not manufacture passing values.
\nThe generated file also includes `diagnostics` and `acceptanceRows`.
\nDiagnostics preview the three KPI metric values against the Era C thresholds,
\nname the next evidence needed for any gaps, and include a production-only
\n`capturePlan` with the exact fields and commands operators need for the next
\nexport. The enrichment evidence needs also include a capped
\n`sampleRecordGaps[]` repair sample with record IDs, labels, current unique
\nrecognized authority reference counts, and remaining authority URI gaps. Full
\nauthority URLs and compact IDs such as `aat:...`, `ulan:...`, and `tgn:...`
\nare counted as references when they resolve to a recognized authority family.
\n`diagnostics.handoffSummary`
\ncondenses that packet into ready-row
\ncounts, the remaining metric blocker count, and the next command/evidence pair
\nan operator should run first. Acceptance rows do not replace the Era C KPI
\nthreshold check; they only prove that the aggregate KPI inputs are
\nproduction-shaped enough to hand to telemetry. Strict readiness still requires
\n`pnpm era-c:exit-gate:public` to consume the export and pass the configured
\nthresholds.
\n\n{\n "version": 1,\n "updatedAt": "2026-06-10T00:00:00.000Z",\n "records": {\n "totalRecords": 10000,\n "enrichedRecords": 8300,\n "source": "warehouse materialized records"\n },\n "reconciliation": {\n "scoreDistribution": {\n "autoLink": 920,\n "weeklyDigest": 50,\n "humanReview": 30,\n "dropCandidate": 80\n },\n "reviewedAutoLinkedCandidates": 200,\n "acceptedAutoLinkedCandidates": 190,\n "reviewedPrecisionSource": "production curator review export",\n "source": "production reconciliation export"\n },\n "diagnostics": {\n "metricsPreview": {\n "dataQualityEnrichedShare": 0.83,\n "reconciliationAutoApproveRate": 0.92,\n "reconciliationPrecisionReviewed": 0.95\n },\n "evidenceNeeds": {\n "enrichment": {\n "requiredEnrichedRecords": 8300,\n "remainingEnrichedRecords": 0,\n "totalRecords": 10000,\n "enrichedRecords": 8300,\n "sampleRecordGaps": []\n }\n },\n "blockers": [],\n "capturePlan": {\n "productionOnly": true,\n "rows": [\n {\n "id": "record-enrichment-capture",\n "status": "ready"\n },\n {\n "id": "reconciliation-distribution-capture",\n "status": "ready"\n },\n {\n "id": "reviewed-precision-capture",\n "status": "ready"\n },\n {\n "id": "strict-era-c-refresh",\n "status": "ready"\n }\n ]\n },\n "handoffSummary": {\n "status": "ready-to-refresh",\n "readyAcceptanceRows": 5,\n "totalAcceptanceRows": 5,\n "readyCaptureRows": 4,\n "totalCaptureRows": 4,\n "metricBlockerCount": 0,\n "strictRefreshBlocked": false,\n "nextCommand": "pnpm monitoring:telemetry:public && pnpm era-c:exit-gate:public",\n "nextEvidenceNeeded": "Run public telemetry and Era C exit-gate evidence to consume the KPI export.",\n "nextMetricCommand": null,\n "nextMetricEvidenceNeeded": null\n }\n },\n "acceptanceRows": [\n {\n "id": "production-record-export",\n "status": "ready"\n },\n {\n "id": "reviewed-precision-counts",\n "status": "ready"\n }\n ]\n}\n`pnpm monitoring:kpi-evidence:production` writes these handoff rows:
\n`diagnostics.metricsPreview` mirrors the KPI values telemetry will derive from
\nthe aggregate evidence file. `diagnostics.blockers[]` is empty only when those
\npreviews meet the Era C KPI targets. Blockers currently use these IDs:
\n`diagnostics.capturePlan.rows[]` turns those gaps into a capture checklist:
\n`diagnostics.evidenceNeeds.enrichment.sampleRecordGaps[]` is a repair aid, not
\na separate strict gate. It lists up to 10 under-enriched records sorted by the
\nlargest remaining unique authority reference gap first. Regenerate it with
\n`pnpm monitoring:kpi-evidence:production` before treating the rows as production
\ncleanup evidence.
\n`diagnostics.handoffSummary` is the compact operator entry point. Its
\n`metricBlockerCount` counts only the KPI metric gaps, not the synthetic
\n`strict-era-c-refresh` blocker, so the handoff can say exactly how many input
\nrows still need real production evidence before the strict refresh can succeed.
\nUse `nextCommand` and `nextEvidenceNeeded` as the first remediation step when
\nthe summary status is `needs-production-evidence`. When production source shape
\nand metric thresholds are different problems, `nextMetricCommand` and
\n`nextMetricEvidenceNeeded` point at the first KPI threshold blocker directly
\nwhile `nextCommand` remains the strict source-shape handoff.
\n`pnpm monitoring:kpi-evidence:production` loads the same repo `.env*` files as
\nthe launch evidence scripts before enforcing Postgres storage. Set
\n`METAMUSEUM_SKIP_ENV_FILES=1` only when supplying explicit shell/CI variables.
","updatedAt":"2018-10-20T01:46:40.000Z","checksum":"bc46be9b160e6f77d11a20f401cae2a8e5304f991c1e38ddddd515c6d54538f4","checksumPrefix":"bc46be9b160e","anchorCount":7,"lineCount":194,"rawUrl":"/api/docs/content?path=ops%2Fkpi-evidence.md","htmlUrl":"/docs?doc=ops%2Fkpi-evidence.md","apiUrl":"/api/docs/content?path=ops%2Fkpi-evidence.md"}