MetaMuseum
Menu
← Documentation home

Canonical Markdown source · Oct 20, 2018

External Evidence Request Pack

ops/external-evidence-request-pack.md · 246 lines · SHA-256 4d89c9ef0410

Use this pack when moving the remaining 10/10 blockers from scaffolded proof to real public evidence. Do not use demo, fixture, placeholder, sample, smoke, test, localhost, or private-network evidence for these gates.

Current Order

  1. Keep the nightly Era C workflow green and committing rolling evidence inputs.
  1. Let public SLO and uptime windows fill to 30 distinct days.
  1. Send one real pilot outreach and record it.
  1. Generate the buyer-specific pilot capture pack, convert one pilot to an invoice-backed manual entitlement, then record activation, support, KPI, retention, and gross-margin evidence.
  1. Ask three external ActivityStreams consumers to read `/api/activity` with stable declared IDs and enough page depth to observe `Create`, `Update`, and `Delete` activity types.
  1. Ask those same consumers to register durable HTTPS callbacks and provide recent 2xx verification evidence.
  1. Rerun `pnpm review:goals:local`; use `pnpm review:goals:check` only when strict external evidence is expected to pass.

Pilot Ask

Send a short pilot note like:


Subject: Managed Linked Art launch pilot

We are inviting one collection partner into a managed Linked Art launch pilot for Meta Museum.

Scope:
- ingest one collection export into a hosted workspace
- produce a Linked Art API/public browse surface
- deliver a data-quality and rights/provenance review packet
- include one monthly evidence packet with activation, support, KPI, retention, and gross-margin proof

If this is useful, the next step is a signed fixed-scope pilot and manual invoice reference so we can create the tenant entitlement and start the evidence ledger.

Record sent outreach:


pnpm pilot:outreach --account=<real-account-id> --stage=sent --occurred-at=<ISO timestamp> --owner=<owner> --evidence=channel=<email-or-crm> --evidence=messageRef=<crm-or-email-ref> --evidence=followUpAt=<YYYY-MM-DD>

Generate the buyer-specific capture checklist before creating the packet:


pnpm pilot:buyer-pack -- --account <real-account-id> --tenant <tenant-id> --organization "<buyer organization>" --invoice-ref <real-invoice-ref> --owner "<owner>"

This writes `artifacts/pilot-evidence/pilot-buyer-onboarding-pack-latest.md`,

`.json`, and a timestamped

`artifacts/pilot-evidence/runs/pilot-buyer-onboarding-pack-<timestamp>.json`

audit copy with the exact outreach, activation, support, KPI, packaging, and

packet verification commands. Without real buyer and invoice fields, the pack

remains `needs-buyer-evidence`.

With real buyer fields, use the generated `acceptanceRows` as the paid-pilot

ledger. Each row stays pending until outreach, invoice-backed entitlement,

activation, support load, KPI completeness, retention signal, gross-margin

proof, and the refreshed strict pilot packet all point to the same real account

and tenant.

After a real pilot is signed, create the manual entitlement through the existing operator path, then record activation and KPI evidence with:


pnpm pilot:activation --tenant=<tenant-id> --event=<activation-event> --occurred-at=<ISO timestamp> --evidence=tenantId=<tenant-id> --evidence=evidenceRef=<real-evidence-ref>
pnpm pilot:support --tenant=<tenant-id> --issue=<real-issue-id> --opened-at=<ISO timestamp> --requester="<buyer contact>" --severity=normal --title="<support summary>" --decision-owner="<owner>" --next-response-at=<ISO timestamp> --status=resolved --resolved-at=<ISO timestamp> --resolution-summary="<resolution summary>"
pnpm pilot:support --tenant=<tenant-id> --summary
pnpm pilot:kpi --tenant=<tenant-id> --metric=<metric-id> --value=<number> --measured-at=<ISO timestamp> --source=<real-source> --evidence-ref=<real-evidence-ref> --owner=<owner>
pnpm pilot:evidence --tenant=<tenant-id> --account=<real-account-id> --markdown --check

Required KPI metrics:

  • `pilot.first_value_days`
  • `pilot.imported_record_count`
  • `pilot.customer_view_count`
  • `pilot.support_minutes`
  • `pilot.validation_critical_open_count`
  • `pilot.retention_signal_count`
  • `pilot.gross_margin_percent`

The final packet must pass because the tenant has real invoice-backed billing, outreach, activation, support, KPI, retention, and gross-margin evidence. A researched account or internal demo is not enough.

SOTA Section 26 KPI Evidence

Export production KPI inputs with:


pnpm monitoring:kpi-evidence:production -- --auto-link <count> --weekly-digest <count> --human-review <count> --drop-candidate <count> --reviewed-auto-linked <count> --accepted-auto-linked <count> --reviewed-precision-source "production curator review export"

Then inspect `monitoring/kpi-evidence.json`. Start with

`diagnostics.handoffSummary`; it should report `ready-to-refresh`, `5/5`

acceptance rows ready, `4/4` capture rows ready, `0` metric blockers, and the

next command pair for public telemetry plus Era C. Its `acceptanceRows` must all

be `ready`: production record export, record-enrichment denominator, production

reconciliation export, reviewed precision counts/source, and strict Era C

refresh readiness. These rows only prove the input is production-shaped enough

for telemetry; strict KPI proof still requires `pnpm monitoring:telemetry:public`

and `pnpm era-c:exit-gate:public` to pass the configured thresholds.

ActivityStreams Consumer Ask

If a broad community ask has already been posted, record the outreach context

first. This preserves the trail for operators, but it does not satisfy

ActivityStreams adoption proof:


pnpm activity:community-outreach -- --channel "<community/channel>" --message-ref "<permalink-or-thread-ref>" --posted-at "<ISO timestamp>" --owner "<owner>"

The generated `activity-community-outreach` artifact remains `strictEvidence:

false`. Strict proof starts only when three partner-owned consumer IDs produce

feed-read telemetry and durable callback evidence.

Generate the current partner handoff artifact first:


pnpm activity:partner-pack -- --consumer-id <id-1> --consumer-id <id-2> --consumer-id <id-3>

This writes `artifacts/activity-syndication/activity-partner-onboarding-pack-latest.md`,

`.json`, and a timestamped

`artifacts/activity-syndication/runs/activity-partner-onboarding-pack-<timestamp>.json`

audit copy with per-consumer feed-read and durable-subscription curl requests.

Without three real partner-owned IDs, the pack remains `needs-consumer-ids`.

With real IDs, use the generated `acceptanceRows` as the per-partner ledger:

each row remains pending until declared feed-read evidence, Create/Update/Delete

activity-type coverage, a durable HTTPS subscription, recent 2xx callback proof,

and the refreshed `pnpm activity:syndication:evidence` packet all agree on the

same partner-owned consumer ID.

Send each external consumer this request:


Please read the Meta Museum ActivityStreams feed with your stable consumer ID so we can verify external syndication adoption:

curl -H "Accept: application/activity+json" \
  -H "x-linked-art-consumer-id: <your-stable-consumer-id>" \
  "https://www.metamuseum.org/api/activity?limit=100"

curl -H "Accept: application/activity+json" \
  -H "x-linked-art-consumer-id: <your-stable-consumer-id>" \
  "https://www.metamuseum.org/api/activity?limit=100&type=Create"

curl -H "Accept: application/activity+json" \
  -H "x-linked-art-consumer-id: <your-stable-consumer-id>" \
  "https://www.metamuseum.org/api/activity?limit=100&type=Update"

curl -H "Accept: application/activity+json" \
  -H "x-linked-art-consumer-id: <your-stable-consumer-id>" \
  "https://www.metamuseum.org/api/activity?limit=100&type=Delete"

Use an ID owned by your system, not a test or demo ID. We will verify it appears as a declared external consumer. If `Update` or `Delete` returns zero items, report that explicitly; strict coverage waits until real update or tombstone rows exist in the public feed.

Verify all three IDs:


pnpm activity:adoption:matrix -- --base-url https://www.metamuseum.org --consumer-id <id-1> --consumer-id <id-2> --consumer-id <id-3>

If the public feed still lacks `Update` or `Delete` rows, create only real

source events before asking partners to re-read them. A reviewed existing-record

metadata correction emits `Update`; a confirmed upstream `404`/`410` tombstone

can be recorded with:


pnpm activity:tombstone:evidence -- --object-id <canonical-record-uri> --provider <provider-key> --upstream-status 410 --observed-at <ISO timestamp> --evidence-ref artifacts/activity-syndication/<real-upstream-404-or-410-proof>.json

Durable Subscription Ask

After the same consumers are observed, ask each to register a callback:


curl -X POST "https://www.metamuseum.org/api/activity/subscriptions" \
  -H "content-type: application/json" \
  -H "accept: application/activity+json" \
  -H "x-linked-art-consumer-id: <your-stable-consumer-id>" \
  --data '{
    "callbackUrl": "https://<public-consumer-host>/<activity-callback-path>",
    "eventTypes": ["Create", "Update", "Delete"]
  }'

Then attach recent callback verification evidence to the subscription ledger with a public callback host, 2xx `statusCode`, `verifiedAt`, and a real `evidenceRef`. Recheck:


pnpm activity:subscription:verify -- --consumer-id <your-stable-consumer-id> --callback-url https://<public-consumer-host>/<activity-callback-path> --status-code 204 --verified-at <ISO timestamp> --evidence-ref artifacts/activity-syndication/<real-callback-proof>.json
pnpm activity:syndication:evidence
pnpm activity:syndication:check
pnpm era-c:exit-gate:public

The durable gate passes only when the same three declared external consumer IDs have observed feed-read telemetry covering `Create`, `Update`, and `Delete`, active HTTPS subscriptions covering those same event types, and recent 2xx callback verification evidence.

MetaHistoryBook's verified callback row:


curl -X POST "https://www.metamuseum.org/api/activity/subscriptions" \
  -H "content-type: application/json" \
  -H "accept: application/activity+json" \
  -H "x-linked-art-consumer-id: metahistorybook-harvester-prod" \
  --data '{
    "callbackUrl": "https://www.metahistorybook.com/api/metamuseum/callback",
    "eventTypes": ["Create", "Update", "Delete"],
    "signingRequired": true,
    "signingSecretRef": "metahistorybook-harvester-prod-v1"
  }'

MetaMuseum rechecked liveness, sent one signed production `Create` Activity

payload, and received `204` from that callback at

`2026-07-05T17:13:13.312Z`. The proof artifact intentionally omits the HMAC

secret and signature value:


pnpm activity:subscription:verify -- --consumer-id metahistorybook-harvester-prod --callback-url https://www.metahistorybook.com/api/metamuseum/callback --status-code 204 --verified-at 2026-07-05T17:13:13.312Z --evidence-ref artifacts/activity-syndication/metahistorybook-harvester-prod-callback-2026-07-05T17-13-13-312Z.json

Their 2026-07-05T16:37Z typed read reported `Create: 108`, `Update: 0`, and

`Delete: 0`; the explicit zeroes should remain filed as honest partner feedback,

not as complete activity-type coverage.

Reliability Evidence

Nightly `.github/workflows/era-c-exit-gate-evidence.yml` now commits compact rolling evidence inputs back to `main`, including the k6 trend and public uptime history. Keep these repository variables configured:

  • `METAMUSEUM_EVIDENCE_BASE_URL=https://www.metamuseum.org`
  • `METAMUSEUM_PUBLIC_READ_BASE_URL=https://www.metamuseum.org`
  • `METAMUSEUM_EVIDENCE_IIIF_TILE_URL=<public tile/static image URL>`
  • `METAMUSEUM_ACTIVITY_CONSUMER_IDS=<id-1>,<id-2>,<id-3>` after real consumers exist

Daily review:


pnpm longterm:evidence:public
pnpm review:goals:local

`pnpm longterm:evidence:public` also runs the daily maintenance companion for

the runway. It identifies failed SLO samples, failed public-read probe paths,

passing distinct evidence days, the earliest failure-free window date, and the

real ActivityStreams consumers still needed; it does not mutate the evidence

history. Use the generated `acceptanceRows` as the 30-day reliability ledger:

SLO sample depth, SLO failure-free window, uptime observation depth, uptime

availability, ActivityStreams consumer depth, activity-type coverage, and strict

long-term packet refresh must all be `ready` before this lane can support a

strict production-readiness claim.