Meta Museum's A+ validation claim requires more than a successful usability session. The frozen Rosenberg protocol compares a conventional external-source workflow with Meta Museum, requires three genuine qualified researchers, and separately measures five non-specialist readers with two independent human coders. The A+ projection adds the missing qualification, subject-expert, correction, novelty, and version controls.
Commands and artifacts
- Participants export device-local responses from the existing validation pages. Nothing is uploaded automatically.
- An operator uses `pnpm research:validation:assemble -- <manifest.json> <new-output.json>` to create a non-overwriting, receipt-hashed bundle.
- The operator records only the attributable HTTPS source/receipt,
measurement time, pseudonymous qualification attestations, and versioned
subject-expert correction decision in a private version-1 evidence-envelope
JSON file. The checked-in configuration remains an optional scheduled-run
contract; do not place returned personal data in it.
Before invoking the CLI, an editor may inspect both files in the private
`/research-validation-preflight` workspace. Parsing and hashing stay in the
browser; its privacy-safe receipt is advisory and never establishes
qualification or validation.
- Run
`pnpm research:validation:readiness -- --bundle=<validation-bundle.json> --evidence=<private-evidence-envelope.json>`.
The command hashes the frozen protocol and exact returned bundle, validates
the genuine evidence, and writes
`artifacts/research-a-plus/validation/research-validation-readiness-latest.json`.
Use `--evaluated-at=<ISO>` and `--output=<directory>` for a controlled run.
- The readiness artifact carries its own canonical integrity digest. On the
next `pnpm research:a-plus:readiness` run, a passing untampered artifact
automatically supplies only the investigation and reviewer fields. The
separately receipt-reconciled conventional-baseline artifact remains
authoritative for measured improvement; no manual metric copy is accepted.
The command also derives the canonical current Research Commons release
digest into the projected investigation. A+ requires that digest to match
the open-research release being scored, so review of an older workflow cannot
validate a changed implementation.
Qualification evidence
Every researcher and subject expert needs exactly one facilitator-verified attestation. It contains a pseudonymous participant code, role, substantive qualification statement, facilitator code, consent and independence flags, verification time, HTTPS evidence reference, receipt digest, and `synthetic: false`. Names, email addresses, phone numbers, postal addresses, contacts, and IP addresses are rejected from the retained projection. The facilitator holds any identity-to-code mapping privately under an approved retention policy.
Qualification rows must originate from replay-verified facilitator artifacts. Each
artifact retains the signed public-return candidate and a distinct, later facilitator
decision with an attributable HTTPS verification receipt. Validation readiness rejects
inline rows, modified candidates or decisions, packet mismatches, incomplete checks,
and any derived attestation that does not reproduce exactly.
The three `RES-…` codes must correspond exactly to the three researcher sessions. The independent `REV-…` code is the subject expert. Self-reported eligibility or polished notes alone cannot satisfy the requirement.
Material correction and versioning
The subject expert must bind the reviewed dossier v1 digest to a different v2 packet digest, identify at least one material correction, give a substantive correction summary, decide whether novelty is established, not established, or inconclusive, and confirm that limitations are published. A correction string inside the original review does not count until it is incorporated into the distinct v2 packet.
The material-correction row cannot be entered inline. A signed decision artifact must
replay the exact original packet and correction envelope, reconstruct the unapproved
v2 candidate, bind the same verified subject expert, and retain a later attributable
human acceptance, novelty, and limitations decision. Edited v1/v2 digests, revision
numbers, novelty, limitations, checks, reviewer, or authority fail before projection.
Comparison threshold
The existing counterbalanced study must pass its frozen requirements: at least three researcher sessions, at least 20% median time improvement, and at least 80% successful reproduction with three distinct HTTPS sources. Five non-specialists must also reach 80% independently coded comprehension. The projection uses the frozen protocol digest rather than allowing thresholds to change after results are observed.
Authority boundary
No command in this program recruits, contacts, identifies, compensates, or approves a participant; publishes research; establishes novelty; or authorizes outreach, email, or payment. Empty evidence remains a blocked state, and synthetic examples or self-review cannot be substituted for returned external evidence.
Changing a projected reviewer, investigation, status, blocker, authority flag,
or correction after the readiness command invalidates the artifact digest and
prevents automatic A+ ingestion. The exact raw validation-bundle SHA-256 is
also retained inside the signed readiness artifact; substituting a different
returned bundle therefore changes the evidence identity even when summary
metrics happen to match.
Every facilitator qualification attestation and the material-correction decision
must occur no later than the declared study measurement time. The correction must
also follow both the independent expert review and assembled returned-evidence
bundle. Future-dated qualifications or retrospective pre-review corrections fail.
The readiness artifact retains the full identifier-free evidence used for its
projection, not only summary counts and digests. Verification reruns the frozen
projection and requires canonical equality before A+, publication, or impact
consumers may import it. An attacker cannot make a changed reviewer roster,
correction, investigation version, or measurement pass merely by recomputing the
artifact's outer SHA-256.