This runbook standardizes local OTLP wiring for:
- Next.js runtime (`instrumentation.ts`, `@vercel/otel`)
- Python validation service
- Python reconciliation service
- Dagster pipeline runner
All services honor `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` (or `OTEL_EXPORTER_OTLP_ENDPOINT`).
Env templates
Use one of the root templates:
- `.env.otlp.tempo.example`
- `.env.otlp.jaeger.example`
Copy the selected template values into `.env.local` and export the same values
in any shell that starts Python services/pipeline processes.
Quick wiring (PowerShell)
Tempo (or OTLP collector in front of Tempo):
Get-Content .env.otlp.tempo.example | Where-Object { $_ -and -not $_.StartsWith("#") }
Jaeger all-in-one (OTLP HTTP receiver enabled):
Get-Content .env.otlp.jaeger.example | Where-Object { $_ -and -not $_.StartsWith("#") }
If you want the current shell to use template variables directly:
Get-Content .env.otlp.tempo.example `
| Where-Object { $_ -and -not $_.StartsWith("#") } `
| ForEach-Object {
$pair = $_ -split "=", 2
[Environment]::SetEnvironmentVariable($pair[0], $pair[1], "Process")
}
Then start services as usual (`pnpm dev`, `pnpm validate:service`, `pnpm reconcile:service`, `python pipeline/run_materialize.py`).
Local endpoints
- OTLP HTTP: `http://127.0.0.1:4318/v1/traces`
- Jaeger UI (typical all-in-one): `http://127.0.0.1:16686`
- Grafana UI (if running local LGTM/Tempo stack): implementation-specific (commonly `http://127.0.0.1:3000`)
Explicit GraphDB / Solr span attributes
Shared DB span helper:
- `src/utils/otel-db-spans.ts`
Current finalized call sites:
- GraphDB SPARQL runtime calls are wrapped in:
- `src/services/ai-query.ts` (`graphdb.sparql.query`)
- Solr select client calls are wrapped in:
- `src/services/solr-client.ts` (`solr.select.query`)
Emitted attributes include:
- `db.system.name` (`graphdb` or `solr`)
- `db.operation.name`
- `db.namespace`
- `server.address`
- `server.port`
- `url.full`
- `metamuseum.db.backend`
This keeps distributed traces queryable by backend and operation when C4/C5 search/graph traffic scales up.
Credibility engine span/metric conventions
Shared conventions are centralized in:
- `src/utils/otel-credibility.ts`
Span names:
- `metamuseum.trust.citation_coverage.evaluate`
- `metamuseum.trust.wiki_publish.preflight`
- `metamuseum.trust.wiki_publish.execute`
- `metamuseum.originality.score.evaluate`
- `metamuseum.distribution.publish_queue.enqueue`
- `metamuseum.distribution.publish_queue.process`
- `metamuseum.distribution.publish_queue.dispatch`
Metric names:
- `metamuseum.credibility.events.total`
- `metamuseum.credibility.event.duration.ms`
- `metamuseum.credibility.failures.total`
Required shared attributes:
- `metamuseum.credibility.layer` (`trust|originality|distribution`)
- `metamuseum.credibility.event`
- `metamuseum.credibility.kind` (`gate|publish|queue|delivery|evaluation`)
- `metamuseum.credibility.outcome`
Current call sites instrumented for these conventions:
- trust gate events: `src/utils/citation-coverage.ts`
- originality gate events: `src/utils/originality-score.ts`
- wiki preflight/publish events: `src/services/wiki-publish.ts`
- distribution queue events: `src/services/publish-queue-worker.ts`