← Documentation home

Canonical Markdown source · Oct 20, 2018

Local OpenTelemetry Wiring (Tempo / Jaeger)

ops/otel-local.md · 115 lines · SHA-256 1ebbc3b33f92

This runbook standardizes local OTLP wiring for:

  • Next.js runtime (`instrumentation.ts`, `@vercel/otel`)
  • Python validation service
  • Python reconciliation service
  • Dagster pipeline runner

All services honor `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` (or `OTEL_EXPORTER_OTLP_ENDPOINT`).

Env templates

Use one of the root templates:

  • `.env.otlp.tempo.example`
  • `.env.otlp.jaeger.example`

Copy the selected template values into `.env.local` and export the same values

in any shell that starts Python services/pipeline processes.

Quick wiring (PowerShell)

Tempo (or OTLP collector in front of Tempo):


Get-Content .env.otlp.tempo.example | Where-Object { $_ -and -not $_.StartsWith("#") }

Jaeger all-in-one (OTLP HTTP receiver enabled):


Get-Content .env.otlp.jaeger.example | Where-Object { $_ -and -not $_.StartsWith("#") }

If you want the current shell to use template variables directly:


Get-Content .env.otlp.tempo.example `
  | Where-Object { $_ -and -not $_.StartsWith("#") } `
  | ForEach-Object {
      $pair = $_ -split "=", 2
      [Environment]::SetEnvironmentVariable($pair[0], $pair[1], "Process")
    }

Then start services as usual (`pnpm dev`, `pnpm validate:service`, `pnpm reconcile:service`, `python pipeline/run_materialize.py`).

Local endpoints

  • OTLP HTTP: `http://127.0.0.1:4318/v1/traces`
  • Jaeger UI (typical all-in-one): `http://127.0.0.1:16686`
  • Grafana UI (if running local LGTM/Tempo stack): implementation-specific (commonly `http://127.0.0.1:3000`)

Explicit GraphDB / Solr span attributes

Shared DB span helper:

  • `src/utils/otel-db-spans.ts`

Current finalized call sites:

  • GraphDB SPARQL runtime calls are wrapped in:
  • `src/services/ai-query.ts` (`graphdb.sparql.query`)
  • Solr select client calls are wrapped in:
  • `src/services/solr-client.ts` (`solr.select.query`)

Emitted attributes include:

  • `db.system.name` (`graphdb` or `solr`)
  • `db.operation.name`
  • `db.namespace`
  • `server.address`
  • `server.port`
  • `url.full`
  • `metamuseum.db.backend`

This keeps distributed traces queryable by backend and operation when C4/C5 search/graph traffic scales up.

Credibility engine span/metric conventions

Shared conventions are centralized in:

  • `src/utils/otel-credibility.ts`

Span names:

  • `metamuseum.trust.citation_coverage.evaluate`
  • `metamuseum.trust.wiki_publish.preflight`
  • `metamuseum.trust.wiki_publish.execute`
  • `metamuseum.originality.score.evaluate`
  • `metamuseum.distribution.publish_queue.enqueue`
  • `metamuseum.distribution.publish_queue.process`
  • `metamuseum.distribution.publish_queue.dispatch`

Metric names:

  • `metamuseum.credibility.events.total`
  • `metamuseum.credibility.event.duration.ms`
  • `metamuseum.credibility.failures.total`

Required shared attributes:

  • `metamuseum.credibility.layer` (`trust|originality|distribution`)
  • `metamuseum.credibility.event`
  • `metamuseum.credibility.kind` (`gate|publish|queue|delivery|evaluation`)
  • `metamuseum.credibility.outcome`

Current call sites instrumented for these conventions:

  • trust gate events: `src/utils/citation-coverage.ts`
  • originality gate events: `src/utils/originality-score.ts`
  • wiki preflight/publish events: `src/services/wiki-publish.ts`
  • distribution queue events: `src/services/publish-queue-worker.ts`