← Documentation home

Canonical Markdown source · Oct 20, 2018

Solr 9 + GraphDB Provisioning

ops/search-graph-provisioning.md · 132 lines · SHA-256 fc1b15279a84

This runbook provisions the Era C1 search/graph stack:

  • Solr 9 (LUX-aligned search/faceting path)
  • GraphDB CE (Ontotext OSS path)

For local trace export wiring (Tempo / Jaeger), see `docs/ops/otel-local.md`.

Dev runtime (Docker Compose)

Compose stack file: `ops/docker-compose.yml`

  • Postgres stays in the default profile (`pnpm db:up`).
  • Solr + GraphDB are in the `sota` profile (`pnpm infra:sota:up`).

Commands:


pnpm infra:sota:up
pnpm infra:sota:down

Exposed ports:

  • Solr: `http://localhost:8983/solr`
  • GraphDB: `http://localhost:7200`

Images:

  • `solr:9.6`
  • `ontotext/graphdb:10.8.14` (GraphDB CE path via license-less default runtime)

Kubernetes runtime (Helm)

Chart path:

  • `ops/helm/metamuseum-search-graph`

Render manifests:


pnpm infra:sota:helm:template

Install:


helm upgrade --install metamuseum-search-graph ops/helm/metamuseum-search-graph --namespace metamuseum --create-namespace

Port-forward for local validation:


kubectl -n metamuseum port-forward svc/metamuseum-search-graph-solr 8983:8983
kubectl -n metamuseum port-forward svc/metamuseum-search-graph-graphdb 7200:7200

Persistence defaults

Default PVC requests in `values.yaml`:

  • Solr: `20Gi`
  • GraphDB: `50Gi`

Override storage class and size with Helm `--set` or a values override file.

GraphDB repository + SPARQL + Lucene bootstrap

After GraphDB is running, initialize the repository and Lucene connector:


pnpm graphdb:bootstrap

This script:

  • creates repository `metamuseum` when missing (GraphDB REST `POST /rest/repositories` with `.ttl` config),
  • enforces the runtime reasoning policy (`GRAPHDB_RULESET` must be `rdfsplus` or `rdfsplus-optimized`; OWL-family rulesets are rejected),
  • confirms SPARQL 1.1 query/update endpoints:
  • query: `/repositories/metamuseum`
  • update/Graph Store: `/repositories/metamuseum/statements`
  • creates Lucene connector `metamuseum_index` if missing via `luc:createConnector`,
  • configures a `graph()`-indexed field for hybrid text + graph filtering by named graph.

Reasoning profile policy:

  • Runtime inference: `RDFS` (`rdfsplus` or `rdfsplus-optimized`).
  • Constraint validation: `SHACL` via the validation service (`/api/validate`).
  • Disallowed at runtime: full OWL-family reasoning profiles.

Named-graph provenance partitioning by source institution

Load provider RDF payloads into source-specific named graphs:


pnpm graphdb:load:named-graph -- --provider=met --file=./artifacts/records-met.nq
pnpm graphdb:load:named-graph -- --provider=getty --file=./artifacts/records-getty.nq

By default, loads run in replace mode per named graph.

Use `--append` to append instead of replace.

Named graph pattern:

  • `https://lod.metamuseum.org/graph/source/{institution-slug}`

Examples:

  • Met: `https://lod.metamuseum.org/graph/source/metropolitan-museum-of-art`
  • Getty: `https://lod.metamuseum.org/graph/source/j-paul-getty-museum`

Hybrid text + graph query example (Lucene + named graph)

After bootstrap and data load, a query pattern can combine full-text retrieval with

graph partition filtering:


PREFIX luc: <http://www.ontotext.com/connectors/lucene#>
PREFIX luc-index: <http://www.ontotext.com/connectors/lucene/instance#>

SELECT ?entity ?score WHERE {
  ?search a luc-index:metamuseum_index ;
          luc:query "label:portrait AND labelGraph:https\\://lod.metamuseum.org/graph/source/metropolitan-museum-of-art" ;
          luc:entities ?entity .
  ?entity luc:score ?score .
}

The Lucene connector behavior and `graph()` chain semantics used here follow

GraphDB Lucene Connector documentation.