Comprehensive Technical Features & Architectural Design Plan
Project Overview: A full-stack, microservice-driven Linked Open Data (LOD) ecosystem designed to bridge the gap between rigorous digital humanities research, museum data infrastructure, and undergraduate art history pedagogy. Built strictly on CIDOC-CRM and Linked Art JSON-LD standards, this platform solves industry-wide challenges regarding legacy data mapping, uncertainty modeling, and semantic web accessibility.
---
⚙️ 1. Core Architecture & Technology Stack
The system utilizes a decoupled, API-first architecture combining microservices for heavy data processing with micro-frontends for modular user interfaces.
- Backend / Data Layer (The Engine):
- Primary Language: Python (FastAPI) & Node.js.
- Graph Database: Neo4j or Stardog (native RDF/Graph storage) + PostgreSQL (JSONB) for application state.
- LOD Libraries: Crom (Python) for Linked Art instantiation, PyLD for JSON-LD framing/compaction.
- ETL Orchestration: Apache Airflow or Dagster for tracking data lineage and provenance.
- Frontend / UI Layer (The Interface):
- Framework: Next.js 15+ (React) with TypeScript (SSR/SSG for SEO and performance).
- Styling: Tailwind CSS / Chakra UI.
- Visualization: D3.js, Visx, and Cytoscape.js (for force-directed graphs).
- Mapping: ReactFlow (for visual ETL mapping) and Leaflet/Mapbox (for geospatial).
- Infrastructure & CI/CD:
- Docker containerization for isolated microservices.
- GitHub Actions with automated SHACL validation scripts to ensure data integrity.
---
🧩 2. The 4 Core Modular Tools (Micro-Frontends)
To solve the "skills gap" in the GLAM sector, the application provides a suite of low-code/no-code tools for curators and researchers.
🛠️ Tool 1: Visual ETL Mapper (CSV-to-Linked-Art)
- Purpose: Allows non-technical staff to map messy legacy CSVs to complex CIDOC-CRM event graphs.
- Features: Node-link drag-and-drop canvas (ReactFlow). Users drag a "Donor Name" column and connect it to the transferred_title_from node in an Acquisition event.
- Reconciliation: Built-in API calls to Getty AAT, ULAN, and Wikidata to reconcile string data into URIs before export.
📝 Tool 2: Linked Art Form Builder (Middle-Tier Editor)
- Purpose: A "Goldilocks" data entry form that looks simple but outputs strict JSON-LD.
- Features: Vocabulary-driven input (auto-fetches Getty URIs). Uses pattern-based generation (e.g., user types "Artist", backend builds the produced_by → carried_out_by → Person graph).
- Validation: Real-time domain-specific feedback (e.g., warns if an object's creation date predates the artist's birth).
🕸️ Tool 3: Dynamic Graph Explorer (Interactive Debugger)
- Purpose: A progressive-disclosure visualization tool for debugging dense Linked Art graphs.
- Features: Force-directed layouts (Cytoscape.js). Drill-down expandability to prevent node-clutter. Contextual JSON inspector side-panel for developers.
🧱 Tool 4: Reusable UI Component Library
- Purpose: Abstracting repetitive JSON-LD traversal logic into plug-and-play React components.
- Components: <LinkedArtAttribution /> (parses artist/technique), <IIIFCanvasViewer /> (auto-mounts OpenSeadragon via digitally_shown_by), <ProvenanceStepper /> (renders custody timelines).
---
🧠 3. Data Modeling & Semantic Patterns
The application strictly adheres to Linked Art patterns while addressing edge cases like uncertainty and contested histories.
- Activity/Event Pattern: Direct relationships are avoided. Artworks are connected to Artists via a "Creation Activity," allowing the attachment of Time, Place, and Technique to the event itself.
- Uncertainty & Contested Attributions: Robust modeling for "probably by X, possibly by Y." Utilizes the AttributeAssignment class to expand upon previous or contested attributions without breaking the primary graph.
- TimeSpan Pattern: Implements begin_of_the_begin and end_of_the_end ISO 8601 timestamps alongside human-readable strings to handle approximate historical dates.
- Statement Pattern: Uses LinguisticObject to capture nuanced curatorial notes and complex provenance narratives that cannot be fully structured into triples.
---
📊 4. Advanced Visualization & Discovery Suite
Transforming complex Linked Art datasets into readable, interactive visual formats for analysis and presentation.
- Interactive Provenance Timelines: Visualizing ownership chains chronologically. Includes visual indicators (shaded ranges/gradient bars) for timeline uncertainty.
- Geospatial Time-Sliders: Maps tracking object movement across cities, exhibitions, and collections, animated over time.
- Faceted Browsing: Search powered by Getty AAT classifications (medium, time period, style) with auto-suggest powered by alternate names and identifiers.
- Complexity Toggling (Art Tracks Model): UI gracefully handles complexity. Shows basic info by default, with a "See full scholarly details" toggle for researchers.
---
🎓 5. Pedagogy, Storytelling & Accessibility
Engineered to lower the barrier to entry for undergraduate students and the public.
- Narrative Storytelling: Integration with frameworks like StoryMapJS to build multimedia narratives from Linked Art data.
- Pedagogical Scaffolding: Pre-built modules (e.g., "The Provenance of a Renaissance Painting") to introduce students to linked data concepts.
- Export & Citation: One-click exports to CSV (for Gephi network analysis), JSON-LD, and Zotero integration for seamless citation management.
- Multilingual & Inclusive Access: Leverages the _label property and multiple Name objects for screen-reader compatibility and multilingual support.
---
🚀 6. Phased Development Roadmap
Phase 0: Foundation & Standards (Weeks 1-2)
- Define strict Linked Art JSON-LD patterns (Creation, Acquisition, Exhibition).
- Set up Docker infrastructure, Neo4j graph database, and API Gateway.
- Establish GitHub Actions for automated SHACL validation.
Phase 1: Core Data Engines & APIs (Weeks 3-5)
- Build the Validation & Production Microservice (Python/FastAPI + Crom).
- Build the Authority Reconciliation Microservice (Getty/Wikidata API wrappers).
- Develop the core ETL Transformation scripts for legacy data ingestion.
Phase 2: Micro-Frontends & Tooling (Weeks 6-9)
- Develop the Shared UI Component Library (<LinkedDate />, <IIIFViewer />).
- Build the Visual ETL Mapper (ReactFlow drag-and-drop interface).
- Build the Linked Art Form Builder and Dynamic Graph Explorer.
Phase 3: Assembly, UI & Pedagogy (Weeks 10-12)
- Develop the Next.js Dashboard Shell (Single Page Application).
- Integrate the micro-frontends into the shell via Module Federation.
- Build the public-facing discovery UI (Search, Timelines, Maps).
- Implement Zotero exports, StoryMap integrations, and "Student Mode" toggles.