Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

PurRDF is an RDF 1.2 toolkit: primitives, codecs, SPARQL, SHACL, ShEx, entailment, and graph transport, implemented once in Rust and carried verbatim into Python, WebAssembly/JavaScript, and C. It is developed by Blackcat Informatics® Inc. and published under MIT OR Apache-2.0.

One RDF engine. One behavior. Every language.

Why does PurRDF exist?

RDF tooling fragments along two axes.

Across languages: every ecosystem has its own parser, with its own bugs, its own corner-case interpretations, and its own subset of the spec. Move a graph from a Rust service to a Python pipeline to a browser and you have silently changed what the data means three times.

Across time: RDF 1.2 — triple terms, reifiers, base-direction literals — is where the standard is going, and almost no incumbent library carries it.

PurRDF exists so that a graph is the same graph everywhere. It is a from-scratch, dependency-light Rust core — parser to SPARQL engine to SHACL validator to binary transport — exposed through native bindings rather than reimplemented per language.

What’s inside

  • RDF 1.2 primitives — an immutable, value-interned dataset IR (TermId space, string arena, copy-on-write mutation), with triple terms in object position, reifier/annotation side-tables, and base-direction literals. See The Interned Dataset IR.
  • Native codecs — first-party parsers/serializers for Turtle, TriG, N-Triples, N-Quads, RDF/XML, JSON-LD (star), and YAML-LD, with byte-deterministic output. See Codecs & Determinism.
  • Canonicalization — W3C RDFC-1.0 plus dataset diff and isomorphism. See Canonicalization & Diff.
  • SPARQL 1.1/1.2 — native parser → algebra → multiset evaluator, gated by the W3C conformance suites. See SPARQL.
  • SHACL and ShEx — native validators for both shape languages. See Validation.
  • Entailment — Simple/RDF/RDFS/OWL-RL materialization, an OWL-Direct tableau, and RIF-Core rules. See Entailment.
  • GTS graph transport — a single-file, content-addressed, append-only container for RDF 1.2 graphs and binary payloads. See GTS Graph Transport.
  • Slices, mappings, and provenance — a slice catalog, an explicit RDF↔GTS loss ledger, SSSOM, and FnO. See Slices, Mappings & Provenance.

Two design rules worth knowing on day one

No feature flags — ever. There are deliberately no Cargo feature flags anywhere in the workspace, and CI enforces this. A data carrier must not have optional behavior: optionality changes semantics per consumer, so every consumer gets the same byte-identical semantics instead.

PurRDF is a toolkit, not an ontology — it mints no vocabulary IRIs. Every vocabulary the library reads or writes is caller-supplied configuration with no fabricated default. A feature exercised without its vocabulary hard-errors or stays inactive; it never invents an IRI for you. (Test fixtures use example.org.)

The full invariant list is in Design Rules & Invariants.

Why RDF 1.2?

RDF 1.2 (and SPARQL 1.2) add first-class statement-level metadata to the data model: triple terms that can appear in object position, reifiers that name occurrences of a triple, and base-direction literals (rdf:dirLangString) for bidirectional text. PurRDF treats these as core data model, not an extension: they flow through the IR, the codecs, SPARQL, SHACL (a scoped SHACL 1.2 draft feature), the RDF/JS surface, and the GTS transport. See RDF 1.2 Features.

Where PurRDF sits

PurRDF is the library layer of a small family of linked-data projects: it is the data backbone of the GMEOW stack and the reference home of the Rust GTS engine — but it assumes nothing about your ontology or application.

How to read this book

API reference documentation lives on docs.rs/purrdf; the repository is github.com/Blackcat-Informatics/purrdf.

Getting Started: Rust

The single dependency a Rust downstream needs is the umbrella purrdf crate. It re-exports the RDF 1.2 implementation surface at its root and carries every other published crate under a stable module (purrdf::sparql, purrdf::shapes, purrdf::shex, purrdf::gts, purrdf::entail, purrdf::validate, purrdf::slice, purrdf::iri, purrdf::xsd, purrdf::events) — anything a consumer legitimately imports is reachable from purrdf alone, never by reaching into a sub-crate.

cargo add purrdf

The MSRV is Rust 1.96 (stable toolchain only; the workspace is nightly-free by policy).

Build, freeze, serialize, parse

use purrdf::{parse_dataset, serialize_dataset, RdfDatasetBuilder, RdfLiteral, SerializeGraph};

// Build a dataset in interned TermId space.
let mut b = RdfDatasetBuilder::new();
let alice = b.intern_iri("https://example.org/alice");
let knows = b.intern_iri("http://xmlns.com/foaf/0.1/knows");
let bob = b.intern_iri("https://example.org/bob");
let name = b.intern_iri("http://xmlns.com/foaf/0.1/name");
let hi = b.intern_literal(RdfLiteral::simple("Alice"));
b.push_quad(alice, knows, bob, None);
b.push_quad(alice, name, hi, None);
let ds = b.freeze().expect("freeze");

// Serialize to any native codec and parse back, losslessly.
let ttl = serialize_dataset(&ds, "text/turtle", SerializeGraph::Dataset).unwrap();
let back = parse_dataset(&ttl, "text/turtle", None).unwrap();
assert_eq!(back.quad_count(), 2);

The builder→freeze split is the heart of the API: you intern terms and push quads on a mutable RdfDatasetBuilder, then freeze() into an immutable, indexed RdfDataset that every engine (SPARQL, SHACL, ShEx, entailment) evaluates over. See The Interned Dataset IR.

Parsing text directly

let turtle = r#"
    @prefix ex: <https://example.org/> .
    ex:cat ex:says "meow" .
"#;
let dataset = purrdf::parse_dataset(turtle.as_bytes(), "text/turtle", None)
    .expect("valid Turtle");
assert_eq!(dataset.quad_count(), 1);

Malformed input is a typed RdfDiagnostic with a source location where the codec can provide one — never a silent partial parse.

Reaching the other engines

Every engine hangs off the same facade. For example, the zero-dependency IRI leaf and the ShEx schema layer:

let iri = purrdf::iri::parse("https://example.org/cat").expect("valid IRI");
assert_eq!(iri.as_str(), "https://example.org/cat");

let schema = purrdf::shex::parse_shexc(
    "PREFIX ex: <https://example.org/>\nex:Cat { ex:says . }",
    None,
).expect("valid ShExC");

When to depend on a sub-crate instead

Most applications should stop at purrdf. The sub-crates (purrdf-core, purrdf-rdf, purrdf-columnar, purrdf-sparql-eval, purrdf-shapes, purrdf-shex, purrdf-gts, purrdf-entail, purrdf-validate, purrdf-slice, purrdf-iri, purrdf-xsd, purrdf-events) exist for consumers that want exactly one engine — for example, a tool that only needs IRI parsing can depend on the zero-dependency purrdf-iri alone. The crate map is in the repository README.

Every release crate builds cleanly for wasm32-unknown-unknown, so the same Rust code paths work in native and wasm hosts.

Next steps

Getting Started: Python

The Python package wraps the same native Rust engine — not a reimplementation — so parsing, serialization, SPARQL, and validation behave identically to the Rust, JavaScript, and C surfaces.

pip install purrdf

Parsing

import purrdf

quads = purrdf.parse(
    '<https://example.org/alice> <http://xmlns.com/foaf/0.1/name> "Alice" .',
    purrdf.RdfFormat.TURTLE,
)

Validation: SHACL and ShEx

The native validation engines are exposed from the purrdf_native extension module:

from purrdf_native import shacl, shex

report = shacl.validate(shapes_ttl=my_shapes, data_nt=my_data)
print(report["conforms"])

result = shex.validate(my_schema_shexc, my_data_ttl,
                       [("https://example.org/alice", "https://example.org/PersonShape")])
print(result["conforms"])

SHACL result dicts keep the stable keys focus, path, value, severity, component, source_shape, and message. See SHACL and ShEx for what the engines cover.

rdflib compatibility

The package ships an rdflib compatibility layer:

from purrdf.compat.rdflib import Graph

For a literal, zero-change import rdflib, there is an opt-in extra:

pip install purrdf[rdflib]

This pulls in the separate purrdf-rdflib distribution, whose top-level rdflib package re-exports the compat surface, so existing third-party code doing import rdflib / from rdflib.namespace import RDF transparently runs on purrdf. Caveat: that shadow claims the rdflib import name and must never be installed alongside the genuine rdflib — the two cannot co-inhabit one environment. It is a separate distribution (never bundled into the main purrdf wheel) precisely so environments that need the real rdflib simply omit it.

The compat layer is gated in CI against rdflib 7.6’s own vendored test suite plus a first-party differential parity suite — see rdflib Compatibility for details and the known, ledgered divergences.

GTS relational exports

The Python package also ships GTS relational exports for analytics pipelines:

from purrdf import gts_to_sqlite, gts_to_duckdb, gts_to_parquet

These project a GTS container into SQLite, DuckDB, or Parquet tables.

Next steps

Getting Started: JavaScript / WebAssembly

The npm package @blackcatinformatics/purrdf is the same Rust engine compiled to wasm32 and surfaced through an RDF/JS-shaped API (DataFactory, DatasetCore, Stream/Sink). It runs in the browser and in Node, entirely in memory.

npm install @blackcatinformatics/purrdf

Prefer to try it before installing anything? The RDF-1.2 playground runs this exact wasm build in your browser — parse, SPARQL, SHACL, serialize, and canonicalize/compare RDF-1.2 graphs client-side, with no toolchain and no server.

First dataset

Await ready() once before anything else — it performs the one-time async wasm instantiation:

import { ready, DataFactory, Dataset, QueryEngine } from "@blackcatinformatics/purrdf";

await ready(); // one-time async wasm instantiation

const f = new DataFactory();
const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");

const ds = new Dataset();
ds.add(f.quad(f.namedNode("https://ex/s"), f.namedNode("https://ex/says"), rtl));

const nq = ds.serialize("nquads");           // directions survive the round-trip
const reparsed = Dataset.parse(nq, "nquads");

const engine = new QueryEngine();
const ask = engine.ask(reparsed, "ASK { <https://ex/s> <https://ex/says> ?msg }");

The RDF 1.2 wedge

No incumbent RDF/JS library carries RDF 1.2 quoted-triple terms or directional literals. PurRDF’s DataFactory exposes both:

// A quoted triple, usable as a subject/object (RDF-star / RDF 1.2).
const quoted = f.quotedTriple(
  f.namedNode("https://ex/alice"),
  f.namedNode("https://ex/knows"),
  f.namedNode("https://ex/bob"),
);

// A base-direction literal (rdf:dirLangString).
const hello = f.directionalLiteral("مرحبا", "ar", "rtl");

API surface

  • ready(bytesOrUrl?) — await once before anything else.
  • DataFactorynamedNode, blankNode, literal(value, languageOrDatatype?), typedLiteral, directionalLiteral, variable, defaultGraph, quad, quotedTriple, fromTerm, fromQuad.
  • Dataset (RDF/JS DatasetCore) — Dataset.parse(input, format, base?), serialize(format), add/delete/has/match/quads/size, and iteration (for (const quad of dataset)). Formats: turtle, ntriples, nquads, trig, rdfxml (or their media types); serialize additionally accepts jsonld.
  • Graph identityDataset.canonicalize() returns the RDFC-1.0 canonical, flat N-Quads for the graph; Dataset.isomorphic(other) decides RDF graph equality under blank-node relabeling (an exact oracle backed by full RDFC-1.0 canonicalization).
  • SPARQLQueryEngine keeps the native plan cache alive across calls and exposes typed select / ask / construct / describe, atomic update, and queryRaw serialization. Dataset.query(...) remains the compatibility raw-string helper.
  • SHACLshaclValidateToSarif(shapesTtl, dataNt) validates an N-Triples data graph against a Turtle shapes graph and returns a SARIF 2.1.0 report; shaclEntail(shapesTtl, dataNt) materializes the SHACL-AF sh:rule inferences as N-Triples.
  • Sink — a streaming consumer (push(quad) / finish() → Dataset); datasetToStream / streamToDataset are the async RDF/JS Stream/Sink helpers.

More on the RDF/JS mapping in RDF/JS in JavaScript.

Scope and current limitations

  • In-memory only. SPARQL queries run over the in-memory dataset; this package provides no network resolver, so remote SERVICE and LOAD fail explicitly.
  • A quoted-triple term as a quad object currently round-trips only through N-Quads (a current native serializer limitation for the other formats).

Building from source

The Rust cdylib lives in crates/rdf-wasm; the published ESM package is generated from it:

make wasm-pkg        # release wasm + wasm-bindgen ESM bindings → js/pkg/
make wasm-pkg-test   # the above + TypeScript, Node, and packed-tarball gates

This requires the wasm32-unknown-unknown Rust target and a wasm-bindgen-cli pinned to the crate’s wasm-bindgen version.

Getting Started: C

libpurrdf is a stable, SemVer-disciplined extern "C" surface over the native PurRDF stack: parse, serialize, pattern iteration, copy-on-write mutation, SPARQL, SHACL validation/entailment, and GTS container round-trips. The committed, reproducible header include/purrdf.h is the ABI contract — CI fails if it drifts from the crate.

It is one shared library: libpurrdf statically reuses the purrdf-gts Rust crate, so a language shim links libpurrdf alone and still reads/writes .gts containers.

Building

The library, header, and pkg-config file are produced by cargo-c:

make capi-build                 # cargo capi build: libpurrdf.{so,a} + purrdf.h + purrdf.pc
make capi-install PREFIX=/usr   # cargo capi install into a prefix
make capi-check                 # verify the committed header is current + run the C smoke

A first program

Adapted from the repository’s C smoke test (crates/rdf-capi/tests/smoke.c):

#include "purrdf.h"
#include <stdio.h>
#include <string.h>

int main(void) {
    const char *doc = "<http://a> <http://b> <http://c> .";
    PurrdfDataset *dataset = NULL;
    PurrdfError *error = NULL;

    int rc = purrdf_parse((const uint8_t *)doc, strlen(doc), "text/turtle",
                          NULL, NULL, &dataset, &error);
    if (rc != PURRDF_STATUS_OK) return 1;

    size_t quad_count = 0;
    purrdf_dataset_quad_count(dataset, &quad_count);
    printf("%zu quad(s)\n", quad_count);

    /* Iterate every quad through a pattern cursor. */
    PurrdfGraphMatch any;
    memset(&any, 0, sizeof(any));
    any.kind = PURRDF_GRAPH_MATCH_KIND_ANY;
    PurrdfCursor *cursor = NULL;
    purrdf_quads_for_pattern(dataset, NULL, NULL, NULL, &any, &cursor, &error);

    PurrdfTermView s, p, o, g;
    uint8_t has_graph = 0;
    while (purrdf_cursor_next(cursor, &s, &p, &o, &g, &has_graph) == PURRDF_STATUS_OK) {
        printf("subject=%.*s\n", (int)s.lexical.len, (const char *)s.lexical.ptr);
    }
    purrdf_cursor_free(cursor);
    purrdf_dataset_free(dataset);
    return 0;
}

The ABI contract

  • No unwinding across the boundary. Every function runs inside catch_unwind; a caught panic becomes PURRDF_STATUS_PANIC, never a process abort across FFI.
  • int32_t status + out-params. Fallible functions return a PurrdfStatus value and write results through out-pointers. PURRDF_STATUS_CURSOR_EXHAUSTED is the (non-error) end-of-rows signal.
  • SemVer-frozen ABI. The status enum is append-only; new fields and functions are additive. purrdf_abi_version reports the current ABI version (0.1.x, beta).

Ownership and lifetimes

  • Every handle/buffer/error/cursor has exactly one matching *_free (purrdf_dataset_free, purrdf_graph_free, purrdf_cursor_free, purrdf_rowcursor_free, purrdf_buffer_free, purrdf_error_free). Freeing NULL is a no-op.
  • The C side never free()s a PurrdfStr.ptr — it borrows library-owned memory; copy the bytes out if they must outlive the borrow. Term views from purrdf_cursor_next are valid until the next purrdf_cursor_next on that cursor or purrdf_cursor_free.
  • Pattern cursors pin the dataset and pull rows lazily from the selected core index; opening a cursor does not allocate a matching-row snapshot.
  • PurrdfDataset is frozen and Send + Sync — readable concurrently from many threads. PurrdfGraph (the copy-on-write mutable delta) and cursors are single-threaded.

Known limitation

The GTS star layer round-trip (purrdf_to_gtspurrdf_from_gts of a dataset containing quoted triples / reifier bindings) currently fails with PURRDF_STATUS_GTS_ERROR. This is a pre-existing gap in the kernel path, not in the C ABI; star-free GTS round-trips are lossless, and a characterization test pins the current behavior so a kernel fix will flip it.

Full contract details — status codes, term crossing representations, thread-safety per handle — are in the purrdf-capi README.

The Interned Dataset IR

Everything in PurRDF evaluates over one intermediate representation: an immutable, value-interned RDF 1.2 dataset owned by the ring-fenced purrdf-core kernel.

Terms are interned once

Every term — IRI, blank node, literal, triple term — is stored once in a string arena and addressed by a copyable TermId (a niche-optimized NonZeroU32). Quads are rows of four TermIds. That makes term equality a single integer compare, keeps quads at a fixed small size, and means a term that appears in a million quads costs its bytes exactly once.

Hot maps use fixed-key ahash — deterministic hashing is part of the byte-determinism discipline, not just a speed choice.

Builder → freeze

The IR has a strict two-phase life cycle:

use purrdf_core::{RdfDatasetBuilder, RdfLiteral};

// Intern terms once; quads are rows of copyable TermIds.
let mut b = RdfDatasetBuilder::new();
let cat = b.intern_iri("https://example.org/cat");
let says = b.intern_iri("https://example.org/says");
let meow = b.intern_literal(RdfLiteral::simple("meow"));
b.push_quad(cat, says, meow, None);

// Freeze into the immutable, indexed dataset the engines evaluate over.
let ds = b.freeze().expect("well-formed dataset");
assert_eq!(ds.quad_count(), 1);

RdfDatasetBuilder is the mutable ingestion phase: intern terms, push quads, attach reifiers and annotations. freeze() validates the structure and produces an immutable RdfDataset: quad rows in Box<[QuadRow]> tables with lazy ordinal permutation indexes (roughly 4 bytes per quad per axis). The frozen dataset is what SPARQL, SHACL, ShEx, and entailment all read, through the allocation-free DatasetView trait.

Freezing is also what makes concurrency simple: a frozen dataset is immutable, so it can be shared and read from many threads (the C ABI exposes exactly this as a Send + Sync handle).

Copy-on-write mutation

“Immutable” does not mean “static”. Mutation happens through a copy-on-write delta over a frozen base: edits accumulate in a lightweight overlay, and the result freezes into a new dataset without copying the untouched base rows or re-interning shared terms. SPARQL UPDATE and the C ABI’s mutable PurrdfGraph handle both ride this path.

What else lives in the kernel

Beyond the IR itself, purrdf-core owns:

  • DatasetView — the static read trait every engine evaluates over.
  • Structured diagnostics — typed RdfDiagnostics with source locations (deliberately SARIF-free; the SARIF boundary is purrdf-validate).
  • RDFC-1.0 canonicalization, dataset diff, and isomorphism — see Canonicalization & Diff.
  • Store and engine seams — the narrow parser-ingress, serializer-egress, and SparqlEngine traits that adapters implement in sibling crates.
  • Provenance and the loss ledger — a generic provenance sidecar and the machine-readable RDF↔GTS loss matrix, plus native FnO and SSSOM codecs (see Slices, Mappings & Provenance).

Text codecs are not in the kernel — parsing and serialization live one layer up in purrdf-rdf. The split keeps the kernel small and its invariants enforceable at the crate boundary: no oxigraph, no PyO3 (a hygiene gate asserts the dependency tree), wasm32-clean, and a file-IO-free IR layer.

Why this design

The layout is chosen by measurement, not assertion: the criterion bench crates/rdf-core/benches/ir_layout.rs compares array-of-structs, struct-of-arrays, and predicate-adjacency layouts on allocation counts, high-water memory, and end-to-end latency — the shipped layout is whichever wins. See Performance.

RDF 1.2 Features

PurRDF is RDF 1.2-first: the features RDF 1.2 Concepts adds over RDF 1.1 are part of the core data model, carried through the IR, the codecs, SPARQL, validation, the language bindings, and the GTS transport.

Triple terms

RDF 1.2 lets a triple itself be a term in object position — the “quoted triple” of RDF-star, written <<( s p o )>> in SPARQL 1.2 syntax. In the IR a triple term is interned like any other term and gets a TermId, so it composes with everything else (patterns, results, serialization).

  • The star-capable codecs (Turtle, TriG, N-Triples, N-Quads, JSON-LD star) round-trip triple terms; see Codecs & Determinism for what happens on a star-incapable projection.
  • SPARQL 1.2 quoted-triple syntax is parsed by purrdf-sparql-algebra and evaluated natively — the W3C SPARQL 1.2 triple-term surface passes in the conformance harness (Conformance & Testing).
  • In JavaScript, DataFactory.quotedTriple(...) produces the same term (RDF/JS in JavaScript).

Reifiers and annotations

RDF 1.2 replaces old-style reification with reifiers: terms that name an occurrence of a triple (rdf:reifies), so you can attach metadata to a statement without asserting anything odd. In PurRDF, reifier bindings and annotations live in dedicated side-tables on the dataset rather than being smeared into the quad table.

Reifier bindings and annotations survive every star-capable codec round-trip; projections into star-incapable formats drop them loudly, with the realized count handed to the loss ledger (see Slices, Mappings & Provenance). SHACL support for validating reified statements — the draft sh:reifierShape / sh:reificationRequired surface — is covered in SHACL.

Base-direction literals

RDF 1.2 adds rdf:dirLangString: a language-tagged literal that also carries a base direction (ltr or rtl) for correct bidirectional-text handling. These are first-class in the IR and in every binding:

const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");

Directions survive serialization round-trips through the star-capable codecs — the JavaScript quickstart demonstrates the N-Quads round-trip.

RDF 1.2 is a complete target, not a draft excuse

PurRDF treats the RDF 1.2 / SPARQL 1.2 specifications as a complete, implementable target. Where a feature is scoped (for example, the SHACL 1.2 reifier-shape support is a scoped Working Draft feature, not full SHACL 1.2 conformance), the scope is stated explicitly and gated by tests — never left as a silent partial implementation. The live per-feature status is the conformance matrix in docs/CONFORMANCE.md.

Where each feature shows up

FeatureIRCodecsSPARQLSHACLRDF/JSGTS
Triple terms (object position)interned termstar-capable formats<<( s p o )>>via paths/valuesquotedTriplemapped per spec
Reifiers / annotationsside-tablesstar-capable formatsreifier surfacesh:reifierShape (draft)rdf:reifies mapping
Base-direction literalsliteral kindround-tripsmatched/producedvalue nodesdirectionalLiteralcarried

The GTS mapping of triple terms and rdf:reifies is formalized in the GTS specification, which pins its RDF 1.2 substrate to the 07 April 2026 W3C Candidate Recommendation Snapshot.

RDF 1.2 Visualization

PurRDF projects RDF into a renderer-neutral statement model before producing a layout or SVG. The projection keeps structural triple terms, assertion occurrences, reifier identity, annotations, graph context, nesting, and RDF dialect diagnostics distinct. The SVG carries that model as embedded JSON metadata, so it is both a visual document and a lossless machine-readable export.

The examples below are generated directly by the Rust visualization surface. Line colour varies subtly within each semantic relation class to make dense routes easier to follow; line width, dash pattern, labels, and arrow grammar continue to carry the RDF meaning without relying on colour.

Ordinary shared resources

The compact view preserves the familiar RDF resource graph while separating routes that share nodes.

Open compact SVG

Ordinary RDF graph with shared resources

Exact statement incidence view

Open exact SVG

Ordinary RDF exact statement incidence graph

Statement table

Open table SVG

Ordinary RDF statement table

Asserted, reified, and annotated statements

Solid arrows remain assertions. Addressable statement anchors connect those assertions to reifier resources, whose ordinary RDF properties carry provenance, confidence, timestamps, and directional language literals.

Open compact SVG

Asserted RDF statements with reifiers and annotations

Exact statement incidence view

Open exact SVG

Exact incidence graph for asserted and reified statements

Statement table

Open table SVG

Statement table for asserted and reified RDF

Quoted-only and nested triple terms

A quoted-only triple is a bounded statement glyph, never a solid assertion arrow. Reifier resources stay distinct from the structural statement they reify.

Open quoted-only compact SVG

Quoted-only RDF triple terms and their reifiers

Quoted-only exact view and table

Open exact SVG · Open table SVG

Quoted-only exact incidence graph

Quoted-only statement table

Nested triple terms retain recursive statement identity. Dialect badges make symmetric or generalized RDF positions explicit rather than silently rendering them as ordinary RDF 1.2.

Open nested compact SVG

Nested triple terms with RDF dialect diagnostics

Nested exact view and table

Open exact SVG · Open table SVG

Nested exact statement incidence graph

Nested statement table

Dense connected data

The exact view uses separated vertical channels and rounded orthogonal turns so individual subject, predicate, object, reification, and annotation routes remain traceable through a dense connected dataset.

Open dense exact SVG

Dense RDF exact statement incidence graph

Dense compact view and table

Open compact SVG · Open table SVG

Dense compact RDF resource graph

Dense RDF statement table

Regenerating the samples

The committed SVGs are projections of the Rust fixtures, not hand-edited book artwork:

make book-samples
make book

make check regenerates the same artifacts in a temporary directory and rejects any drift between the renderer and the book.

Codecs & Determinism

PurRDF ships first-party parsers and serializers — no wrapped third-party codec — for seven formats:

FormatMedia typeStar-capable
Turtletext/turtleyes
TriGapplication/trigyes
N-Triplesapplication/n-triplesyes
N-Quadsapplication/n-quadsyes
RDF/XMLapplication/rdf+xmlno
JSON-LD (star)application/ld+jsonyes
YAML-LDapplication/ld+yamlyes

They live in purrdf-rdf, one layer above the kernel, and are reachable through the umbrella crate:

use purrdf::{parse_dataset, serialize_dataset, SerializeGraph};

let turtle = br#"
    @prefix ex: <https://example.org/> .
    ex:cat ex:says "meow" .
"#;

// Parse into the frozen, value-interned RDF 1.2 dataset IR.
let ds = parse_dataset(turtle, "text/turtle", None).expect("valid Turtle");
assert_eq!(ds.quad_count(), 1);

// Serialize back out through any native codec — byte-deterministic output.
let nq = serialize_dataset(&ds, "application/n-quads", SerializeGraph::Dataset)
    .expect("serializes");

Open Knowledge Format bundles

The native OKF codec maps caller-profiled RDF 1.2 datasets to agent-facing Markdown files with YAML frontmatter and lifts them back through the RDF event seam. OKF is an in-memory bundle API rather than another media type: callers choose how to store the files, so the same code remains deterministic and wasm-clean.

OkfConfig::new requires the vocabulary namespace, document base IRI, and recognized frontmatter keys. There is no built-in ontology or namespace. Use lift_okf_bundle to drive an RdfEventSink, or write_okf_bundle (backed by OkfWriter, an RdfDatasetVisitor) to project a frozen dataset. Both directions always return a loss ledger. A lossless profile yields an empty ledger; named graphs, non-profile/OWL rows, and unrelated reifier or annotation rows are pinpointed explicitly when writing.

Byte determinism

Every serializer is byte-deterministic: the same dataset always produces the same bytes, on every platform and in every language binding. This is a hard workspace invariant, not a best effort — no iteration-order, time, or RNG dependence is allowed in any output path (hashers are fixed-key ahash for exactly this reason), and golden-file tests pin the emitted bytes.

Determinism is what makes the rest of the toolkit composable: content addressing in GTS and the slice catalog, diffable serializations in review, and cross-language conformance vectors that can be compared byte-for-byte.

Diagnostics, not partial parses

Malformed input is a typed RdfDiagnostic with a source location where the codec can provide one — never a silent partial parse. Parsing can optionally record a source-position span table for richer diagnostics. Diagnostics stay structured (SARIF-free) in the core; render them as byte-deterministic SARIF 2.1.0 for editors and CI with purrdf-validate (see SHACL).

Lossy projections are loud

RDF 1.2 statement-level data (triple terms, reifier bindings, annotations) survives every star-capable round-trip. Serializing into a star-incapable projection drops that layer loudly: the realized drop count is handed to the machine-readable loss ledger (generated/rdf-loss-matrix.json) rather than disappearing. The same discipline applies at the SPARQL results boundary (Result Formats) and the RDF↔GTS boundary.

The succinct pack codec

Alongside the text codecs above, purrdf-core ships a binary codec for a different job: a read-only, query-the-compressed-form encoding of a whole dataset for large-scale reference bundles, not an interchange format with a media type. PackBuilder::build_bytes(&dataset) writes a self-contained, byte-deterministic pack — a value dictionary, graph-partitioned succinct bitmap-triples, and RDF 1.2 side-tables (reifier bindings, statement annotations) — into one Vec<u8>. PackView::from_bytes(&[u8]) opens it zero-copy over a borrowed slice and answers pattern queries directly against the packed bytes, with no decompression or materialization step first.

Reach for a pack when a dataset is done changing and needs to be distributed, archived, or served at a scale where re-parsing text on every load is too slow: RDF 1.2 (named graphs, quoted triples, reifiers, annotations) is fully supported, and verify_pack independently recomputes the dataset’s RDFC-1.0 digest from the pack’s own decoded contents — a certified read-only projection, not merely a compressed file. The library never memory-maps a pack itself (every published crate stays wasm32-unknown-unknown-clean); a native consumer that wants a durable, larger-than-heap tier mmaps the file and hands PackView::from_bytes the resulting borrowed slice. See the “Pack backend” section of the backend contract for the full contract.

The columnar Parquet codec

purrdf::columnar exposes the bidirectional SQL/DataFrame interchange path. It maps any DatasetView plus a content-addressed blob store to five standard Parquet files (terms, quads, reifiers, annotations, and blobs) and reads that exact profile back without Arrow or a general Parquet runtime. The mapping retains RDF 1.2 triple terms, reifiers, annotations, graph scope, directional literals, blank-node scope, and explicitly empty named graphs.

The files are byte-deterministic and readable by engines such as DuckDB. See the normative columnar schema for every field and the deliberately narrow Parquet profile.

Conformance

The codecs are gated by the W3C rdf-tests syntax corpus, vendored and frozen in-repo — 250/250 round-trip cases across N-Quads, N-Triples, RDF/XML, TriG, and Turtle at the time of writing. The live scoreboard is docs/CONFORMANCE.md.

  • Canonicalization & Diff — when you need a canonical serialization rather than just a deterministic one.
  • The Interned Dataset IR — what the text codecs parse into, and the DatasetView read seam the pack codec implements alongside RdfDataset.

Canonicalization & Diff

Byte-deterministic serialization (Codecs & Determinism) means the same dataset always emits the same bytes. Canonicalization is the stronger property: two different in-memory datasets that are isomorphic — the same graph up to blank-node relabeling — canonicalize to the same bytes.

RDFC-1.0

PurRDF implements W3C RDF Dataset Canonicalization (RDFC-1.0) natively in the kernel, tested against the W3C rdf-canon fixture suite (65 vectors — 64 eval plus 1 negative — all green; see docs/CONFORMANCE.md).

The entry point is canonicalize (with a canonicalize_with variant for choosing the hash), producing canonical blank-node labels and, one layer up in purrdf-rdf, canonical flat N-Quads over the frozen IR:

use purrdf::canonicalize;

let canon = canonicalize(&ds);
// Canonical labels are stable across runs, hosts, and language bindings.

Use canonicalization when you need a content identity for a graph: hashing, signing, deduplication, or comparing datasets produced by different writers.

Isomorphism

datasets_isomorphic(a, b) decides whether two frozen datasets are RDF-structurally isomorphic: the same quads under a blank-node bijection. Canonicalization gives the equivalent verdict — two datasets are isomorphic iff their canonicalizations are equal — but the direct check is the convenient form for tests and harnesses. PurRDF’s own conformance harnesses use RDFC-1.0 isomorphism to compare, for example, SHACL Rules output graphs against expected inferred graphs.

Diff

dataset_diff(a, b) produces a structural diff between two frozen datasets, including an isomorphic verdict. For a human-facing review flow, purrdf-rdf additionally provides per-subject Symmetric-CBD extraction (“describe”) and a review-friendly Turtle normalizer, so a graph change reads like a code change.

Choosing the right tool

NeedUse
Same dataset → same bytesany native serializer (always true)
Same graph (up to blank nodes) → same bytesRDFC-1.0 canonicalize
“Are these two datasets the same graph?”datasets_isomorphic
“What changed between these datasets?”dataset_diff + describe/normalize
Content-addressed transport of a graphGTS (BLAKE3 content ids)

API details are on docs.rs/purrdf-core (the canonicalize, datasets_isomorphic, and dataset_diff items) and docs.rs/purrdf-rdf (describe and normalization).

SPARQL: Querying

PurRDF’s SPARQL stack is native and three-layered, gated by the W3C SPARQL 1.1 and 1.2 conformance suites:

  1. purrdf-sparql-algebra — parses query and update text into a PurRDF-owned, RDF 1.2-native query algebra (Query/GraphPattern, Update/GraphUpdateOperation). Parse and algebra only.
  2. purrdf-sparql-eval — the multiset evaluator over the frozen IR’s DatasetView, entirely in interned TermId space.
  3. purrdf-sparql-results — the results boundary (next chapter).

All three are re-exported under purrdf::sparql.

A first query

use purrdf::{RdfDatasetBuilder, RdfLiteral, SparqlEngine, SparqlRequest, SparqlResult};
use purrdf::sparql::NativeSparqlEngine;

// A tiny dataset in interned TermId space.
let mut b = RdfDatasetBuilder::new();
let cat = b.intern_iri("https://example.org/cat");
let says = b.intern_iri("https://example.org/says");
let meow = b.intern_literal(RdfLiteral::simple("meow"));
b.push_quad(cat, says, meow, None);
let ds = b.freeze().expect("freeze");

// Evaluate through the SparqlEngine seam; parsed plans are memoized.
let engine = NativeSparqlEngine::new();
let result = engine.query(&ds, SparqlRequest {
    query: "SELECT ?what WHERE { <https://example.org/cat> <https://example.org/says> ?what }",
    base_iri: None,
    substitutions: &[],
}).expect("evaluates");

if let SparqlResult::Solutions { rows, .. } = result {
    assert_eq!(rows.len(), 1);
}

The SparqlEngine trait itself lives in purrdf-core, so hosts can swap engines behind one seam; NativeSparqlEngine is the shipped implementation.

What the front-end covers

  • Query — all four query forms (SELECT/ASK/CONSTRUCT/DESCRIBE), basic graph patterns, OPTIONAL, UNION, MINUS, GRAPH, FILTER/BIND/VALUES, property paths, GROUP BY/aggregates, EXISTS/NOT EXISTS, solution modifiers, and RDF 1.2 quoted triple terms (<<( s p o )>>).
  • UpdateINSERT DATA/DELETE DATA, the DELETE/INSERT … WHERE family (WITH/USING, DELETE WHERE), LOAD, and CLEAR/DROP/CREATE/ADD/MOVE/COPY.

Anything outside this surface — and every malformed query — is a typed ParseError, never a silently degraded parse.

How the evaluator works

  • Multiset (bag) semantics — solutions are a bag, preserved until DISTINCT/REDUCED, per the SPARQL algebra.
  • Interned evaluation — constants resolve to a dataset TermId once; solution comparison is an integer compare; computed FILTER/BIND values that already exist in the dataset are promoted to the interned id at mint time.
  • Property paths in-engine — the full path algebra (* + ? / | ^ !()) evaluated over the same indexed surface, wasm-safe.
  • Cost-based BGP planning — join order is chosen by a cost model; NativeSparqlEngine::explain_query exposes the chosen order as an ordered list of triple-pattern strings so you can audit planner decisions without running the query.
  • EXISTS decorrelation — correlated EXISTS/NOT EXISTS filters are decorrelated rather than re-evaluated per row.
  • The SERVICE seamSERVICE federation is evaluated through a host-injectable transport: the engine itself performs no I/O, so federation stays wasm-portable and the host decides how (and whether) remote endpoints are reached. All seven W3C service federation cases pass through this seam.
  • Hard-fail — an out-of-scope algebra node or unimplemented builtin is a typed EvalError::Unsupported, never a partial or wrong answer.

Entailment regimes

SPARQL queries can be answered under an entailment regime by materializing the dataset first with purrdf-entailRegime::from_iri maps a sparql:entailmentRegime IRI to the matching engine.

Conformance

The full W3C SPARQL 1.1 query + update evaluation suites plus the SPARQL 1.2 suite are vendored verbatim and run by purrdf-sparql-conformance; every non-pass is a typed, ledgered expected-failure. See Conformance & Testing and docs/CONFORMANCE.md for the live matrix.

SPARQL: Result Formats

purrdf-sparql-results is the results boundary of the SPARQL stack: the canonical authority for turning a SparqlResult (SELECT solutions, ASK boolean, or CONSTRUCT graph) into the four W3C SPARQL Results formats — JSON (SRJ), XML, CSV, and TSV — plus an additive, provenance-carrying PurRDF extension where the format can carry one. JSON and XML documents can also be read back (from_json, from_xml).

use purrdf::sparql::{serialize, ResultProvenance, SparqlResultsFormat};

// `result` is the SparqlResult produced by purrdf-sparql-eval (or any engine
// implementing the purrdf-core SparqlEngine seam).
let outcome = serialize(&result, SparqlResultsFormat::Json, &ResultProvenance::default())
    .expect("SELECT serializes to SRJ");

assert!(!outcome.provenance_dropped);
let json = String::from_utf8(outcome.bytes).unwrap();

Per-format writers (to_json, to_xml, to_csv, to_tsv) and readers (from_json, from_json_boolean, from_xml, from_xml_boolean) are also exported directly.

Behavior worth knowing before you pick a format

  • Byte-deterministic output — the same result always serializes to the same bytes, like every other PurRDF output path (Codecs & Determinism).
  • The support matrix is enforced, not fudged — XML rejects CONSTRUCT graphs, and CSV/TSV reject both ASK booleans and CONSTRUCT graphs, each as a typed Error::Format, rather than emitting something spec-shaped but wrong.
  • Lossy projections are flagged — CSV/TSV have no extension point, so a populated provenance is trimmed at the exit gate and SerializeOutcome::provenance_dropped is set; the drop is never silent.
FormatSELECTASKCONSTRUCTProvenance extension
JSON (SRJ)yesyesyesyes
XMLyesyesrejectedyes
CSVyesrejectedrejecteddropped, flagged
TSVyesrejectedrejecteddropped, flagged

The provenance extension

The PurRDF extension is additive: a standard SPARQL results consumer can read the JSON/XML documents unchanged, while a PurRDF-aware consumer can recover per-result provenance carried alongside the bindings. Where the format has no extension point (CSV/TSV), the provenance is dropped loudly, per the loss discipline described in Slices, Mappings & Provenance.

One term-syntax authority

The crate depends only on purrdf-core and stays wasm-clean; term and N-Triples syntax come exclusively from the kernel’s emit primitives, so there is exactly one term-syntax authority in the workspace — results, codecs, and diagnostics can never disagree about how a term is written.

SHACL

purrdf-shapes (re-exported as purrdf::shapes) is PurRDF’s native SHACL validator: the complete SHACL Core feature set — all constraint components, full property paths, qualified value shapes, property pairs — plus SHACL-SPARQL constraints and targets and the SHACL-AF surface, running entirely on PurRDF’s own interned IR and native SPARQL engine (no oxigraph, no PyO3).

It validates an RDF 1.2 data graph against a SHACL shapes graph with no inference (parity with pySHACL inference="none"); combine with Entailment if you want to validate a materialized closure.

What it covers

  • SHACL Core — every constraint component, full property paths, qualified value shapes, property pairs. The W3C data-shapes suite passes clean (126/126, zero ledgered gaps at the time of writing — the live number is in docs/CONFORMANCE.md).
  • SHACL-SPARQL — SPARQL-based constraints and targets, custom constraint components with pre-binding semantics, user-defined sh:SPARQLFunction calls, and sh:SPARQLTargetType, evaluated on the native SPARQL engine.
  • SHACL-AF — node expressions (including sh:ExpressionConstraintComponent) and SHACL Rules (sh:TripleRule and sh:SPARQLRule, with sh:condition, sh:order, sh:deactivated): rules fire in an iterative fixpoint and the derivation is materialized as a new dataset (base ⊎ derived), leaving the input graph untouched. Some node-expression conveniences (sh:if, aggregations, ordering wrappers) are DASH/TopBraid conventions with no normative RDF definition; PurRDF documents its adopted reading and pins it with a frozen corpus — see the SHACL-AF section of docs/CONFORMANCE.md.

The SHACL 1.2 reifier-shape draft scope

The crate implements a scoped SHACL 1.2 Working Draft feature: sh:reifierShape and sh:reificationRequired for direct IRI property paths, so shapes can constrain the RDF 1.2 reifier metadata attached to statements (see RDF 1.2 Features). The relevant SHACL 1.2 Core draft is dated 2026-06-02. This is not a claim of full SHACL 1.2 conformance — it is one draft feature, explicitly scoped and tested.

Pydantic v2 projection

purrdf-shapes can transliterate a compiled SHACL-derived JSON Schema into a deterministic, typed Pydantic v2 package entirely in memory. The public emit_pydantic function consumes CompiledSchema; PydanticConfig requires the caller to supply the package name and package/module prose, so the library does not invent a vocabulary, namespace, or downstream brand.

Every $defs entry gets a stable import path, JSON property names remain exact through Pydantic aliases, and generated classes expose the originating definition through model_json_schema(by_alias=True). Pydantic runtime annotations enforce the representable portion. A JSON Schema assertion with no exact runtime annotation remains visible on that schema surface and produces a located entry in the always-computed json-schemapydantic-v2 LossLedger; a lossless input yields an empty ledger. The renderer itself has no Python dependency and stays wasm-clean. A dev-only Python oracle executes the generated code and checks the live reverse/schema surface.

LinkML 1.11 projection

The same CompiledSchema carrier can be projected to canonical LinkML 1.11 with emit_linkml. LinkmlConfig requires the caller’s schema IRI, name, description, default prefix, and complete prefix map, so PurRDF never mints a consumer vocabulary or identity. The returned LinkmlPackage includes the typed document, deterministic YAML, a reversible $defs-key mapping, and a located json-schemalinkml-1.11 loss ledger.

Classes and exact property aliases, types, enums, local references, inline objects, requiredness, homogeneous arrays, patterns, inclusive bounds, and LinkML boolean expressions are represented directly. Every unsupported assertion is classified by a closed capability table; malformed inputs, external/dynamic/dangling references, prefix mistakes, and deterministic-name collisions fail closed. parse_linkml and write_linkml preserve all JSON-compatible metamodel fields and provide byte-stable read/write round trips while rejecting YAML-only tags, duplicate keys, non-string keys, and non-finite numbers.

The Rust production path has no LinkML-toolkit dependency. CI uses the locked official LinkML 1.11.1 Python packages only as a differential oracle:

make linkml-oracle

TypeScript 7.0 projection

emit_typescript projects the same CompiledSchema into deterministic TypeScript 7.0 declarations. The caller supplies the package name and all package/module prose through TypeScriptConfig. The returned package contains one index.d.ts, a reversible $defs-key to exported-type map, and a located json-schematypescript-7.0 loss ledger; PurRDF invents no consumer identity or vocabulary.

The fixed declaration dialect uses strict plus exactOptionalPropertyTypes. Type aliases preserve JSON primitives and literals, required versus optional fields, explicit null, local recursive references, unions, intersections, homogeneous arrays, and bounded tuples. There are no runtime enums, mergeable interfaces, branded pseudo-validators, or any escape hatches. Invalid keywords, open/dangling references, and name collisions fail before bytes are emitted.

Runtime assertions outside TypeScript structural assignability are never silently erased: integer, numeric/string predicate, closure, pattern-property, dependency, conditional, negation, contains/unique, evaluation-state, and bounded-expansion gaps receive stable codes and JSON Pointer locations. CI classifies instances independently with a draft 2020-12 validator and compiles the generated declarations with the locked TypeScript 7.0.2 compiler, including fresh-literal and through-variable probes:

make typescript-oracle

The projection intentionally has no arbitrary TypeScript reader. TypeScript declarations do not define a unique runtime JSON acceptance relation, and the projection is many-to-one. The retained CompiledSchema plus the reversible name map remains the authoritative reverse surface. TypeScript is only a dev-time oracle dependency; the Rust emitter is filesystem-free and wasm-clean.

GraphQL September 2025 projection

emit_graphql projects CompiledSchema into deterministic GraphQL September 2025 SDL. GraphqlConfig has no defaults: the caller supplies the schema name, package and module prose, and a non-built-in fallback-scalar name. The returned GraphqlPackage contains schema.graphql, canonical name-map.json, the same name map as typed Rust data, a located json-schemagraphql-september-2025 loss ledger, and the production value codec.

The SDL is deliberately a type-system fragment. PurRDF emits paired output type and input input objects, but no query, mutation, or subscription root, resolver, pagination rule, authorization policy, federation directive, or other application behavior. A caller composes the fragment with its own executable schema.

The exact grammar includes GraphQL booleans, strings, numbers, the signed 32-bit Int domain, explicit nullability, finite JSON const/enum sets, closed object fields, requiredness, homogeneous lists, direct local $defs references and aliases, descriptions, and inline object helpers. One global collision-checked namespace covers types, helpers, and the fallback scalar; fields and enum symbols are checked in their GraphQL-local namespaces. The typed/canonical name maps retain the source definition keys, property keys, and finite JSON values.

GraphqlPackage::encode_input maps source JSON keys and finite values to input field names and enum symbols. decode_output performs the inverse for fields present in a GraphQL response, without inventing omitted selections. Unknown or incompatible values fail. This package codec is the precise reverse boundary; arbitrary GraphQL SDL has no unique JSON Schema acceptance relation and is not accepted as an inverse format.

GraphQL variable coercion differs from JSON Schema validation at these closed boundaries:

BoundaryLocated loss families
object fields and namesadditional properties, pattern properties, property names/counts
requiredness and recursionnullable-presence widening, one deterministic recursive-input nullability relaxation
listssingleton coercion, cardinality, contains, uniqueness, tuples, unevaluated items
scalar assertionsinteger domain delegation, numeric predicates, string predicates
applicatorsconditionals, dependencies, intersections, unions, oneOf, negation
runtime boundarycustom-scalar and unknown-keyword validation delegation

The caller-named fallback scalar is declared but PurRDF does not invent its parseValue, parseLiteral, or serialization semantics. Every delegated use is therefore ledgered. Loss entries carry stable codes and source JSON Pointer locations; an exact package has an empty ledger.

Emission fails before returning bytes for invalid caller configuration, malformed schema keywords, $id rebasing, external/indirect/dangling $ref, $dynamicRef/$recursiveRef, alias cycles, unsatisfiable closed required fields, and generated-name collisions. The fixed limits are 16 MiB for the input schema, each artifact, and one codec value; 65,536 definitions, fields per object, or finite values; depth 128; and 255 bytes per GraphQL name.

The independent dev oracle classifies source values with boon, builds the SDL with locked official GraphQL.js 16.14.0, and executes real variable coercion. It verifies exact agreement, every closed loss family and location, the name map and production codec, and deliberate corruption failures:

make graphql-oracle

GraphQL.js is dev-only. Emission and value translation remain filesystem-free, wasm-clean Rust.

From Python

from purrdf_native import shacl

report = shacl.validate(shapes_ttl="...", data_nt="...")
print(report["conforms"])  # True / False
print(report["results"])   # list of violation dicts

Each result dict keeps the stable keys focus, path, value, severity, component, source_shape, and message.

SARIF output

Validation reports stay structured in the engine; the SARIF 2.1.0 boundary is the separate purrdf-validate crate (purrdf::validate), which renders a report — or parser diagnostics — as a source-traced, byte-deterministic SARIF log for editors, CI, and code-scanning dashboards:

use purrdf::validate::{validate_to_sarif_string, SarifOptions};

let shapes = r#"
    @prefix sh:  <http://www.w3.org/ns/shacl#> .
    @prefix ex:  <http://example.org/> .
    @prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
    ex:PersonShape a sh:NodeShape ;
      sh:targetClass ex:Person ;
      sh:property [ sh:path ex:age ; sh:datatype xsd:integer ] .
"#;
let data = r#"<http://example.org/alice> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/Person> .
<http://example.org/alice> <http://example.org/age> "nope" .
"#;

let sarif = validate_to_sarif_string(shapes, data, &SarifOptions::default())
    .expect("sarif produced");
assert!(sarif.contains("\"version\": \"2.1.0\""));

Lower-level entry points (build_report_sarif, build_diagnostics_sarif) build a SarifLog value instead of a string, so a host can merge runs before serializing.

Conformance

The validator is gated by the vendored W3C data-shapes suite, a vendored DASH SHACL-AF/rules corpus, and a first-party frozen corpus of 69 cases with byte-frozen expected reports; SHACL Rules output is compared to expected inferred graphs by RDFC-1.0 isomorphism. See Conformance & Testing.

ShEx

purrdf-shex (re-exported as purrdf::shex) is PurRDF’s native Shape Expressions Language 2.1 engine: the schema layer and the shape-map validator, pure Rust and wasm-clean.

Schemas: ShExC and ShExJ

  • ShExC (the compact syntax, spec §6) — a hand-rolled lexer and recursive-descent parser covering the full grammar: directives, start, the AND/OR/NOT shape algebra, node constraints and the facet table, value sets with stems/ranges/exclusions, triple expressions with all cardinality forms, $/& labels and inclusions, ^ inverse, annotations and %…{ … %} semantic actions, with relative-IRI resolution against BASE via purrdf-iri.
  • ShExJ (the JSON wire format, spec Appendix A) — strict, round-tripping serde support matching the shexTest ground truth.
  • Structural checks (spec §5.7) — dangling references, label collisions, reference-only cycles, and the negation-stratification requirement.
use purrdf::shex::{check_structure, parse_shexc, to_shexj};

let schema = parse_shexc(
    "PREFIX ex: <http://example.org/>\n\
     PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>\n\
     ex:S { ex:p xsd:integer? }",
    None,
)?;
check_structure(&schema).expect("well-formed");
let json = to_shexj(&schema);

Hard-fail discipline: every malformed schema is a typed ShexError — no lenient mode, no panics on any input.

Validation with shape maps

Validation (spec §5.2–§5.5) is fixed shape-map validation over the frozen purrdf-core dataset IR, in interned TermId space: you supply (node, shape) associations and the validator decides conformance for each. It covers node constraints (node kind, datatype with lexical-validity checking, string/numeric facets, value sets with stems and exclusions), EXTRA/CLOSED triple-expression matching with EachOf/OneOf partitioning and group cardinalities, inverse constraints, typing-based recursion, and an EXTERNAL resolver hook.

From Python:

from purrdf_native import shex

result = shex.validate(my_schema_shexc, my_data_ttl,
                       [("https://example.org/alice", "https://example.org/PersonShape")])
print(result["conforms"])

Conformance

The engine is gated against the vendored official shexTest suite, pinned at tag v2.1.0 (vectors/shexTest/): the full validation/ manifest, the schemas/ ShExC/ShExJ pairs and round-trips, and the negative syntax and negative structure suites. At the time of writing the validation manifest passes 1,105/1,105 attempted cases with zero expected-failures and an empty trait-skip list (imports and semantic actions included); the live scoreboard is docs/CONFORMANCE.md.

Reported conformance is logic-level (pass/fail parity), per suite convention; result-structure conformance is upstream-experimental.

SHACL or ShEx?

PurRDF implements both natively over the same IR, so the choice is yours, not the toolkit’s: SHACL suits constraint reporting (violations with severities, SARIF output) and SHACL-SPARQL escape hatches; ShEx suits schema-like conformance decisions over explicit shape maps.

Entailment

purrdf-entail (re-exported as purrdf::entail) is native, wasm32-clean entailment for the PurRDF RdfDataset IR. A family of engines sits behind one facade, each the right tool for its SPARQL entailment regime — closing a dataset to its inferred fixpoint entirely in interned TermId space, with no external reasoner, no async runtime, and no string round-trip.

Surface map

Entry pointRegime(s)Engine
materialize(ds, regime)Simple, RDF, RDFS, OWL-RLForward materialization (“chase”) over a fixed rule set via a native semi-naive fixpoint.
materialize_dl(...)OWL-DirectOpen-world OWL DL over an ALCOIQ tableau — it needs the query’s class expressions, so it is not reachable through the plain materialize facade.
materialize_rif(...)RIFRIF-Core rule entailment over a parsed RuleSet.
Regime::from_iri(iri)Parse a sparql:entailmentRegime IRI to its enum.
use purrdf::entail::{materialize, Regime};

// Close a frozen dataset to its RDFS fixpoint; the result is a new dataset.
let closed = materialize(&ds, Regime::Rdfs).expect("materializes");

The chase (Simple / RDF / RDFS / OWL-RL)

materialize runs a forward-materialization chase: a fixed rule set for the selected regime, applied by a semi-naive fixpoint until no new quads appear. Because it runs over the frozen IR, it is deterministic — a given input and regime always yields the same closure — and because it works in TermId space, no term is ever re-parsed or re-serialized along the way.

Typical use: materialize first, then query with the plain SPARQL engine or validate the closure with SHACL (the SHACL validator itself performs no inference).

OWL-Direct: the tableau

OWL-Direct semantics is open-world Description Logic, which a forward chase cannot answer. materialize_dl runs an ALCOIQ tableau instead — answering instance and subsumption queries via classification, realization, and query-directed materialization. Because it needs the query’s class expressions, it has its own entry point rather than hiding behind materialize.

RIF

materialize_rif evaluates RIF-Core rules over a parsed RuleSet, covering the SPARQL RIF entailment regime.

The D-entailment boundary

D (datatype) entailment is a typed, spec-inherent boundary: requesting it returns EntailError::Unsupported rather than silently defaulting to a weaker regime. (No case in the vendored W3C corpus exercises D-entailment alone.) This is the workspace-wide hard-fail discipline: an unsupported regime is an error you can handle, never a wrong answer.

Invariants

  • No minted vocabulary. Every constant in the crate’s vocab module is a standard rdf:/rdfs:/owl: IRI drawn from the entailment specs themselves — the crate fabricates none, per the toolkit-not-ontology rule.
  • Dependency-lean. The only dependency is purrdf-core, so the engines carry into Rust, WebAssembly, and C unchanged.
  • Deterministic. Same input + regime → same closure, always.

Conformance

All 70 W3C entailment cases pass at the time of writing — RDF/RDFS/OWL-RL chase, OWL-Direct (DL) tableau, RIF-rule, and RDF-axiomatic predicate typing — run through the SPARQL conformance harness. The live scoreboard is docs/CONFORMANCE.md.

GTS Graph Transport

GTS (Graph Transport Substrate) is an ontology-independent binary container and transport format for RDF 1.2 datasets and content-addressed binary payloads. PurRDF hosts the reference Rust engine, purrdf-gts (re-exported as purrdf::gts).

This chapter is a user-level tour. The wire format itself is specified in docs/GTS-SPEC.md — consult the spec for framing, fold semantics, registries, and conformance classes; nothing here supersedes it.

The container model, at a high level

A GTS file is a CBOR Sequence of one or more append-only segments. Each segment is a deterministic CBOR header followed by deterministic CBOR frames chained by BLAKE3 content identifiers. The logical dataset is obtained by a deterministic fold over the segment sequence: quads, reifiers, annotations, and binary blobs all become rows of the folded container graph.

Properties that fall out of this design:

  • Content-addressed and append-only — history is never rewritten; suppression is itself an appended record. Multi-segment files compose by simple concatenation.
  • Partial readability, total reader — the reader verifies the BLAKE3 chain and folds what it can; undecodable frames (unknown codec, encrypted without a key) degrade to opaque nodes plus a diagnostic instead of aborting.
  • Binary payloads ride along — the blobs a graph references travel in the same file, content-addressed like everything else.
  • RDF 1.2-native — the spec formalizes the triple-term and rdf:reifies mapping, blank-node scoping, and multi-segment value union.

Reading and writing from Rust

The container engine (purrdf-gts) owns the wire-format machinery — reader, writer, fold, verify, COSE, trust policy:

use purrdf::gts::reader;

// Fold GTS bytes into the container graph model, verifying the BLAKE3 chain.
let graph = reader::read(&bytes, /* allow_segments */ true, /* expected_head */ None);

// The fold is total: quads, reifiers, annotations, and blobs are all rows,
// and anything undecodable is preserved as an opaque node plus a diagnostic.
println!("{} quads, {} blobs", graph.quads.len(), graph.blobs.len());

The writer authors frames and produces byte-deterministic single-segment snapshots (Writer::deterministic) — the GTS writer is under the same determinism invariant as every PurRDF serializer.

The RdfDataset import/export path lives one layer up: the umbrella crate’s gts module combines the container engine with the RDF-level adapter (snapshot composition, content-chain verification), so RDF-facing GTS work goes through purrdf directly.

Signing and encryption

Frames can be signed and encrypted with COSE; OpenPGP-based checks and a trust-policy layer are also part of the engine. All cryptography is pure Rust — no C toolchain, threads, or syscall dependencies — which is what keeps the whole engine wasm-friendly. An encrypted frame you cannot decrypt is simply an opaque node in the fold: the container remains readable.

Conformance vectors

GTS conformance is defined against a frozen, language-neutral vector corpus (vectors/ in the repository), shared byte-exact across the sibling GTS engines in other languages. The vectors are never regenerated or “fixed” in this repository — the format is governed in the gmeow-gts project, alongside the specification and the other reference engines.

Known limitation at the C ABI

The GTS star-layer round-trip of a dataset containing quoted triples / reifier bindings currently fails through the kernel to_gtsread_graphimport_gts_graph path (star-free round-trips are lossless). See Getting Started: C.

Slices, Mappings & Provenance

Beyond the engines, PurRDF carries the plumbing a serious vocabulary or data-pipeline project needs: a slice catalog for organizing authored RDF, an explicit loss ledger for lossy projections, and native codecs for the SSSOM and FnO interchange formats.

The slice catalog

purrdf-slice (re-exported as purrdf::slice) is tooling for ontology/vocabulary repositories organized as slices — directories of authored RDF (slices/<group>/<name>/), each described by a manifest.ttl:

  • Catalog — manifest-based discovery (SliceCatalog::discover), typed slice metadata (SliceRecord, SliceTier), and artifact roles. Slice identity comes from the manifest, not the directory name.
  • Ownership & dependencies — term-ownership analysis (every declared term has exactly one owning slice), dependency edges with evidence, forbidden-edge rules (extension slices depend only on core), and machine-applicable fix suggestions.
  • Content addressing — deterministic artifact digests and cache keys for incremental pipelines.
  • Emitters — projection/mapping emitters and lints: prefix maps, JSON-LD contexts, FnO function catalogs, claim views.

True to the rule that PurRDF mints no vocabulary IRIs, every term the slice framework reads or emits belongs to the caller’s vocabulary: a SliceVocab is caller-constructed (it has no Default) and threaded through every public entry point.

use std::path::Path;
use purrdf::slice::{SliceCatalog, SliceVocab};

// Your vocabulary namespace — PurRDF fabricates none.
let vocab = SliceVocab::for_namespace("https://example.org/vocab/");
assert_eq!(vocab.slice_class(), "https://example.org/vocab/Slice");

// Discover every slice under the repository root from its manifest.ttl.
let catalog = SliceCatalog::discover(Path::new("slices"), vocab)
    .expect("slices discovered");
for slice in catalog.records() {
    println!("{} ({:?})", slice.manifest.slice_iri, slice.manifest.tier);
}

The loss ledger

PurRDF’s projections are allowed to be lossy — but never silently. The kernel carries a machine-readable RDF↔GTS loss matrix (generated/rdf-loss-matrix.json, a generated artifact) and a LossLedger API: when a star-incapable codec drops reifier bindings, or CSV results drop provenance, the realized count is recorded and surfaced to the caller. See Codecs & Determinism and Result Formats.

Provenance

purrdf-core includes a generic provenance sidecar for the frozen IR — attribution, origin sets, and per-quad provenance that engines can carry without polluting the data graph. The SPARQL results extension (Result Formats) and the SARIF boundary both resolve these runtime-only provenance ids to public IRIs at their serialization edges.

SSSOM and FnO

Two native interchange codecs live in the kernel:

  • SSSOMSimple Standard for Sharing Ontological Mappings mapping-set TSV support (SssomMappingSet, SssomMapping, with typed diagnostics), for carrying cross-vocabulary mappings alongside your data.
  • FnO — a Function Ontology function-catalog codec (FnoCatalog, fno_to_quads, fno_to_ntriples), used by the slice emitters to describe function catalogs as RDF.

As with everything else in the toolkit, these are codecs for caller data — PurRDF does not define mappings or functions of its own.

rdflib Compatibility

Python’s rdflib is the incumbent RDF library of the ecosystem, and PurRDF meets it in two tiers: an explicit compat module, and an opt-in drop-in shadow.

Tier 1: the explicit compat layer

The main purrdf wheel ships an rdflib compatibility layer backed by the native engine:

from purrdf.compat.rdflib import Graph

g = Graph()
g.parse(data="<https://example.org/a> <https://example.org/b> <https://example.org/c> .",
        format="turtle")

This is the recommended path for new code that wants an rdflib-shaped API on the PurRDF engine: the import name is honest, and it coexists with a genuine rdflib installation.

Tier 2: the purrdf[rdflib] shadow distribution

For a literal, zero-change import rdflib, install the opt-in extra:

pip install purrdf[rdflib]

This pulls in the separate purrdf-rdflib distribution, whose top-level rdflib package re-exports the compat surface, so existing third-party code doing import rdflib / from rdflib.namespace import RDF transparently runs on purrdf. Caveat: that shadow claims the rdflib import name and must never be installed alongside the genuine rdflib — the two cannot co-inhabit one environment. It is a separate distribution (never bundled into the main purrdf wheel) precisely so environments that need the real rdflib simply omit it.

How compatibility is kept honest

The compat layer is not “best effort” — it is gated in CI as part of the single conformance matrix (docs/CONFORMANCE.md):

  • The rdflib drop-in (LSP) gate runs rdflib 7.6’s own vendored test suite against the purrdf drop-in.
  • The parity suite runs first-party differential tests of purrdf.compat against the real rdflib 7.6.

Both use strict expected-failure ledgers: every known divergence is listed with a per-test reason, an unexpected failure breaks the build, and a silently fixed divergence also breaks the build until the ledger shrinks. The ledgered residuals cover corners like Graph-subclass identity through set operators, rdf:List/Collection mutation, Result.bindings / SELECT * subselect projection, graph-prefix forwarding, and legacy ConjunctiveGraph semantics — consult the ledgers for the current, exact list.

Performance

A report-only benchmark harness times the native-backed purrdf.compat.rdflib drop-in against the real rdflib on parse, serialize, SPARQL, and triple-pattern iteration (make bench-python). Methodology and a representative (host-dependent) results table live in docs/BENCHMARKS.md — numbers vary by host, so reproduce locally rather than trusting a fixed multiplier. See Performance for the philosophy.

RDF/JS in JavaScript

In JavaScript, PurRDF does not invent its own API shape: the npm package @blackcatinformatics/purrdf implements the RDF/JS community specifications — DataFactory, DatasetCore, and Stream/Sink — over the wasm-compiled native engine. Code written against RDF/JS interfaces works with PurRDF terms and datasets.

The data factory

DataFactory covers the standard RDF/JS term constructors — namedNode, blankNode, literal(value, languageOrDatatype?), variable, defaultGraph, quad, fromTerm, fromQuad — plus the deliberate RDF 1.2 extensions no incumbent RDF/JS library carries:

import { ready, DataFactory } from "@blackcatinformatics/purrdf";
await ready();
const f = new DataFactory();

// RDF/JS standard surface.
const q = f.quad(
  f.namedNode("https://ex/alice"),
  f.namedNode("https://ex/knows"),
  f.namedNode("https://ex/bob"),
);

// RDF 1.2 extensions: quoted triple terms and base-direction literals.
const quoted = f.quotedTriple(q.subject, q.predicate, q.object);
const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");

typedLiteral is a convenience alongside the spec’s overloaded literal.

Datasets

Dataset implements RDF/JS DatasetCore: add, delete, has, match, size, and iteration (for (const quad of dataset)), plus parsing and serialization through the native codecs:

const ds = Dataset.parse("<https://ex/s> <https://ex/p> <https://ex/o> .", "ntriples");
for (const quad of ds.match(null, f.namedNode("https://ex/p"), null)) {
  console.log(quad.subject.value);
}
const trig = ds.serialize("trig");

Accepted format names: turtle, ntriples, nquads, trig, rdfxml, or their media types. Because these are the native codecs, output is byte-deterministic and identical to what the Rust, Python, and C surfaces emit (Codecs & Determinism).

SPARQL

Use QueryEngine when running more than one query or when the caller wants typed results instead of raw strings. The engine owns the native SPARQL plan cache and returns package-root terms and datasets:

import { QueryEngine } from "@blackcatinformatics/purrdf";

const engine = new QueryEngine();
const result = engine.select(
  ds,
  "PREFIX ex: <https://ex/> SELECT ?o WHERE { ex:s ex:p ?o }",
);
console.log(result.rows.take(0)?.o?.value);

const graph = engine.construct(
  ds,
  "PREFIX ex: <https://ex/> CONSTRUCT { ex:copy ex:p ?o } WHERE { ex:s ex:p ?o }",
);

SELECT rows are single-owner and lazy across the wasm boundary. Iterate result.rows, call result.rows.take(index) for indexed consumption, or call result.rows.toArray() to materialize the remaining rows. A row can be consumed once; call result.free() when abandoning a result before exhaustion.

QueryEngine.queryRaw(...) serializes SELECT/ASK results as SPARQL Results JSON/XML/CSV/TSV and graph results through the same graph formats accepted by Dataset.serialize. QueryEngine.update(...) applies SPARQL UPDATE atomically: the dataset changes only after the whole update succeeds.

Streams and sinks

The package speaks the async RDF/JS Stream/Sink protocol over the purrdf-events ingestion seam:

  • Sink — a streaming consumer: push(quad) per quad, finish() returns the accumulated Dataset.
  • datasetToStream(dataset) / streamToDataset(stream) — bridge between a Dataset and RDF/JS streams, for piping into or out of other RDF/JS tooling.

Scope notes

  • The engine is in-memory; there is no persistent store in the wasm build. SPARQL runs over the in-memory dataset; this package provides no network resolver, so remote SERVICE and LOAD fail explicitly.
  • A quoted-triple term as a quad object currently round-trips only through N-Quads (a current native serializer limitation for the other formats).

Design Rules & Invariants

PurRDF must stay fast, deterministic, and boring: one engine, one behavior, carried verbatim into Rust, Python, WebAssembly, and C. That promise is kept by a small set of hard invariants, each enforced by CI rather than by convention. The canonical statement is AGENTS.md in the repository; this chapter explains the why.

No Cargo features, ever

The workspace has zero feature flags, and a CI script gates it. PurRDF is a data carrier, and optionality changes semantics per consumer — two builds of “the same version” that parse or serialize differently would defeat the whole point. No [features], no optional dependencies, no cfg-gated behavior differences. Every consumer gets the same byte-identical semantics.

PurRDF is NOT an ontology — it mints no vocabulary IRIs

Every vocabulary the library reads or writes — slice manifests, statement-metadata downcast, box roles, SPARQL extension-function namespaces, JSON-Schema namespaces — is caller-supplied configuration with no fabricated default. A feature exercised without its vocabulary hard-errors or stays inactive. The library never hardcodes a vendor namespace (the GMEOW ontology is a consumer; the dependency arrow never points from purrdf to it), and test fixtures use example.org. Consumer-config types (SliceVocab, Namespaces, StatementMetadataVocab) are unified behind an OntologyProfile a downstream builds once.

Byte determinism

Serializers and the GTS writer are byte-deterministic. No iteration-order, time, or RNG dependence is permitted in any output path; hot maps use fixed-key ahash for this reason. Changes that alter emitted bytes must update the affected golden files, visibly.

The kernel ring-fence

purrdf-core must never depend on oxigraph or PyO3 — the whole workspace is oxigraph-free, and a hygiene gate asserts the dependency tree. The three foundation leaves (purrdf-iri, purrdf-xsd, purrdf-events) keep zero runtime dependencies. Diagnostics stay structured and SARIF-free in the kernel; the SARIF boundary is the purrdf-validate leaf.

Everything is wasm-able

Every release crate must build for wasm32-unknown-unknown, and CI hard-fails otherwise. No dependency may drag in threads, the filesystem, C toolchains, or wall-clock/RNG syscalls on the wasm path — cryptography stays pure Rust for exactly this reason. This is what makes the JavaScript package the same engine rather than a port.

Hard-fail, never wrong

Across the toolkit, out-of-scope input is a typed error, never a partial answer: malformed RDF is an RdfDiagnostic, an unsupported SPARQL builtin is EvalError::Unsupported, a malformed ShEx schema is a ShexError, D-entailment is EntailError::Unsupported, and an unsupported results projection is a typed format error. Lossy-by-design projections are permitted but loud, via the loss ledger.

Conformance corpora are the contract

The W3C and community test suites are vendored, byte-frozen, and SHA-256-verified; harnesses assert exact counts and enforce XPASS discipline on their expected-failure ledgers. See Conformance & Testing.

Supporting rules

  • Measured performance — perf claims require a criterion bench, not an adjective (Performance).
  • One version, lockstep releases — crates.io, PyPI, and npm ship one workspace version (Versioning & Releases).
  • Stable toolchain only — the workspace is nightly-free; the MSRV floor (currently 1.96) is enforced by a dedicated CI job.
  • Brand — the project is PurRDF in prose and purrdf in identifiers (docs/BRAND.md).

Conformance & Testing

Every PurRDF engine is gated by its official test suite. The suites are vendored and byte-frozen in-repo — never hand-edited, SHA-256-verified on every make check so a silent content edit fails the build. The full, live scoreboard is docs/CONFORMANCE.md; this chapter explains how the machine works.

The single conformance matrix

The native Rust W3C harnesses and the Python rdflib drop-in gate are reported together as one scoreboard:

make conformance    # aggregates every suite into one table

The aggregator runs each suite in a fixed order and prints per-suite pass / xfail-or-skip / fail counts with an overall RED/GREEN verdict. It exits non-zero on any unexpected failure, and the rendered matrix in docs/CONFORMANCE.md is itself drift-guarded in CI (the gate fails if the committed block is stale).

What is gated

EngineSuite
IRI (RFC 3987)W3C IRI + RFC 3986 §5.4 resolution vectors
Syntax codecsW3C rdf-tests (Turtle/TriG/N-Triples/N-Quads/RDF-XML)
RDFC-1.0W3C rdf-canon fixtures
SPARQL 1.1/1.2full W3C sparql11 + sparql12 + entailment suites
SHACLW3C data-shapes + DASH SHACL-AF/rules + a first-party frozen corpus
ShEx 2.1shexTest v2.1.0 (validation, schemas, negative syntax/structure)
Entailmentthe W3C entailment cases (via the SPARQL harness)
GTSfrozen cross-language vectors, byte-exact
rdflib drop-inrdflib 7.6’s own vendored tests + first-party parity

At the time of writing every suite is green — for example 1,105/1,105 attempted shexTest validation cases, 126/126 W3C SHACL, 250/250 codec round-trips, and 70/70 entailment cases — with the handful of remaining non-passes strictly ledgered (e.g. five SPARQL fixtures with upstream-errata non-canonical XSD lexicals). Always read the current numbers from docs/CONFORMANCE.md rather than this snapshot.

Ledger discipline

A harness never skips silently. Four mechanisms keep the scoreboard honest:

  1. Exact totals — each harness asserts the number of discovered tests, so corpus drift fails loudly.
  2. XFAIL ledgers — every known gap is listed with a reason string, and the harness asserts it still fails. Fixing a gap without removing its ledger entry breaks the build (XPASS discipline), so the ledgers double as roadmaps.
  3. Trait skips (ShEx only) — whole spec features can be skipped by manifest trait tags, counted exactly. The list is currently empty.
  4. A monotone budget (ratchet) — a committed baseline records the exact allowed count of ledgered gaps per suite. A larger live count fails RED (regression), and a smaller one also fails RED until the budget is lowered to lock the gain in. The budget may only ever be edited downward, which makes “the skip list only shrinks” a mechanical guarantee rather than a convention.

Running the suites locally

make conformance                                     # the single matrix
cargo test -p purrdf-shex                            # all four ShEx suites
cargo test -p purrdf-shapes --test w3c_conformance   # W3C SHACL scoreboard
cargo test -p purrdf-sparql-conformance              # W3C SPARQL
cargo test -p purrdf-rdf                             # RDFC-1.0 + codec goldens
cargo test -p purrdf-gts                             # GTS vectors

make check — the full local gate (fmt, clippy, build, tests, hygiene) — runs the Rust suites as part of the workspace gate.

Frozen means frozen

The GTS vectors in vectors/ are shared byte-exact with the sibling GTS engines in other languages and are never regenerated in this repository; the wire format is governed in gmeow-gts. The same never-hand-edit rule applies to every vendored corpus and everything under generated/.

Performance

PurRDF’s performance is measured, never asserted. No number in the project’s documentation is a guarantee: benchmarks are report-only, timing-sensitive, and vary by host, CPU, allocator, and build flags. Treat every figure as a host-dependent illustration you reproduce locally — not a promise of “N× faster.” The methodology and a representative results table live in docs/BENCHMARKS.md.

Fast by construction

The engine-level speed comes from the IR design (The Interned Dataset IR): every term stored once in a string arena addressed by copyable NonZeroU32 ids, fixed-key ahash everywhere hot, frozen Box<[QuadRow]> quad tables with lazy ordinal permutation indexes (~4 bytes/quad per axis), and evaluation in TermId space so solution comparison is an integer compare.

Crucially, the layout itself was chosen by benchmark: the crates/rdf-core/benches/ir_layout.rs criterion suite measures array-of-structs vs. struct-of-arrays vs. predicate-adjacency layouts on allocation counts, high-water memory, and end-to-end latency — and the shipped layout is whichever wins.

The two benchmark layers

LayerWhat it measuresHow to run
Rust criterion suitesNative engine hot paths — IR layout, copy-on-write mutation, pack index alternatives, codecs, SPARQL lexing/evaluation/planning, SHACL validation, entailment chase, GTS authoring, IRI parsing.make bench
Python compat harnesspurrdf.compat.rdflib (the native-backed drop-in) vs. the real rdflib 7.x on parse, serialize, SPARQL, and triple-pattern iteration, over a deterministic example.org corpus.make bench-python

Both layers are report-only: they are never part of make check, and no test gate asserts a speedup.

The discipline for changes

Any change claiming a performance win must extend the criterion benches rather than asserting the speedup in prose. Where a planner or algorithm choice matters for correctness-adjacent behavior, it is gated by deterministic tests instead of timings — for example, the cost-based BGP planner’s win over the retired structural heuristic is asserted by unit tests that count real intermediate rows, and by a differential corpus test, while the criterion bench merely watches for regressions.

NativeSparqlEngine::explain_query exposes the chosen BGP join order so planner decisions can be audited without running the query (SPARQL: Querying).

Reproducing locally

make bench                              # the default criterion set
cargo bench -p purrdf-iri --bench parse # a single package's bench
cargo bench -p purrdf-core --bench pack_index_compare # pack index experiment
make bench-python                       # the rdflib comparison harness

Benchmark on a quiet machine, and compare like with like: allocator, CPU scaling, and build flags all move the numbers.

Versioning & Releases

PurRDF ships to three registries — the crates.io crate suite, the PyPI purrdf package, and the npm @blackcatinformatics/purrdf package — from one workspace version, in lockstep. The full process is docs/RELEASE.md.

Pre-1.0 semver policy

While the version is 0.x:

  • a minor bump (0.x0.(x+1)) may include breaking API changes;
  • a patch bump (0.x.y0.x.(y+1)) is bugfix-only and API-compatible.

All three published surfaces share one workspace version and are released together. That coherence is enforced in CI: a version-coherence check fails the build if the three version sources disagree.

MSRV policy

The supported minimum Rust is rust-version in the root Cargo.toml — currently 1.96 — pinned to the stable toolchain (the workspace is nightly-free by policy) and enforced by a dedicated CI MSRV job. Raising the MSRV is a notable change recorded in the changelog and, pre-1.0, rides a minor bump.

Tag-driven trusted publishing

Releases are tag-driven: rust-v<version> publishes the crate suite to crates.io, py-v<version> publishes to PyPI, and npm-v<version> publishes the wasm package to npm. The lanes share the supply-chain posture of the cargo lane:

  • publication uses Trusted Publishing through GitHub Actions OIDC — no long-lived registry secret;
  • the privileged publish jobs use pinned actions and no dependency cache;
  • every .crate package receives a GitHub build-provenance attestation;
  • the package set receives an SPDX SBOM and SBOM attestation;
  • the release crate set is checked on wasm32-unknown-unknown before publishing;
  • every workspace crate version must match the tag version.

Two crates are deliberately never published: purrdf-capi (built via cargo-c, distributed as libpurrdf) and purrdf-sparql-conformance (the test harness).

Cutting a release

The coherent flow from main uses the make helpers so the three lanes can never drift:

# 1. Bump all three version sources in lockstep (fails unless they end up equal).
make bump VERSION=0.2.2

# 2. Regenerate the changelog from the conventional-commit history.
make changelog

# 3. Review, then commit the release bump + changelog.
git add -A && git commit -m "chore(release): 0.2.2"

# 4. Gate: fmt, clippy, tests, hygiene, and the version-coherence + wasm checks.
make check

# 5. From an up-to-date main, cut and push all three tags in one command.
make release-tags VERSION=0.2.2

make release-tags refuses to run unless the working tree is clean, the branch is main, the version check passes, and VERSION matches the tree — then it creates and pushes the rust-v, py-v, and npm-v tags together. Each tag triggers its own lane, and the cargo lane additionally publishes a GitHub Release built from the committed CHANGELOG.md.

Citing PurRDF

Releases carry a DOI; if you use PurRDF in research, please cite it — see CITATION.cff in the repository.