Introduction
PurRDF is an RDF 1.2 toolkit: primitives, codecs, SPARQL, SHACL, ShEx, entailment, and graph transport, implemented once in Rust and carried verbatim into Python, WebAssembly/JavaScript, and C. It is developed by Blackcat Informatics® Inc. and published under MIT OR Apache-2.0.
One RDF engine. One behavior. Every language.
Why does PurRDF exist?
RDF tooling fragments along two axes.
Across languages: every ecosystem has its own parser, with its own bugs, its own corner-case interpretations, and its own subset of the spec. Move a graph from a Rust service to a Python pipeline to a browser and you have silently changed what the data means three times.
Across time: RDF 1.2 — triple terms, reifiers, base-direction literals — is where the standard is going, and almost no incumbent library carries it.
PurRDF exists so that a graph is the same graph everywhere. It is a from-scratch, dependency-light Rust core — parser to SPARQL engine to SHACL validator to binary transport — exposed through native bindings rather than reimplemented per language.
What’s inside
- RDF 1.2 primitives — an immutable, value-interned dataset IR (
TermIdspace, string arena, copy-on-write mutation), with triple terms in object position, reifier/annotation side-tables, and base-direction literals. See The Interned Dataset IR. - Native codecs — first-party parsers/serializers for Turtle, TriG, N-Triples, N-Quads, RDF/XML, JSON-LD (star), and YAML-LD, with byte-deterministic output. See Codecs & Determinism.
- Canonicalization — W3C RDFC-1.0 plus dataset diff and isomorphism. See Canonicalization & Diff.
- SPARQL 1.1/1.2 — native parser → algebra → multiset evaluator, gated by the W3C conformance suites. See SPARQL.
- SHACL and ShEx — native validators for both shape languages. See Validation.
- Entailment — Simple/RDF/RDFS/OWL-RL materialization, an OWL-Direct tableau, and RIF-Core rules. See Entailment.
- GTS graph transport — a single-file, content-addressed, append-only container for RDF 1.2 graphs and binary payloads. See GTS Graph Transport.
- Slices, mappings, and provenance — a slice catalog, an explicit RDF↔GTS loss ledger, SSSOM, and FnO. See Slices, Mappings & Provenance.
Two design rules worth knowing on day one
No feature flags — ever. There are deliberately no Cargo feature flags anywhere in the workspace, and CI enforces this. A data carrier must not have optional behavior: optionality changes semantics per consumer, so every consumer gets the same byte-identical semantics instead.
PurRDF is a toolkit, not an ontology — it mints no vocabulary IRIs. Every
vocabulary the library reads or writes is caller-supplied configuration with no
fabricated default. A feature exercised without its vocabulary hard-errors or
stays inactive; it never invents an IRI for you. (Test fixtures use
example.org.)
The full invariant list is in Design Rules & Invariants.
Why RDF 1.2?
RDF 1.2 (and SPARQL 1.2) add first-class statement-level metadata to the data
model: triple terms that can appear in object position, reifiers that
name occurrences of a triple, and base-direction literals
(rdf:dirLangString) for bidirectional text. PurRDF treats these as core data
model, not an extension: they flow through the IR, the codecs, SPARQL, SHACL
(a scoped SHACL 1.2 draft feature), the RDF/JS surface, and the GTS transport.
See RDF 1.2 Features.
Where PurRDF sits
PurRDF is the library layer of a small family of linked-data projects: it is the data backbone of the GMEOW stack and the reference home of the Rust GTS engine — but it assumes nothing about your ontology or application.
How to read this book
- New users: start with Getting Started in your language, then read the Concepts chapters.
- Engine users: jump to SPARQL, Validation, or Entailment.
- Integrators: see Interop and GTS Graph Transport.
- Contributors: read the Project chapters, then AGENTS.md in the repository.
API reference documentation lives on docs.rs/purrdf; the repository is github.com/Blackcat-Informatics/purrdf.
Getting Started: Rust
The single dependency a Rust downstream needs is the umbrella
purrdf crate. It re-exports the RDF 1.2
implementation surface at its root and carries every other published crate
under a stable module (purrdf::sparql, purrdf::shapes, purrdf::shex,
purrdf::gts, purrdf::entail, purrdf::validate, purrdf::slice,
purrdf::iri, purrdf::xsd, purrdf::events) — anything a consumer
legitimately imports is reachable from purrdf alone, never by reaching into
a sub-crate.
cargo add purrdf
The MSRV is Rust 1.96 (stable toolchain only; the workspace is nightly-free by policy).
Build, freeze, serialize, parse
use purrdf::{parse_dataset, serialize_dataset, RdfDatasetBuilder, RdfLiteral, SerializeGraph};
// Build a dataset in interned TermId space.
let mut b = RdfDatasetBuilder::new();
let alice = b.intern_iri("https://example.org/alice");
let knows = b.intern_iri("http://xmlns.com/foaf/0.1/knows");
let bob = b.intern_iri("https://example.org/bob");
let name = b.intern_iri("http://xmlns.com/foaf/0.1/name");
let hi = b.intern_literal(RdfLiteral::simple("Alice"));
b.push_quad(alice, knows, bob, None);
b.push_quad(alice, name, hi, None);
let ds = b.freeze().expect("freeze");
// Serialize to any native codec and parse back, losslessly.
let ttl = serialize_dataset(&ds, "text/turtle", SerializeGraph::Dataset).unwrap();
let back = parse_dataset(&ttl, "text/turtle", None).unwrap();
assert_eq!(back.quad_count(), 2);
The builder→freeze split is the heart of the API: you intern terms and push
quads on a mutable RdfDatasetBuilder, then freeze() into an immutable,
indexed RdfDataset that every engine (SPARQL, SHACL, ShEx, entailment)
evaluates over. See The Interned Dataset IR.
Parsing text directly
let turtle = r#"
@prefix ex: <https://example.org/> .
ex:cat ex:says "meow" .
"#;
let dataset = purrdf::parse_dataset(turtle.as_bytes(), "text/turtle", None)
.expect("valid Turtle");
assert_eq!(dataset.quad_count(), 1);
Malformed input is a typed RdfDiagnostic with a source location where the
codec can provide one — never a silent partial parse.
Reaching the other engines
Every engine hangs off the same facade. For example, the zero-dependency IRI leaf and the ShEx schema layer:
let iri = purrdf::iri::parse("https://example.org/cat").expect("valid IRI");
assert_eq!(iri.as_str(), "https://example.org/cat");
let schema = purrdf::shex::parse_shexc(
"PREFIX ex: <https://example.org/>\nex:Cat { ex:says . }",
None,
).expect("valid ShExC");
When to depend on a sub-crate instead
Most applications should stop at purrdf. The sub-crates
(purrdf-core, purrdf-rdf, purrdf-columnar, purrdf-sparql-eval, purrdf-shapes,
purrdf-shex, purrdf-gts, purrdf-entail, purrdf-validate,
purrdf-slice, purrdf-iri, purrdf-xsd, purrdf-events) exist for
consumers that want exactly one engine — for example, a tool that only needs
IRI parsing can depend on the zero-dependency purrdf-iri alone. The crate
map is in the
repository README.
Every release crate builds cleanly for wasm32-unknown-unknown, so the same
Rust code paths work in native and wasm hosts.
Next steps
- The Interned Dataset IR — how the IR works and why it is fast.
- SPARQL: Querying — running queries over a frozen dataset.
- docs.rs/purrdf — the full API reference.
Getting Started: Python
The Python package wraps the same native Rust engine — not a reimplementation — so parsing, serialization, SPARQL, and validation behave identically to the Rust, JavaScript, and C surfaces.
pip install purrdf
Parsing
import purrdf
quads = purrdf.parse(
'<https://example.org/alice> <http://xmlns.com/foaf/0.1/name> "Alice" .',
purrdf.RdfFormat.TURTLE,
)
Validation: SHACL and ShEx
The native validation engines are exposed from the purrdf_native extension
module:
from purrdf_native import shacl, shex
report = shacl.validate(shapes_ttl=my_shapes, data_nt=my_data)
print(report["conforms"])
result = shex.validate(my_schema_shexc, my_data_ttl,
[("https://example.org/alice", "https://example.org/PersonShape")])
print(result["conforms"])
SHACL result dicts keep the stable keys focus, path, value, severity,
component, source_shape, and message. See SHACL
and ShEx for what the engines cover.
rdflib compatibility
The package ships an rdflib compatibility layer:
from purrdf.compat.rdflib import Graph
For a literal, zero-change import rdflib, there is an opt-in extra:
pip install purrdf[rdflib]
This pulls in the separate purrdf-rdflib distribution, whose top-level
rdflib package re-exports the compat surface, so existing third-party code
doing import rdflib / from rdflib.namespace import RDF transparently runs
on purrdf. Caveat: that shadow claims the rdflib import name and must
never be installed alongside the genuine
rdflib — the two cannot co-inhabit one
environment. It is a separate distribution (never bundled into the main
purrdf wheel) precisely so environments that need the real rdflib simply
omit it.
The compat layer is gated in CI against rdflib 7.6’s own vendored test suite plus a first-party differential parity suite — see rdflib Compatibility for details and the known, ledgered divergences.
GTS relational exports
The Python package also ships GTS relational exports for analytics pipelines:
from purrdf import gts_to_sqlite, gts_to_duckdb, gts_to_parquet
These project a GTS container into SQLite, DuckDB, or Parquet tables.
Next steps
- rdflib Compatibility — the drop-in story in depth.
- Validation — SHACL and ShEx from Python.
- GTS Graph Transport — the container format the exports read.
Getting Started: JavaScript / WebAssembly
The npm package
@blackcatinformatics/purrdf
is the same Rust engine compiled to wasm32 and surfaced through an
RDF/JS-shaped API (DataFactory, DatasetCore,
Stream/Sink). It runs in the browser and in Node, entirely in memory.
npm install @blackcatinformatics/purrdf
Prefer to try it before installing anything? The RDF-1.2 playground runs this exact wasm build in your browser — parse, SPARQL, SHACL, serialize, and canonicalize/compare RDF-1.2 graphs client-side, with no toolchain and no server.
First dataset
Await ready() once before anything else — it performs the one-time async
wasm instantiation:
import { ready, DataFactory, Dataset, QueryEngine } from "@blackcatinformatics/purrdf";
await ready(); // one-time async wasm instantiation
const f = new DataFactory();
const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");
const ds = new Dataset();
ds.add(f.quad(f.namedNode("https://ex/s"), f.namedNode("https://ex/says"), rtl));
const nq = ds.serialize("nquads"); // directions survive the round-trip
const reparsed = Dataset.parse(nq, "nquads");
const engine = new QueryEngine();
const ask = engine.ask(reparsed, "ASK { <https://ex/s> <https://ex/says> ?msg }");
The RDF 1.2 wedge
No incumbent RDF/JS library carries RDF 1.2 quoted-triple terms or
directional literals. PurRDF’s DataFactory exposes both:
// A quoted triple, usable as a subject/object (RDF-star / RDF 1.2).
const quoted = f.quotedTriple(
f.namedNode("https://ex/alice"),
f.namedNode("https://ex/knows"),
f.namedNode("https://ex/bob"),
);
// A base-direction literal (rdf:dirLangString).
const hello = f.directionalLiteral("مرحبا", "ar", "rtl");
API surface
ready(bytesOrUrl?)— await once before anything else.DataFactory—namedNode,blankNode,literal(value, languageOrDatatype?),typedLiteral,directionalLiteral,variable,defaultGraph,quad,quotedTriple,fromTerm,fromQuad.Dataset(RDF/JSDatasetCore) —Dataset.parse(input, format, base?),serialize(format),add/delete/has/match/quads/size, and iteration (for (const quad of dataset)). Formats:turtle,ntriples,nquads,trig,rdfxml(or their media types);serializeadditionally acceptsjsonld.- Graph identity —
Dataset.canonicalize()returns the RDFC-1.0 canonical, flat N-Quads for the graph;Dataset.isomorphic(other)decides RDF graph equality under blank-node relabeling (an exact oracle backed by full RDFC-1.0 canonicalization). - SPARQL —
QueryEnginekeeps the native plan cache alive across calls and exposes typedselect/ask/construct/describe, atomicupdate, andqueryRawserialization.Dataset.query(...)remains the compatibility raw-string helper. - SHACL —
shaclValidateToSarif(shapesTtl, dataNt)validates an N-Triples data graph against a Turtle shapes graph and returns a SARIF 2.1.0 report;shaclEntail(shapesTtl, dataNt)materializes the SHACL-AFsh:ruleinferences as N-Triples. Sink— a streaming consumer (push(quad)/finish() → Dataset);datasetToStream/streamToDatasetare the async RDF/JS Stream/Sink helpers.
More on the RDF/JS mapping in RDF/JS in JavaScript.
Scope and current limitations
- In-memory only. SPARQL queries run over the in-memory dataset;
this package provides no network resolver, so remote
SERVICEandLOADfail explicitly. - A quoted-triple term as a quad object currently round-trips only through N-Quads (a current native serializer limitation for the other formats).
Building from source
The Rust cdylib lives in
crates/rdf-wasm;
the published ESM package is generated from it:
make wasm-pkg # release wasm + wasm-bindgen ESM bindings → js/pkg/
make wasm-pkg-test # the above + TypeScript, Node, and packed-tarball gates
This requires the wasm32-unknown-unknown Rust target and a
wasm-bindgen-cli pinned to the crate’s wasm-bindgen version.
Getting Started: C
libpurrdf is a stable, SemVer-disciplined extern "C" surface over the
native PurRDF stack: parse, serialize, pattern iteration, copy-on-write
mutation, SPARQL, SHACL validation/entailment, and GTS container round-trips.
The committed, reproducible header
include/purrdf.h
is the ABI contract — CI fails if it drifts from the crate.
It is one shared library: libpurrdf statically reuses the purrdf-gts Rust
crate, so a language shim links libpurrdf alone and still reads/writes
.gts containers.
Building
The library, header, and pkg-config file are produced by
cargo-c:
make capi-build # cargo capi build: libpurrdf.{so,a} + purrdf.h + purrdf.pc
make capi-install PREFIX=/usr # cargo capi install into a prefix
make capi-check # verify the committed header is current + run the C smoke
A first program
Adapted from the repository’s C smoke test
(crates/rdf-capi/tests/smoke.c):
#include "purrdf.h"
#include <stdio.h>
#include <string.h>
int main(void) {
const char *doc = "<http://a> <http://b> <http://c> .";
PurrdfDataset *dataset = NULL;
PurrdfError *error = NULL;
int rc = purrdf_parse((const uint8_t *)doc, strlen(doc), "text/turtle",
NULL, NULL, &dataset, &error);
if (rc != PURRDF_STATUS_OK) return 1;
size_t quad_count = 0;
purrdf_dataset_quad_count(dataset, &quad_count);
printf("%zu quad(s)\n", quad_count);
/* Iterate every quad through a pattern cursor. */
PurrdfGraphMatch any;
memset(&any, 0, sizeof(any));
any.kind = PURRDF_GRAPH_MATCH_KIND_ANY;
PurrdfCursor *cursor = NULL;
purrdf_quads_for_pattern(dataset, NULL, NULL, NULL, &any, &cursor, &error);
PurrdfTermView s, p, o, g;
uint8_t has_graph = 0;
while (purrdf_cursor_next(cursor, &s, &p, &o, &g, &has_graph) == PURRDF_STATUS_OK) {
printf("subject=%.*s\n", (int)s.lexical.len, (const char *)s.lexical.ptr);
}
purrdf_cursor_free(cursor);
purrdf_dataset_free(dataset);
return 0;
}
The ABI contract
- No unwinding across the boundary. Every function runs inside
catch_unwind; a caught panic becomesPURRDF_STATUS_PANIC, never a process abort across FFI. int32_tstatus + out-params. Fallible functions return aPurrdfStatusvalue and write results through out-pointers.PURRDF_STATUS_CURSOR_EXHAUSTEDis the (non-error) end-of-rows signal.- SemVer-frozen ABI. The status enum is append-only; new fields and
functions are additive.
purrdf_abi_versionreports the current ABI version (0.1.x, beta).
Ownership and lifetimes
- Every handle/buffer/error/cursor has exactly one matching
*_free(purrdf_dataset_free,purrdf_graph_free,purrdf_cursor_free,purrdf_rowcursor_free,purrdf_buffer_free,purrdf_error_free). FreeingNULLis a no-op. - The C side never
free()s aPurrdfStr.ptr— it borrows library-owned memory; copy the bytes out if they must outlive the borrow. Term views frompurrdf_cursor_nextare valid until the nextpurrdf_cursor_nexton that cursor orpurrdf_cursor_free. - Pattern cursors pin the dataset and pull rows lazily from the selected core index; opening a cursor does not allocate a matching-row snapshot.
PurrdfDatasetis frozen andSend + Sync— readable concurrently from many threads.PurrdfGraph(the copy-on-write mutable delta) and cursors are single-threaded.
Known limitation
The GTS star layer round-trip (purrdf_to_gts → purrdf_from_gts of a
dataset containing quoted triples / reifier bindings) currently fails with
PURRDF_STATUS_GTS_ERROR. This is a pre-existing gap in the kernel path, not
in the C ABI; star-free GTS round-trips are lossless, and a characterization
test pins the current behavior so a kernel fix will flip it.
Full contract details — status codes, term crossing representations,
thread-safety per handle — are in the
purrdf-capi README.
The Interned Dataset IR
Everything in PurRDF evaluates over one intermediate representation: an
immutable, value-interned RDF 1.2 dataset owned by the ring-fenced
purrdf-core kernel.
Terms are interned once
Every term — IRI, blank node, literal, triple term — is stored once in a
string arena and addressed by a copyable TermId (a niche-optimized
NonZeroU32). Quads are rows of four TermIds. That makes term equality a
single integer compare, keeps quads at a fixed small size, and means a term
that appears in a million quads costs its bytes exactly once.
Hot maps use fixed-key ahash — deterministic hashing is part of the
byte-determinism discipline, not just a speed choice.
Builder → freeze
The IR has a strict two-phase life cycle:
use purrdf_core::{RdfDatasetBuilder, RdfLiteral};
// Intern terms once; quads are rows of copyable TermIds.
let mut b = RdfDatasetBuilder::new();
let cat = b.intern_iri("https://example.org/cat");
let says = b.intern_iri("https://example.org/says");
let meow = b.intern_literal(RdfLiteral::simple("meow"));
b.push_quad(cat, says, meow, None);
// Freeze into the immutable, indexed dataset the engines evaluate over.
let ds = b.freeze().expect("well-formed dataset");
assert_eq!(ds.quad_count(), 1);
RdfDatasetBuilder is the mutable ingestion phase: intern terms, push quads,
attach reifiers and annotations. freeze() validates the structure and
produces an immutable RdfDataset: quad rows in Box<[QuadRow]> tables with
lazy ordinal permutation indexes (roughly 4 bytes per quad per axis). The
frozen dataset is what SPARQL, SHACL, ShEx, and entailment all read, through
the allocation-free DatasetView trait.
Freezing is also what makes concurrency simple: a frozen dataset is immutable,
so it can be shared and read from many threads (the C ABI exposes exactly this
as a Send + Sync handle).
Copy-on-write mutation
“Immutable” does not mean “static”. Mutation happens through a copy-on-write
delta over a frozen base: edits accumulate in a lightweight overlay, and the
result freezes into a new dataset without copying the untouched base rows or
re-interning shared terms. SPARQL UPDATE and the C ABI’s mutable
PurrdfGraph handle both ride this path.
What else lives in the kernel
Beyond the IR itself, purrdf-core owns:
DatasetView— the static read trait every engine evaluates over.- Structured diagnostics — typed
RdfDiagnostics with source locations (deliberately SARIF-free; the SARIF boundary ispurrdf-validate). - RDFC-1.0 canonicalization, dataset diff, and isomorphism — see Canonicalization & Diff.
- Store and engine seams — the narrow parser-ingress, serializer-egress,
and
SparqlEnginetraits that adapters implement in sibling crates. - Provenance and the loss ledger — a generic provenance sidecar and the machine-readable RDF↔GTS loss matrix, plus native FnO and SSSOM codecs (see Slices, Mappings & Provenance).
Text codecs are not in the kernel — parsing and serialization live one layer
up in purrdf-rdf. The split keeps the kernel
small and its invariants enforceable at the crate boundary: no oxigraph, no
PyO3 (a hygiene gate asserts the dependency tree), wasm32-clean, and a
file-IO-free IR layer.
Why this design
The layout is chosen by measurement, not assertion: the criterion bench
crates/rdf-core/benches/ir_layout.rs compares array-of-structs,
struct-of-arrays, and predicate-adjacency layouts on allocation counts,
high-water memory, and end-to-end latency — the shipped layout is whichever
wins. See Performance.
RDF 1.2 Features
PurRDF is RDF 1.2-first: the features RDF 1.2 Concepts adds over RDF 1.1 are part of the core data model, carried through the IR, the codecs, SPARQL, validation, the language bindings, and the GTS transport.
Triple terms
RDF 1.2 lets a triple itself be a term in object position — the
“quoted triple” of RDF-star, written <<( s p o )>> in SPARQL 1.2 syntax.
In the IR a triple term is interned like any other term and gets a TermId,
so it composes with everything else (patterns, results, serialization).
- The star-capable codecs (Turtle, TriG, N-Triples, N-Quads, JSON-LD star) round-trip triple terms; see Codecs & Determinism for what happens on a star-incapable projection.
- SPARQL 1.2 quoted-triple syntax is parsed by
purrdf-sparql-algebraand evaluated natively — the W3C SPARQL 1.2 triple-term surface passes in the conformance harness (Conformance & Testing). - In JavaScript,
DataFactory.quotedTriple(...)produces the same term (RDF/JS in JavaScript).
Reifiers and annotations
RDF 1.2 replaces old-style reification with reifiers: terms that name an
occurrence of a triple (rdf:reifies), so you can attach metadata to a
statement without asserting anything odd. In PurRDF, reifier bindings and
annotations live in dedicated side-tables on the dataset rather than being
smeared into the quad table.
Reifier bindings and annotations survive every star-capable codec round-trip;
projections into star-incapable formats drop them loudly, with the realized
count handed to the loss ledger (see
Slices, Mappings & Provenance). SHACL support for validating
reified statements — the draft sh:reifierShape / sh:reificationRequired
surface — is covered in SHACL.
Base-direction literals
RDF 1.2 adds rdf:dirLangString: a language-tagged literal that also carries
a base direction (ltr or rtl) for correct bidirectional-text handling.
These are first-class in the IR and in every binding:
const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");
Directions survive serialization round-trips through the star-capable codecs — the JavaScript quickstart demonstrates the N-Quads round-trip.
RDF 1.2 is a complete target, not a draft excuse
PurRDF treats the RDF 1.2 / SPARQL 1.2 specifications as a complete,
implementable target. Where a feature is scoped (for example, the SHACL 1.2
reifier-shape support is a scoped Working Draft feature, not full SHACL 1.2
conformance), the scope is stated explicitly and gated by tests — never left
as a silent partial implementation. The live per-feature status is the
conformance matrix in
docs/CONFORMANCE.md.
Where each feature shows up
| Feature | IR | Codecs | SPARQL | SHACL | RDF/JS | GTS |
|---|---|---|---|---|---|---|
| Triple terms (object position) | interned term | star-capable formats | <<( s p o )>> | via paths/values | quotedTriple | mapped per spec |
| Reifiers / annotations | side-tables | star-capable formats | reifier surface | sh:reifierShape (draft) | — | rdf:reifies mapping |
| Base-direction literals | literal kind | round-trips | matched/produced | value nodes | directionalLiteral | carried |
The GTS mapping of triple terms and rdf:reifies is formalized in the
GTS specification,
which pins its RDF 1.2 substrate to the 07 April 2026 W3C Candidate
Recommendation Snapshot.
RDF 1.2 Visualization
PurRDF projects RDF into a renderer-neutral statement model before producing a layout or SVG. The projection keeps structural triple terms, assertion occurrences, reifier identity, annotations, graph context, nesting, and RDF dialect diagnostics distinct. The SVG carries that model as embedded JSON metadata, so it is both a visual document and a lossless machine-readable export.
The examples below are generated directly by the Rust visualization surface. Line colour varies subtly within each semantic relation class to make dense routes easier to follow; line width, dash pattern, labels, and arrow grammar continue to carry the RDF meaning without relying on colour.
Ordinary shared resources
The compact view preserves the familiar RDF resource graph while separating routes that share nodes.
Asserted, reified, and annotated statements
Solid arrows remain assertions. Addressable statement anchors connect those assertions to reifier resources, whose ordinary RDF properties carry provenance, confidence, timestamps, and directional language literals.
Quoted-only and nested triple terms
A quoted-only triple is a bounded statement glyph, never a solid assertion arrow. Reifier resources stay distinct from the structural statement they reify.
Nested triple terms retain recursive statement identity. Dialect badges make symmetric or generalized RDF positions explicit rather than silently rendering them as ordinary RDF 1.2.
Dense connected data
The exact view uses separated vertical channels and rounded orthogonal turns so individual subject, predicate, object, reification, and annotation routes remain traceable through a dense connected dataset.
Regenerating the samples
The committed SVGs are projections of the Rust fixtures, not hand-edited book artwork:
make book-samples
make book
make check regenerates the same artifacts in a temporary directory and rejects
any drift between the renderer and the book.
Codecs & Determinism
PurRDF ships first-party parsers and serializers — no wrapped third-party codec — for seven formats:
| Format | Media type | Star-capable |
|---|---|---|
| Turtle | text/turtle | yes |
| TriG | application/trig | yes |
| N-Triples | application/n-triples | yes |
| N-Quads | application/n-quads | yes |
| RDF/XML | application/rdf+xml | no |
| JSON-LD (star) | application/ld+json | yes |
| YAML-LD | application/ld+yaml | yes |
They live in purrdf-rdf, one layer above the
kernel, and are reachable through the umbrella crate:
use purrdf::{parse_dataset, serialize_dataset, SerializeGraph};
let turtle = br#"
@prefix ex: <https://example.org/> .
ex:cat ex:says "meow" .
"#;
// Parse into the frozen, value-interned RDF 1.2 dataset IR.
let ds = parse_dataset(turtle, "text/turtle", None).expect("valid Turtle");
assert_eq!(ds.quad_count(), 1);
// Serialize back out through any native codec — byte-deterministic output.
let nq = serialize_dataset(&ds, "application/n-quads", SerializeGraph::Dataset)
.expect("serializes");
Open Knowledge Format bundles
The native OKF codec maps caller-profiled RDF 1.2 datasets to agent-facing Markdown files with YAML frontmatter and lifts them back through the RDF event seam. OKF is an in-memory bundle API rather than another media type: callers choose how to store the files, so the same code remains deterministic and wasm-clean.
OkfConfig::new requires the vocabulary namespace, document base IRI, and
recognized frontmatter keys. There is no built-in ontology or namespace. Use
lift_okf_bundle to drive an RdfEventSink, or write_okf_bundle (backed by
OkfWriter, an RdfDatasetVisitor) to project a frozen dataset. Both directions
always return a loss ledger. A lossless profile yields an empty ledger; named
graphs, non-profile/OWL rows, and unrelated reifier or annotation rows are
pinpointed explicitly when writing.
Byte determinism
Every serializer is byte-deterministic: the same dataset always produces
the same bytes, on every platform and in every language binding. This is a
hard workspace invariant, not a best effort — no iteration-order, time, or RNG
dependence is allowed in any output path (hashers are fixed-key ahash for
exactly this reason), and golden-file tests pin the emitted bytes.
Determinism is what makes the rest of the toolkit composable: content addressing in GTS and the slice catalog, diffable serializations in review, and cross-language conformance vectors that can be compared byte-for-byte.
Diagnostics, not partial parses
Malformed input is a typed RdfDiagnostic with a source location where the
codec can provide one — never a silent partial parse. Parsing can optionally
record a source-position span table for richer diagnostics. Diagnostics stay
structured (SARIF-free) in the core; render them as byte-deterministic SARIF
2.1.0 for editors and CI with
purrdf-validate (see
SHACL).
Lossy projections are loud
RDF 1.2 statement-level data (triple terms, reifier bindings, annotations)
survives every star-capable round-trip. Serializing into a star-incapable
projection drops that layer loudly: the realized drop count is handed to the
machine-readable loss ledger
(generated/rdf-loss-matrix.json)
rather than disappearing. The same discipline applies at the SPARQL results
boundary (Result Formats) and the RDF↔GTS boundary.
The succinct pack codec
Alongside the text codecs above, purrdf-core ships a binary codec for a
different job: a read-only, query-the-compressed-form encoding of a whole
dataset for large-scale reference bundles, not an interchange format with a
media type. PackBuilder::build_bytes(&dataset) writes a self-contained,
byte-deterministic pack — a value dictionary, graph-partitioned succinct
bitmap-triples, and RDF 1.2 side-tables (reifier bindings, statement
annotations) — into one Vec<u8>. PackView::from_bytes(&[u8]) opens it
zero-copy over a borrowed slice and answers pattern queries directly against
the packed bytes, with no decompression or materialization step first.
Reach for a pack when a dataset is done changing and needs to be distributed,
archived, or served at a scale where re-parsing text on every load is too
slow: RDF 1.2 (named graphs, quoted triples, reifiers, annotations) is fully
supported, and verify_pack independently recomputes the dataset’s RDFC-1.0
digest from the pack’s own decoded contents — a certified read-only
projection, not merely a compressed file. The library never memory-maps a
pack itself (every published crate stays wasm32-unknown-unknown-clean); a
native consumer that wants a durable, larger-than-heap tier mmaps the file
and hands PackView::from_bytes the resulting borrowed slice. See the
“Pack backend” section of
the backend contract
for the full contract.
The columnar Parquet codec
purrdf::columnar exposes the bidirectional SQL/DataFrame interchange path.
It maps any DatasetView plus a content-addressed blob store to five standard
Parquet files (terms, quads, reifiers, annotations, and blobs) and
reads that exact profile back without Arrow or a general Parquet runtime. The
mapping retains RDF 1.2 triple terms, reifiers, annotations, graph scope,
directional literals, blank-node scope, and explicitly empty named graphs.
The files are byte-deterministic and readable by engines such as DuckDB. See the normative columnar schema for every field and the deliberately narrow Parquet profile.
Conformance
The codecs are gated by the W3C rdf-tests syntax corpus, vendored and frozen
in-repo — 250/250 round-trip cases across N-Quads, N-Triples, RDF/XML, TriG,
and Turtle at the time of writing. The live scoreboard is
docs/CONFORMANCE.md.
Related
- Canonicalization & Diff — when you need a canonical serialization rather than just a deterministic one.
- The Interned Dataset IR — what the text codecs parse
into, and the
DatasetViewread seam the pack codec implements alongsideRdfDataset.
Canonicalization & Diff
Byte-deterministic serialization (Codecs & Determinism) means the same dataset always emits the same bytes. Canonicalization is the stronger property: two different in-memory datasets that are isomorphic — the same graph up to blank-node relabeling — canonicalize to the same bytes.
RDFC-1.0
PurRDF implements W3C
RDF Dataset Canonicalization (RDFC-1.0)
natively in the kernel, tested against the W3C rdf-canon fixture suite
(65 vectors — 64 eval plus 1 negative — all green; see
docs/CONFORMANCE.md).
The entry point is canonicalize (with a canonicalize_with variant for
choosing the hash), producing canonical blank-node labels and, one layer up in
purrdf-rdf, canonical flat N-Quads over the frozen IR:
use purrdf::canonicalize;
let canon = canonicalize(&ds);
// Canonical labels are stable across runs, hosts, and language bindings.
Use canonicalization when you need a content identity for a graph: hashing, signing, deduplication, or comparing datasets produced by different writers.
Isomorphism
datasets_isomorphic(a, b) decides whether two frozen datasets are
RDF-structurally isomorphic: the same quads under a blank-node bijection.
Canonicalization gives the equivalent verdict — two datasets are isomorphic
iff their canonicalizations are equal — but the direct check is the
convenient form for tests and harnesses. PurRDF’s own conformance harnesses
use RDFC-1.0 isomorphism to compare, for example, SHACL Rules output graphs
against expected inferred graphs.
Diff
dataset_diff(a, b) produces a structural diff between two frozen datasets,
including an isomorphic verdict. For a human-facing review flow,
purrdf-rdf additionally provides per-subject Symmetric-CBD extraction
(“describe”) and a review-friendly Turtle normalizer, so a graph change reads
like a code change.
Choosing the right tool
| Need | Use |
|---|---|
| Same dataset → same bytes | any native serializer (always true) |
| Same graph (up to blank nodes) → same bytes | RDFC-1.0 canonicalize |
| “Are these two datasets the same graph?” | datasets_isomorphic |
| “What changed between these datasets?” | dataset_diff + describe/normalize |
| Content-addressed transport of a graph | GTS (BLAKE3 content ids) |
API details are on
docs.rs/purrdf-core (the canonicalize,
datasets_isomorphic, and dataset_diff items) and
docs.rs/purrdf-rdf (describe and normalization).
SPARQL: Querying
PurRDF’s SPARQL stack is native and three-layered, gated by the W3C SPARQL 1.1 and 1.2 conformance suites:
purrdf-sparql-algebra— parses query and update text into a PurRDF-owned, RDF 1.2-native query algebra (Query/GraphPattern,Update/GraphUpdateOperation). Parse and algebra only.purrdf-sparql-eval— the multiset evaluator over the frozen IR’sDatasetView, entirely in internedTermIdspace.purrdf-sparql-results— the results boundary (next chapter).
All three are re-exported under purrdf::sparql.
A first query
use purrdf::{RdfDatasetBuilder, RdfLiteral, SparqlEngine, SparqlRequest, SparqlResult};
use purrdf::sparql::NativeSparqlEngine;
// A tiny dataset in interned TermId space.
let mut b = RdfDatasetBuilder::new();
let cat = b.intern_iri("https://example.org/cat");
let says = b.intern_iri("https://example.org/says");
let meow = b.intern_literal(RdfLiteral::simple("meow"));
b.push_quad(cat, says, meow, None);
let ds = b.freeze().expect("freeze");
// Evaluate through the SparqlEngine seam; parsed plans are memoized.
let engine = NativeSparqlEngine::new();
let result = engine.query(&ds, SparqlRequest {
query: "SELECT ?what WHERE { <https://example.org/cat> <https://example.org/says> ?what }",
base_iri: None,
substitutions: &[],
}).expect("evaluates");
if let SparqlResult::Solutions { rows, .. } = result {
assert_eq!(rows.len(), 1);
}
The SparqlEngine trait itself lives in purrdf-core, so hosts can swap
engines behind one seam; NativeSparqlEngine is the shipped implementation.
What the front-end covers
- Query — all four query forms (SELECT/ASK/CONSTRUCT/DESCRIBE), basic
graph patterns,
OPTIONAL,UNION,MINUS,GRAPH,FILTER/BIND/VALUES, property paths,GROUP BY/aggregates,EXISTS/NOT EXISTS, solution modifiers, and RDF 1.2 quoted triple terms (<<( s p o )>>). - Update —
INSERT DATA/DELETE DATA, theDELETE/INSERT … WHEREfamily (WITH/USING,DELETE WHERE),LOAD, andCLEAR/DROP/CREATE/ADD/MOVE/COPY.
Anything outside this surface — and every malformed query — is a typed
ParseError, never a silently degraded parse.
How the evaluator works
- Multiset (bag) semantics — solutions are a bag, preserved until
DISTINCT/REDUCED, per the SPARQL algebra. - Interned evaluation — constants resolve to a dataset
TermIdonce; solution comparison is an integer compare; computed FILTER/BIND values that already exist in the dataset are promoted to the interned id at mint time. - Property paths in-engine — the full path algebra
(
* + ? / | ^ !()) evaluated over the same indexed surface, wasm-safe. - Cost-based BGP planning — join order is chosen by a cost model;
NativeSparqlEngine::explain_queryexposes the chosen order as an ordered list of triple-pattern strings so you can audit planner decisions without running the query. - EXISTS decorrelation — correlated
EXISTS/NOT EXISTSfilters are decorrelated rather than re-evaluated per row. - The SERVICE seam —
SERVICEfederation is evaluated through a host-injectable transport: the engine itself performs no I/O, so federation stays wasm-portable and the host decides how (and whether) remote endpoints are reached. All seven W3Cservicefederation cases pass through this seam. - Hard-fail — an out-of-scope algebra node or unimplemented builtin is a
typed
EvalError::Unsupported, never a partial or wrong answer.
Entailment regimes
SPARQL queries can be answered under an entailment regime by materializing the
dataset first with purrdf-entail — Regime::from_iri
maps a sparql:entailmentRegime IRI to the matching engine.
Conformance
The full W3C SPARQL 1.1 query + update evaluation suites plus the SPARQL 1.2
suite are vendored verbatim and run by purrdf-sparql-conformance; every
non-pass is a typed, ledgered expected-failure. See
Conformance & Testing and
docs/CONFORMANCE.md
for the live matrix.
SPARQL: Result Formats
purrdf-sparql-results is the results
boundary of the SPARQL stack: the canonical authority for turning a
SparqlResult (SELECT solutions, ASK boolean, or CONSTRUCT graph) into the
four W3C SPARQL Results formats — JSON (SRJ), XML, CSV, and TSV — plus an
additive, provenance-carrying PurRDF extension where the format can carry one.
JSON and XML documents can also be read back (from_json, from_xml).
use purrdf::sparql::{serialize, ResultProvenance, SparqlResultsFormat};
// `result` is the SparqlResult produced by purrdf-sparql-eval (or any engine
// implementing the purrdf-core SparqlEngine seam).
let outcome = serialize(&result, SparqlResultsFormat::Json, &ResultProvenance::default())
.expect("SELECT serializes to SRJ");
assert!(!outcome.provenance_dropped);
let json = String::from_utf8(outcome.bytes).unwrap();
Per-format writers (to_json, to_xml, to_csv, to_tsv) and readers
(from_json, from_json_boolean, from_xml, from_xml_boolean) are also
exported directly.
Behavior worth knowing before you pick a format
- Byte-deterministic output — the same result always serializes to the same bytes, like every other PurRDF output path (Codecs & Determinism).
- The support matrix is enforced, not fudged — XML rejects CONSTRUCT
graphs, and CSV/TSV reject both ASK booleans and CONSTRUCT graphs, each as
a typed
Error::Format, rather than emitting something spec-shaped but wrong. - Lossy projections are flagged — CSV/TSV have no extension point, so a
populated provenance is trimmed at the exit gate and
SerializeOutcome::provenance_droppedis set; the drop is never silent.
| Format | SELECT | ASK | CONSTRUCT | Provenance extension |
|---|---|---|---|---|
| JSON (SRJ) | yes | yes | yes | yes |
| XML | yes | yes | rejected | yes |
| CSV | yes | rejected | rejected | dropped, flagged |
| TSV | yes | rejected | rejected | dropped, flagged |
The provenance extension
The PurRDF extension is additive: a standard SPARQL results consumer can read the JSON/XML documents unchanged, while a PurRDF-aware consumer can recover per-result provenance carried alongside the bindings. Where the format has no extension point (CSV/TSV), the provenance is dropped loudly, per the loss discipline described in Slices, Mappings & Provenance.
One term-syntax authority
The crate depends only on purrdf-core and stays wasm-clean; term and
N-Triples syntax come exclusively from the kernel’s emit primitives, so there
is exactly one term-syntax authority in the workspace — results, codecs, and
diagnostics can never disagree about how a term is written.
Related
- SPARQL: Querying — producing the
SparqlResultin the first place. - docs.rs/purrdf-sparql-results — the full API reference.
SHACL
purrdf-shapes (re-exported as
purrdf::shapes) is PurRDF’s native SHACL validator: the complete SHACL
Core feature set — all constraint components, full property paths,
qualified value shapes, property pairs — plus SHACL-SPARQL constraints
and targets and the SHACL-AF surface, running entirely on PurRDF’s own
interned IR and native SPARQL engine (no oxigraph, no PyO3).
It validates an RDF 1.2 data graph against a SHACL shapes graph with no
inference (parity with pySHACL inference="none"); combine with
Entailment if you want to validate a materialized closure.
What it covers
- SHACL Core — every constraint component, full property paths, qualified
value shapes, property pairs. The W3C
data-shapessuite passes clean (126/126, zero ledgered gaps at the time of writing — the live number is indocs/CONFORMANCE.md). - SHACL-SPARQL — SPARQL-based constraints and targets, custom constraint
components with pre-binding semantics, user-defined
sh:SPARQLFunctioncalls, andsh:SPARQLTargetType, evaluated on the native SPARQL engine. - SHACL-AF — node expressions (including
sh:ExpressionConstraintComponent) and SHACL Rules (sh:TripleRuleandsh:SPARQLRule, withsh:condition,sh:order,sh:deactivated): rules fire in an iterative fixpoint and the derivation is materialized as a new dataset (base ⊎ derived), leaving the input graph untouched. Some node-expression conveniences (sh:if, aggregations, ordering wrappers) are DASH/TopBraid conventions with no normative RDF definition; PurRDF documents its adopted reading and pins it with a frozen corpus — see the SHACL-AF section of docs/CONFORMANCE.md.
The SHACL 1.2 reifier-shape draft scope
The crate implements a scoped SHACL 1.2 Working Draft feature:
sh:reifierShape and sh:reificationRequired for direct IRI property paths,
so shapes can constrain the RDF 1.2 reifier metadata attached to statements
(see RDF 1.2 Features). The relevant SHACL 1.2 Core
draft is dated 2026-06-02. This is not a claim of full SHACL 1.2
conformance — it is one draft feature, explicitly scoped and tested.
Pydantic v2 projection
purrdf-shapes can transliterate a compiled SHACL-derived JSON Schema into a
deterministic, typed Pydantic v2 package entirely in memory. The public
emit_pydantic function consumes CompiledSchema; PydanticConfig requires the
caller to supply the package name and package/module prose, so the library does
not invent a vocabulary, namespace, or downstream brand.
Every $defs entry gets a stable import path, JSON property names remain exact
through Pydantic aliases, and generated classes expose the originating
definition through model_json_schema(by_alias=True). Pydantic runtime
annotations enforce the representable portion. A JSON Schema assertion with no
exact runtime annotation remains visible on that schema surface and produces a
located entry in the always-computed json-schema → pydantic-v2
LossLedger; a lossless input yields an empty ledger. The renderer itself has no
Python dependency and stays wasm-clean. A dev-only Python oracle executes the
generated code and checks the live reverse/schema surface.
LinkML 1.11 projection
The same CompiledSchema carrier can be projected to canonical LinkML 1.11
with emit_linkml. LinkmlConfig requires the caller’s schema IRI, name,
description, default prefix, and complete prefix map, so PurRDF never mints a
consumer vocabulary or identity. The returned LinkmlPackage includes the
typed document, deterministic YAML, a reversible $defs-key mapping, and a
located json-schema → linkml-1.11 loss ledger.
Classes and exact property aliases, types, enums, local references, inline
objects, requiredness, homogeneous arrays, patterns, inclusive bounds, and
LinkML boolean expressions are represented directly. Every unsupported
assertion is classified by a closed capability table; malformed inputs,
external/dynamic/dangling references, prefix mistakes, and deterministic-name
collisions fail closed. parse_linkml and write_linkml preserve all
JSON-compatible metamodel fields and provide byte-stable read/write round trips
while rejecting YAML-only tags, duplicate keys, non-string keys, and non-finite
numbers.
The Rust production path has no LinkML-toolkit dependency. CI uses the locked official LinkML 1.11.1 Python packages only as a differential oracle:
make linkml-oracle
TypeScript 7.0 projection
emit_typescript projects the same CompiledSchema into deterministic
TypeScript 7.0 declarations. The caller supplies the package name and all
package/module prose through TypeScriptConfig. The returned package contains
one index.d.ts, a reversible $defs-key to exported-type map, and a located
json-schema → typescript-7.0 loss ledger; PurRDF invents no consumer
identity or vocabulary.
The fixed declaration dialect uses strict plus
exactOptionalPropertyTypes. Type aliases preserve JSON primitives and
literals, required versus optional fields, explicit null, local recursive
references, unions, intersections, homogeneous arrays, and bounded tuples.
There are no runtime enums, mergeable interfaces, branded pseudo-validators,
or any escape hatches. Invalid keywords, open/dangling references, and name
collisions fail before bytes are emitted.
Runtime assertions outside TypeScript structural assignability are never silently erased: integer, numeric/string predicate, closure, pattern-property, dependency, conditional, negation, contains/unique, evaluation-state, and bounded-expansion gaps receive stable codes and JSON Pointer locations. CI classifies instances independently with a draft 2020-12 validator and compiles the generated declarations with the locked TypeScript 7.0.2 compiler, including fresh-literal and through-variable probes:
make typescript-oracle
The projection intentionally has no arbitrary TypeScript reader. TypeScript
declarations do not define a unique runtime JSON acceptance relation, and the
projection is many-to-one. The retained CompiledSchema plus the reversible
name map remains the authoritative reverse surface. TypeScript is only a
dev-time oracle dependency; the Rust emitter is filesystem-free and wasm-clean.
GraphQL September 2025 projection
emit_graphql projects CompiledSchema into deterministic GraphQL September
2025 SDL. GraphqlConfig has no defaults: the caller supplies the schema name,
package and module prose, and a non-built-in fallback-scalar name. The returned
GraphqlPackage contains schema.graphql, canonical name-map.json, the same
name map as typed Rust data, a located json-schema →
graphql-september-2025 loss ledger, and the production value codec.
The SDL is deliberately a type-system fragment. PurRDF emits paired output
type and input input objects, but no query, mutation, or subscription root,
resolver, pagination rule, authorization policy, federation directive, or
other application behavior. A caller composes the fragment with its own
executable schema.
The exact grammar includes GraphQL booleans, strings, numbers, the signed
32-bit Int domain, explicit nullability, finite JSON const/enum sets,
closed object fields, requiredness, homogeneous lists, direct local $defs
references and aliases, descriptions, and inline object helpers. One global
collision-checked namespace covers types, helpers, and the fallback scalar;
fields and enum symbols are checked in their GraphQL-local namespaces. The
typed/canonical name maps retain the source definition keys, property keys, and
finite JSON values.
GraphqlPackage::encode_input maps source JSON keys and finite values to input
field names and enum symbols. decode_output performs the inverse for fields
present in a GraphQL response, without inventing omitted selections. Unknown or
incompatible values fail. This package codec is the precise reverse boundary;
arbitrary GraphQL SDL has no unique JSON Schema acceptance relation and is not
accepted as an inverse format.
GraphQL variable coercion differs from JSON Schema validation at these closed boundaries:
| Boundary | Located loss families |
|---|---|
| object fields and names | additional properties, pattern properties, property names/counts |
| requiredness and recursion | nullable-presence widening, one deterministic recursive-input nullability relaxation |
| lists | singleton coercion, cardinality, contains, uniqueness, tuples, unevaluated items |
| scalar assertions | integer domain delegation, numeric predicates, string predicates |
| applicators | conditionals, dependencies, intersections, unions, oneOf, negation |
| runtime boundary | custom-scalar and unknown-keyword validation delegation |
The caller-named fallback scalar is declared but PurRDF does not invent its
parseValue, parseLiteral, or serialization semantics. Every delegated use
is therefore ledgered. Loss entries carry stable codes and source JSON Pointer
locations; an exact package has an empty ledger.
Emission fails before returning bytes for invalid caller configuration,
malformed schema keywords, $id rebasing, external/indirect/dangling $ref,
$dynamicRef/$recursiveRef, alias cycles, unsatisfiable closed required
fields, and generated-name collisions. The fixed limits are 16 MiB for the
input schema, each artifact, and one codec value; 65,536 definitions, fields
per object, or finite values; depth 128; and 255 bytes per GraphQL name.
The independent dev oracle classifies source values with boon, builds the
SDL with locked official GraphQL.js 16.14.0, and executes real variable
coercion. It verifies exact agreement, every closed loss family and location,
the name map and production codec, and deliberate corruption failures:
make graphql-oracle
GraphQL.js is dev-only. Emission and value translation remain filesystem-free, wasm-clean Rust.
From Python
from purrdf_native import shacl
report = shacl.validate(shapes_ttl="...", data_nt="...")
print(report["conforms"]) # True / False
print(report["results"]) # list of violation dicts
Each result dict keeps the stable keys focus, path, value, severity,
component, source_shape, and message.
SARIF output
Validation reports stay structured in the engine; the SARIF 2.1.0 boundary is
the separate purrdf-validate crate
(purrdf::validate), which renders a report — or parser diagnostics — as a
source-traced, byte-deterministic SARIF log for editors, CI, and
code-scanning dashboards:
use purrdf::validate::{validate_to_sarif_string, SarifOptions};
let shapes = r#"
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix ex: <http://example.org/> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
ex:PersonShape a sh:NodeShape ;
sh:targetClass ex:Person ;
sh:property [ sh:path ex:age ; sh:datatype xsd:integer ] .
"#;
let data = r#"<http://example.org/alice> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/Person> .
<http://example.org/alice> <http://example.org/age> "nope" .
"#;
let sarif = validate_to_sarif_string(shapes, data, &SarifOptions::default())
.expect("sarif produced");
assert!(sarif.contains("\"version\": \"2.1.0\""));
Lower-level entry points (build_report_sarif, build_diagnostics_sarif)
build a SarifLog value instead of a string, so a host can merge runs before
serializing.
Conformance
The validator is gated by the vendored W3C data-shapes suite, a vendored
DASH SHACL-AF/rules corpus, and a first-party frozen corpus of 69 cases with
byte-frozen expected reports; SHACL Rules output is compared to expected
inferred graphs by RDFC-1.0 isomorphism. See
Conformance & Testing.
ShEx
purrdf-shex (re-exported as purrdf::shex)
is PurRDF’s native
Shape Expressions Language 2.1 engine: the
schema layer and the shape-map validator, pure Rust and wasm-clean.
Schemas: ShExC and ShExJ
- ShExC (the compact syntax, spec §6) — a hand-rolled lexer and
recursive-descent parser covering the full grammar: directives,
start, theAND/OR/NOTshape algebra, node constraints and the facet table, value sets with stems/ranges/exclusions, triple expressions with all cardinality forms,$/&labels and inclusions,^inverse, annotations and%…{ … %}semantic actions, with relative-IRI resolution againstBASEviapurrdf-iri. - ShExJ (the JSON wire format, spec Appendix A) — strict, round-tripping serde support matching the shexTest ground truth.
- Structural checks (spec §5.7) — dangling references, label collisions, reference-only cycles, and the negation-stratification requirement.
use purrdf::shex::{check_structure, parse_shexc, to_shexj};
let schema = parse_shexc(
"PREFIX ex: <http://example.org/>\n\
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>\n\
ex:S { ex:p xsd:integer? }",
None,
)?;
check_structure(&schema).expect("well-formed");
let json = to_shexj(&schema);
Hard-fail discipline: every malformed schema is a typed ShexError — no
lenient mode, no panics on any input.
Validation with shape maps
Validation (spec §5.2–§5.5) is fixed shape-map validation over the frozen
purrdf-core dataset IR, in interned TermId space: you supply
(node, shape) associations and the validator decides conformance for each.
It covers node constraints (node kind, datatype with lexical-validity
checking, string/numeric facets, value sets with stems and exclusions),
EXTRA/CLOSED triple-expression matching with EachOf/OneOf
partitioning and group cardinalities, inverse constraints, typing-based
recursion, and an EXTERNAL resolver hook.
From Python:
from purrdf_native import shex
result = shex.validate(my_schema_shexc, my_data_ttl,
[("https://example.org/alice", "https://example.org/PersonShape")])
print(result["conforms"])
Conformance
The engine is gated against the vendored official
shexTest suite, pinned at tag
v2.1.0 (vectors/shexTest/): the full validation/ manifest, the
schemas/ ShExC/ShExJ pairs and round-trips, and the negative syntax and
negative structure suites. At the time of writing the validation manifest
passes 1,105/1,105 attempted cases with zero expected-failures and an
empty trait-skip list (imports and semantic actions included); the live
scoreboard is
docs/CONFORMANCE.md.
Reported conformance is logic-level (pass/fail parity), per suite convention; result-structure conformance is upstream-experimental.
SHACL or ShEx?
PurRDF implements both natively over the same IR, so the choice is yours, not the toolkit’s: SHACL suits constraint reporting (violations with severities, SARIF output) and SHACL-SPARQL escape hatches; ShEx suits schema-like conformance decisions over explicit shape maps.
Entailment
purrdf-entail (re-exported as
purrdf::entail) is native, wasm32-clean entailment for the PurRDF
RdfDataset IR. A family of engines sits behind one facade, each the right
tool for its SPARQL entailment regime — closing a dataset to its inferred
fixpoint entirely in interned TermId space, with no external reasoner,
no async runtime, and no string round-trip.
Surface map
| Entry point | Regime(s) | Engine |
|---|---|---|
materialize(ds, regime) | Simple, RDF, RDFS, OWL-RL | Forward materialization (“chase”) over a fixed rule set via a native semi-naive fixpoint. |
materialize_dl(...) | OWL-Direct | Open-world OWL DL over an ALCOIQ tableau — it needs the query’s class expressions, so it is not reachable through the plain materialize facade. |
materialize_rif(...) | RIF | RIF-Core rule entailment over a parsed RuleSet. |
Regime::from_iri(iri) | — | Parse a sparql:entailmentRegime IRI to its enum. |
use purrdf::entail::{materialize, Regime};
// Close a frozen dataset to its RDFS fixpoint; the result is a new dataset.
let closed = materialize(&ds, Regime::Rdfs).expect("materializes");
The chase (Simple / RDF / RDFS / OWL-RL)
materialize runs a forward-materialization chase: a fixed rule set for the
selected regime, applied by a semi-naive fixpoint until no new quads appear.
Because it runs over the frozen IR, it is deterministic — a given input and
regime always yields the same closure — and because it works in TermId
space, no term is ever re-parsed or re-serialized along the way.
Typical use: materialize first, then query with the plain SPARQL engine or validate the closure with SHACL (the SHACL validator itself performs no inference).
OWL-Direct: the tableau
OWL-Direct semantics is open-world Description Logic, which a forward chase
cannot answer. materialize_dl runs an ALCOIQ tableau instead — answering
instance and subsumption queries via classification, realization, and
query-directed materialization. Because it needs the query’s class
expressions, it has its own entry point rather than hiding behind
materialize.
RIF
materialize_rif evaluates RIF-Core rules over a parsed RuleSet,
covering the SPARQL RIF entailment regime.
The D-entailment boundary
D (datatype) entailment is a typed, spec-inherent boundary: requesting it
returns EntailError::Unsupported rather than silently defaulting to a
weaker regime. (No case in the vendored W3C corpus exercises D-entailment
alone.) This is the workspace-wide hard-fail discipline: an unsupported
regime is an error you can handle, never a wrong answer.
Invariants
- No minted vocabulary. Every constant in the crate’s
vocabmodule is a standardrdf:/rdfs:/owl:IRI drawn from the entailment specs themselves — the crate fabricates none, per the toolkit-not-ontology rule. - Dependency-lean. The only dependency is
purrdf-core, so the engines carry into Rust, WebAssembly, and C unchanged. - Deterministic. Same input + regime → same closure, always.
Conformance
All 70 W3C entailment cases pass at the time of writing — RDF/RDFS/OWL-RL
chase, OWL-Direct (DL) tableau, RIF-rule, and RDF-axiomatic predicate typing —
run through the SPARQL conformance harness. The live scoreboard is
docs/CONFORMANCE.md.
GTS Graph Transport
GTS (Graph Transport Substrate) is an ontology-independent binary
container and transport format for RDF 1.2 datasets and content-addressed
binary payloads. PurRDF hosts the reference Rust engine,
purrdf-gts (re-exported as purrdf::gts).
This chapter is a user-level tour. The wire format itself is specified in
docs/GTS-SPEC.md
— consult the spec for framing, fold semantics, registries, and conformance
classes; nothing here supersedes it.
The container model, at a high level
A GTS file is a CBOR Sequence of one or more append-only segments. Each segment is a deterministic CBOR header followed by deterministic CBOR frames chained by BLAKE3 content identifiers. The logical dataset is obtained by a deterministic fold over the segment sequence: quads, reifiers, annotations, and binary blobs all become rows of the folded container graph.
Properties that fall out of this design:
- Content-addressed and append-only — history is never rewritten; suppression is itself an appended record. Multi-segment files compose by simple concatenation.
- Partial readability, total reader — the reader verifies the BLAKE3 chain and folds what it can; undecodable frames (unknown codec, encrypted without a key) degrade to opaque nodes plus a diagnostic instead of aborting.
- Binary payloads ride along — the blobs a graph references travel in the same file, content-addressed like everything else.
- RDF 1.2-native — the spec formalizes the triple-term and
rdf:reifiesmapping, blank-node scoping, and multi-segment value union.
Reading and writing from Rust
The container engine (purrdf-gts) owns the wire-format machinery — reader,
writer, fold, verify, COSE, trust policy:
use purrdf::gts::reader;
// Fold GTS bytes into the container graph model, verifying the BLAKE3 chain.
let graph = reader::read(&bytes, /* allow_segments */ true, /* expected_head */ None);
// The fold is total: quads, reifiers, annotations, and blobs are all rows,
// and anything undecodable is preserved as an opaque node plus a diagnostic.
println!("{} quads, {} blobs", graph.quads.len(), graph.blobs.len());
The writer authors frames and produces byte-deterministic single-segment
snapshots (Writer::deterministic) — the GTS writer is under the same
determinism invariant as every PurRDF serializer.
The RdfDataset import/export path lives one layer up: the umbrella crate’s
gts module combines the container engine with the RDF-level adapter
(snapshot composition, content-chain verification), so RDF-facing GTS work
goes through purrdf directly.
Signing and encryption
Frames can be signed and encrypted with COSE; OpenPGP-based checks and a trust-policy layer are also part of the engine. All cryptography is pure Rust — no C toolchain, threads, or syscall dependencies — which is what keeps the whole engine wasm-friendly. An encrypted frame you cannot decrypt is simply an opaque node in the fold: the container remains readable.
Conformance vectors
GTS conformance is defined against a frozen, language-neutral vector
corpus (vectors/ in the repository), shared byte-exact across the
sibling GTS engines in other languages. The vectors are never regenerated or
“fixed” in this repository — the format is governed in the
gmeow-gts project,
alongside the specification and the other reference engines.
Known limitation at the C ABI
The GTS star-layer round-trip of a dataset containing quoted triples /
reifier bindings currently fails through the kernel
to_gts → read_graph → import_gts_graph path (star-free round-trips are
lossless). See Getting Started: C.
Related
docs/GTS-SPEC.md— the normative wire format (version 0.9-draft, wire-format major 1).- Slices, Mappings & Provenance — the RDF↔GTS loss ledger.
- Getting Started: Python — projecting a container into SQLite/DuckDB/Parquet.
Slices, Mappings & Provenance
Beyond the engines, PurRDF carries the plumbing a serious vocabulary or data-pipeline project needs: a slice catalog for organizing authored RDF, an explicit loss ledger for lossy projections, and native codecs for the SSSOM and FnO interchange formats.
The slice catalog
purrdf-slice (re-exported as
purrdf::slice) is tooling for ontology/vocabulary repositories organized as
slices — directories of authored RDF (slices/<group>/<name>/), each
described by a manifest.ttl:
- Catalog — manifest-based discovery (
SliceCatalog::discover), typed slice metadata (SliceRecord,SliceTier), and artifact roles. Slice identity comes from the manifest, not the directory name. - Ownership & dependencies — term-ownership analysis (every declared term has exactly one owning slice), dependency edges with evidence, forbidden-edge rules (extension slices depend only on core), and machine-applicable fix suggestions.
- Content addressing — deterministic artifact digests and cache keys for incremental pipelines.
- Emitters — projection/mapping emitters and lints: prefix maps, JSON-LD contexts, FnO function catalogs, claim views.
True to the rule that PurRDF mints no vocabulary IRIs, every term the slice
framework reads or emits belongs to the caller’s vocabulary: a
SliceVocab is caller-constructed (it has no Default) and threaded through
every public entry point.
use std::path::Path;
use purrdf::slice::{SliceCatalog, SliceVocab};
// Your vocabulary namespace — PurRDF fabricates none.
let vocab = SliceVocab::for_namespace("https://example.org/vocab/");
assert_eq!(vocab.slice_class(), "https://example.org/vocab/Slice");
// Discover every slice under the repository root from its manifest.ttl.
let catalog = SliceCatalog::discover(Path::new("slices"), vocab)
.expect("slices discovered");
for slice in catalog.records() {
println!("{} ({:?})", slice.manifest.slice_iri, slice.manifest.tier);
}
The loss ledger
PurRDF’s projections are allowed to be lossy — but never silently. The kernel
carries a machine-readable RDF↔GTS loss matrix
(generated/rdf-loss-matrix.json,
a generated artifact) and a LossLedger API: when a star-incapable codec
drops reifier bindings, or CSV results drop provenance, the realized count is
recorded and surfaced to the caller. See
Codecs & Determinism and
Result Formats.
Provenance
purrdf-core includes a generic provenance sidecar for the frozen IR —
attribution, origin sets, and per-quad provenance that engines can carry
without polluting the data graph. The SPARQL results extension
(Result Formats) and the SARIF
boundary both resolve these runtime-only provenance ids to public IRIs at
their serialization edges.
SSSOM and FnO
Two native interchange codecs live in the kernel:
- SSSOM — Simple Standard for Sharing Ontological
Mappings mapping-set TSV support
(
SssomMappingSet,SssomMapping, with typed diagnostics), for carrying cross-vocabulary mappings alongside your data. - FnO — a Function Ontology function-catalog codec
(
FnoCatalog,fno_to_quads,fno_to_ntriples), used by the slice emitters to describe function catalogs as RDF.
As with everything else in the toolkit, these are codecs for caller data — PurRDF does not define mappings or functions of its own.
Related
- GTS Graph Transport — the other side of the RDF↔GTS ledger.
- Design Rules & Invariants — the mints-no-IRIs rule in full.
rdflib Compatibility
Python’s rdflib is the incumbent RDF library of the ecosystem, and PurRDF meets it in two tiers: an explicit compat module, and an opt-in drop-in shadow.
Tier 1: the explicit compat layer
The main purrdf wheel ships an rdflib compatibility layer backed by the
native engine:
from purrdf.compat.rdflib import Graph
g = Graph()
g.parse(data="<https://example.org/a> <https://example.org/b> <https://example.org/c> .",
format="turtle")
This is the recommended path for new code that wants an rdflib-shaped API on
the PurRDF engine: the import name is honest, and it coexists with a genuine
rdflib installation.
Tier 2: the purrdf[rdflib] shadow distribution
For a literal, zero-change import rdflib, install the opt-in extra:
pip install purrdf[rdflib]
This pulls in the separate purrdf-rdflib distribution, whose top-level
rdflib package re-exports the compat surface, so existing third-party code
doing import rdflib / from rdflib.namespace import RDF transparently runs
on purrdf. Caveat: that shadow claims the rdflib import name and must
never be installed alongside the genuine
rdflib — the two cannot co-inhabit one
environment. It is a separate distribution (never bundled into the main
purrdf wheel) precisely so environments that need the real rdflib simply
omit it.
How compatibility is kept honest
The compat layer is not “best effort” — it is gated in CI as part of the
single conformance matrix
(docs/CONFORMANCE.md):
- The rdflib drop-in (LSP) gate runs rdflib 7.6’s own vendored test suite against the purrdf drop-in.
- The parity suite runs first-party differential tests of
purrdf.compatagainst the real rdflib 7.6.
Both use strict expected-failure ledgers: every known divergence is listed
with a per-test reason, an unexpected failure breaks the build, and a silently
fixed divergence also breaks the build until the ledger shrinks. The ledgered
residuals cover corners like Graph-subclass identity through set operators,
rdf:List/Collection mutation, Result.bindings / SELECT * subselect
projection, graph-prefix forwarding, and legacy ConjunctiveGraph semantics —
consult the ledgers for the current, exact list.
Performance
A report-only benchmark harness times the native-backed
purrdf.compat.rdflib drop-in against the real rdflib on parse, serialize,
SPARQL, and triple-pattern iteration (make bench-python). Methodology and a
representative (host-dependent) results table live in
docs/BENCHMARKS.md
— numbers vary by host, so reproduce locally rather than trusting a fixed
multiplier. See Performance for the philosophy.
Related
- Getting Started: Python
- Conformance & Testing — how the ledger discipline works.
RDF/JS in JavaScript
In JavaScript, PurRDF does not invent its own API shape: the npm package
@blackcatinformatics/purrdf
implements the RDF/JS community specifications —
DataFactory, DatasetCore, and Stream/Sink — over the wasm-compiled
native engine. Code written against RDF/JS interfaces works with PurRDF terms
and datasets.
The data factory
DataFactory covers the standard RDF/JS term constructors — namedNode,
blankNode, literal(value, languageOrDatatype?), variable,
defaultGraph, quad, fromTerm, fromQuad — plus the deliberate RDF 1.2
extensions no incumbent RDF/JS library carries:
import { ready, DataFactory } from "@blackcatinformatics/purrdf";
await ready();
const f = new DataFactory();
// RDF/JS standard surface.
const q = f.quad(
f.namedNode("https://ex/alice"),
f.namedNode("https://ex/knows"),
f.namedNode("https://ex/bob"),
);
// RDF 1.2 extensions: quoted triple terms and base-direction literals.
const quoted = f.quotedTriple(q.subject, q.predicate, q.object);
const rtl = f.directionalLiteral("مرحبا", "ar", "rtl");
typedLiteral is a convenience alongside the spec’s overloaded literal.
Datasets
Dataset implements RDF/JS DatasetCore: add, delete, has, match,
size, and iteration (for (const quad of dataset)), plus parsing and
serialization through the native codecs:
const ds = Dataset.parse("<https://ex/s> <https://ex/p> <https://ex/o> .", "ntriples");
for (const quad of ds.match(null, f.namedNode("https://ex/p"), null)) {
console.log(quad.subject.value);
}
const trig = ds.serialize("trig");
Accepted format names: turtle, ntriples, nquads, trig, rdfxml, or
their media types. Because these are the native codecs, output is
byte-deterministic and identical to what the Rust, Python, and C surfaces
emit (Codecs & Determinism).
SPARQL
Use QueryEngine when running more than one query or when the caller wants
typed results instead of raw strings. The engine owns the native SPARQL plan
cache and returns package-root terms and datasets:
import { QueryEngine } from "@blackcatinformatics/purrdf";
const engine = new QueryEngine();
const result = engine.select(
ds,
"PREFIX ex: <https://ex/> SELECT ?o WHERE { ex:s ex:p ?o }",
);
console.log(result.rows.take(0)?.o?.value);
const graph = engine.construct(
ds,
"PREFIX ex: <https://ex/> CONSTRUCT { ex:copy ex:p ?o } WHERE { ex:s ex:p ?o }",
);
SELECT rows are single-owner and lazy across the wasm boundary. Iterate
result.rows, call result.rows.take(index) for indexed consumption, or call
result.rows.toArray() to materialize the remaining rows. A row can be consumed
once; call result.free() when abandoning a result before exhaustion.
QueryEngine.queryRaw(...) serializes SELECT/ASK results as SPARQL Results
JSON/XML/CSV/TSV and graph results through the same graph formats accepted by
Dataset.serialize. QueryEngine.update(...) applies SPARQL UPDATE atomically:
the dataset changes only after the whole update succeeds.
Streams and sinks
The package speaks the async RDF/JS Stream/Sink protocol over the
purrdf-events ingestion seam:
Sink— a streaming consumer:push(quad)per quad,finish()returns the accumulatedDataset.datasetToStream(dataset)/streamToDataset(stream)— bridge between aDatasetand RDF/JS streams, for piping into or out of other RDF/JS tooling.
Scope notes
- The engine is in-memory; there is no persistent store in the wasm
build. SPARQL runs over the in-memory dataset; this package provides no
network resolver, so remote
SERVICEandLOADfail explicitly. - A quoted-triple term as a quad object currently round-trips only through N-Quads (a current native serializer limitation for the other formats).
Related
- Getting Started: JavaScript / WebAssembly
- RDF 1.2 Features — what the extensions mean.
crates/rdf-wasm— the Rust cdylib behind the package.
Design Rules & Invariants
PurRDF must stay fast, deterministic, and boring: one engine, one behavior, carried verbatim into Rust, Python, WebAssembly, and C. That promise is kept by a small set of hard invariants, each enforced by CI rather than by convention. The canonical statement is AGENTS.md in the repository; this chapter explains the why.
No Cargo features, ever
The workspace has zero feature flags, and a CI script gates it. PurRDF is
a data carrier, and optionality changes semantics per consumer — two builds of
“the same version” that parse or serialize differently would defeat the whole
point. No [features], no optional dependencies, no cfg-gated behavior
differences. Every consumer gets the same byte-identical semantics.
PurRDF is NOT an ontology — it mints no vocabulary IRIs
Every vocabulary the library reads or writes — slice manifests,
statement-metadata downcast, box roles, SPARQL extension-function namespaces,
JSON-Schema namespaces — is caller-supplied configuration with no
fabricated default. A feature exercised without its vocabulary hard-errors
or stays inactive. The library never hardcodes a vendor namespace (the GMEOW
ontology is a consumer; the dependency arrow never points from purrdf to
it), and test fixtures use example.org. Consumer-config types (SliceVocab,
Namespaces, StatementMetadataVocab) are unified behind an
OntologyProfile a downstream builds once.
Byte determinism
Serializers and the GTS writer are byte-deterministic. No iteration-order,
time, or RNG dependence is permitted in any output path; hot maps use
fixed-key ahash for this reason. Changes that alter emitted bytes must
update the affected golden files, visibly.
The kernel ring-fence
purrdf-core must never depend on oxigraph or PyO3 — the whole workspace is
oxigraph-free, and a hygiene gate asserts the dependency tree. The three
foundation leaves (purrdf-iri, purrdf-xsd, purrdf-events) keep zero
runtime dependencies. Diagnostics stay structured and SARIF-free in the
kernel; the SARIF boundary is the purrdf-validate leaf.
Everything is wasm-able
Every release crate must build for wasm32-unknown-unknown, and CI
hard-fails otherwise. No dependency may drag in threads, the filesystem, C
toolchains, or wall-clock/RNG syscalls on the wasm path — cryptography stays
pure Rust for exactly this reason. This is what makes the JavaScript package
the same engine rather than a port.
Hard-fail, never wrong
Across the toolkit, out-of-scope input is a typed error, never a partial
answer: malformed RDF is an RdfDiagnostic, an unsupported SPARQL builtin is
EvalError::Unsupported, a malformed ShEx schema is a ShexError,
D-entailment is EntailError::Unsupported, and an unsupported results
projection is a typed format error. Lossy-by-design projections are permitted
but loud, via the loss ledger.
Conformance corpora are the contract
The W3C and community test suites are vendored, byte-frozen, and SHA-256-verified; harnesses assert exact counts and enforce XPASS discipline on their expected-failure ledgers. See Conformance & Testing.
Supporting rules
- Measured performance — perf claims require a criterion bench, not an adjective (Performance).
- One version, lockstep releases — crates.io, PyPI, and npm ship one workspace version (Versioning & Releases).
- Stable toolchain only — the workspace is nightly-free; the MSRV floor (currently 1.96) is enforced by a dedicated CI job.
- Brand — the project is PurRDF in prose and
purrdfin identifiers (docs/BRAND.md).
Conformance & Testing
Every PurRDF engine is gated by its official test suite. The suites are
vendored and byte-frozen in-repo — never hand-edited, SHA-256-verified on
every make check so a silent content edit fails the build. The full, live
scoreboard is
docs/CONFORMANCE.md;
this chapter explains how the machine works.
The single conformance matrix
The native Rust W3C harnesses and the Python rdflib drop-in gate are reported together as one scoreboard:
make conformance # aggregates every suite into one table
The aggregator runs each suite in a fixed order and prints per-suite
pass / xfail-or-skip / fail counts with an overall RED/GREEN verdict. It
exits non-zero on any unexpected failure, and the rendered matrix in
docs/CONFORMANCE.md is itself drift-guarded in CI (the gate fails if the
committed block is stale).
What is gated
| Engine | Suite |
|---|---|
| IRI (RFC 3987) | W3C IRI + RFC 3986 §5.4 resolution vectors |
| Syntax codecs | W3C rdf-tests (Turtle/TriG/N-Triples/N-Quads/RDF-XML) |
| RDFC-1.0 | W3C rdf-canon fixtures |
| SPARQL 1.1/1.2 | full W3C sparql11 + sparql12 + entailment suites |
| SHACL | W3C data-shapes + DASH SHACL-AF/rules + a first-party frozen corpus |
| ShEx 2.1 | shexTest v2.1.0 (validation, schemas, negative syntax/structure) |
| Entailment | the W3C entailment cases (via the SPARQL harness) |
| GTS | frozen cross-language vectors, byte-exact |
| rdflib drop-in | rdflib 7.6’s own vendored tests + first-party parity |
At the time of writing every suite is green — for example 1,105/1,105
attempted shexTest validation cases, 126/126 W3C SHACL, 250/250 codec
round-trips, and 70/70 entailment cases — with the handful of remaining
non-passes strictly ledgered (e.g. five SPARQL fixtures with upstream-errata
non-canonical XSD lexicals). Always read the current numbers from
docs/CONFORMANCE.md
rather than this snapshot.
Ledger discipline
A harness never skips silently. Four mechanisms keep the scoreboard honest:
- Exact totals — each harness asserts the number of discovered tests, so corpus drift fails loudly.
- XFAIL ledgers — every known gap is listed with a reason string, and the harness asserts it still fails. Fixing a gap without removing its ledger entry breaks the build (XPASS discipline), so the ledgers double as roadmaps.
- Trait skips (ShEx only) — whole spec features can be skipped by manifest trait tags, counted exactly. The list is currently empty.
- A monotone budget (ratchet) — a committed baseline records the exact allowed count of ledgered gaps per suite. A larger live count fails RED (regression), and a smaller one also fails RED until the budget is lowered to lock the gain in. The budget may only ever be edited downward, which makes “the skip list only shrinks” a mechanical guarantee rather than a convention.
Running the suites locally
make conformance # the single matrix
cargo test -p purrdf-shex # all four ShEx suites
cargo test -p purrdf-shapes --test w3c_conformance # W3C SHACL scoreboard
cargo test -p purrdf-sparql-conformance # W3C SPARQL
cargo test -p purrdf-rdf # RDFC-1.0 + codec goldens
cargo test -p purrdf-gts # GTS vectors
make check — the full local gate (fmt, clippy, build, tests, hygiene) —
runs the Rust suites as part of the workspace gate.
Frozen means frozen
The GTS vectors in vectors/ are shared byte-exact with the sibling GTS
engines in other languages and are never regenerated in this repository;
the wire format is governed in
gmeow-gts. The same
never-hand-edit rule applies to every vendored corpus and everything under
generated/.
Performance
PurRDF’s performance is measured, never asserted. No number in the
project’s documentation is a guarantee: benchmarks are report-only,
timing-sensitive, and vary by host, CPU, allocator, and build flags. Treat
every figure as a host-dependent illustration you reproduce locally — not a
promise of “N× faster.” The methodology and a representative results table
live in
docs/BENCHMARKS.md.
Fast by construction
The engine-level speed comes from the IR design
(The Interned Dataset IR): every term
stored once in a string arena addressed by copyable NonZeroU32 ids,
fixed-key ahash everywhere hot, frozen Box<[QuadRow]> quad tables with
lazy ordinal permutation indexes (~4 bytes/quad per axis), and evaluation in
TermId space so solution comparison is an integer compare.
Crucially, the layout itself was chosen by benchmark: the
crates/rdf-core/benches/ir_layout.rs criterion suite measures
array-of-structs vs. struct-of-arrays vs. predicate-adjacency layouts on
allocation counts, high-water memory, and end-to-end latency — and the
shipped layout is whichever wins.
The two benchmark layers
| Layer | What it measures | How to run |
|---|---|---|
| Rust criterion suites | Native engine hot paths — IR layout, copy-on-write mutation, pack index alternatives, codecs, SPARQL lexing/evaluation/planning, SHACL validation, entailment chase, GTS authoring, IRI parsing. | make bench |
| Python compat harness | purrdf.compat.rdflib (the native-backed drop-in) vs. the real rdflib 7.x on parse, serialize, SPARQL, and triple-pattern iteration, over a deterministic example.org corpus. | make bench-python |
Both layers are report-only: they are never part of make check, and no test
gate asserts a speedup.
The discipline for changes
Any change claiming a performance win must extend the criterion benches rather than asserting the speedup in prose. Where a planner or algorithm choice matters for correctness-adjacent behavior, it is gated by deterministic tests instead of timings — for example, the cost-based BGP planner’s win over the retired structural heuristic is asserted by unit tests that count real intermediate rows, and by a differential corpus test, while the criterion bench merely watches for regressions.
NativeSparqlEngine::explain_query exposes the chosen BGP join order so
planner decisions can be audited without running the query
(SPARQL: Querying).
Reproducing locally
make bench # the default criterion set
cargo bench -p purrdf-iri --bench parse # a single package's bench
cargo bench -p purrdf-core --bench pack_index_compare # pack index experiment
make bench-python # the rdflib comparison harness
Benchmark on a quiet machine, and compare like with like: allocator, CPU scaling, and build flags all move the numbers.
Versioning & Releases
PurRDF ships to three registries — the crates.io crate suite, the PyPI
purrdf package, and the npm @blackcatinformatics/purrdf package — from
one workspace version, in lockstep. The full process is
docs/RELEASE.md.
Pre-1.0 semver policy
While the version is 0.x:
- a minor bump (
0.x→0.(x+1)) may include breaking API changes; - a patch bump (
0.x.y→0.x.(y+1)) is bugfix-only and API-compatible.
All three published surfaces share one workspace version and are released together. That coherence is enforced in CI: a version-coherence check fails the build if the three version sources disagree.
MSRV policy
The supported minimum Rust is rust-version in the root Cargo.toml —
currently 1.96 — pinned to the stable toolchain (the workspace is
nightly-free by policy) and enforced by a dedicated CI MSRV job. Raising the
MSRV is a notable change recorded in the changelog and, pre-1.0, rides a
minor bump.
Tag-driven trusted publishing
Releases are tag-driven: rust-v<version> publishes the crate suite to
crates.io, py-v<version> publishes to PyPI, and npm-v<version> publishes
the wasm package to npm. The lanes share the supply-chain posture of the
cargo lane:
- publication uses Trusted Publishing through GitHub Actions OIDC — no long-lived registry secret;
- the privileged publish jobs use pinned actions and no dependency cache;
- every
.cratepackage receives a GitHub build-provenance attestation; - the package set receives an SPDX SBOM and SBOM attestation;
- the release crate set is checked on
wasm32-unknown-unknownbefore publishing; - every workspace crate version must match the tag version.
Two crates are deliberately never published: purrdf-capi (built via
cargo-c, distributed as libpurrdf) and purrdf-sparql-conformance (the
test harness).
Cutting a release
The coherent flow from main uses the make helpers so the three lanes can
never drift:
# 1. Bump all three version sources in lockstep (fails unless they end up equal).
make bump VERSION=0.2.2
# 2. Regenerate the changelog from the conventional-commit history.
make changelog
# 3. Review, then commit the release bump + changelog.
git add -A && git commit -m "chore(release): 0.2.2"
# 4. Gate: fmt, clippy, tests, hygiene, and the version-coherence + wasm checks.
make check
# 5. From an up-to-date main, cut and push all three tags in one command.
make release-tags VERSION=0.2.2
make release-tags refuses to run unless the working tree is clean, the
branch is main, the version check passes, and VERSION matches the tree —
then it creates and pushes the rust-v, py-v, and npm-v tags together.
Each tag triggers its own lane, and the cargo lane additionally publishes a
GitHub Release built from the committed CHANGELOG.md.
Citing PurRDF
Releases carry a DOI; if you use PurRDF in research, please cite it — see
CITATION.cff
in the repository.