Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

GTS Graph Transport

GTS (Graph Transport Substrate) is an ontology-independent binary container and transport format for RDF 1.2 datasets and content-addressed binary payloads. PurRDF hosts the reference Rust engine, purrdf-gts (re-exported as purrdf::gts).

This chapter is a user-level tour. The wire format itself is specified in docs/GTS-SPEC.md — consult the spec for framing, fold semantics, registries, and conformance classes; nothing here supersedes it.

The container model, at a high level

A GTS file is a CBOR Sequence of one or more append-only segments. Each segment is a deterministic CBOR header followed by deterministic CBOR frames chained by BLAKE3 content identifiers. The logical dataset is obtained by a deterministic fold over the segment sequence: quads, reifiers, annotations, and binary blobs all become rows of the folded container graph.

Properties that fall out of this design:

  • Content-addressed and append-only — history is never rewritten; suppression is itself an appended record. Multi-segment files compose by simple concatenation.
  • Partial readability, total reader — the reader verifies the BLAKE3 chain and folds what it can; undecodable frames (unknown codec, encrypted without a key) degrade to opaque nodes plus a diagnostic instead of aborting.
  • Binary payloads ride along — the blobs a graph references travel in the same file, content-addressed like everything else.
  • RDF 1.2-native — the spec formalizes the triple-term and rdf:reifies mapping, blank-node scoping, and multi-segment value union.

Reading and writing from Rust

The container engine (purrdf-gts) owns the wire-format machinery — reader, writer, fold, verify, COSE, trust policy:

#![allow(unused)]
fn main() {
use purrdf::gts::reader;
use purrdf::gts::{model::{Graph, Term}, writer::Writer};

let mut source = Graph::default();
source.terms.extend([
    Term::iri("https://example.org/alice"),
    Term::iri("https://example.org/knows"),
    Term::iri("https://example.org/bob"),
]);
source.quads.push((0, 1, 2, None));
let bytes = Writer::deterministic(&source, "example-profile")?.into_bytes();

// Fold GTS bytes into the container graph model, verifying the BLAKE3 chain.
let graph = reader::read(&bytes, /* allow_segments */ true, /* expected_head */ None);

// The fold is total: quads, reifiers, annotations, and blobs are all rows,
// and anything undecodable is preserved as an opaque node plus a diagnostic.
println!("{} quads, {} blobs", graph.quads.len(), graph.blobs.len());
assert_eq!(graph.quads.len(), 1);
assert!(graph.blobs.is_empty());
assert!(graph.diagnostics.is_empty());
assert_eq!(bytes, Writer::deterministic(&graph, "example-profile")?.into_bytes());
Ok::<(), Box<dyn std::error::Error>>(())
}

The writer authors frames and produces byte-deterministic single-segment snapshots (Writer::deterministic) — the GTS writer is under the same determinism invariant as every PurRDF serializer.

The RdfDataset import/export path lives one layer up: the umbrella crate’s gts module combines the container engine with the RDF-level adapter (snapshot composition, content-chain verification), so RDF-facing GTS work goes through purrdf directly.

Signing and encryption

Frames can be signed and encrypted with COSE; OpenPGP-based checks and a trust-policy layer are also part of the engine. All cryptography is pure Rust — no C toolchain, threads, or syscall dependencies — which is what keeps the whole engine wasm-friendly. An encrypted frame you cannot decrypt is simply an opaque node in the fold: the container remains readable.

Rust-library-only: compaction certificates, proofs and keyring verification

Beyond reading, writing and folding, purrdf-gts and the umbrella’s gts module carry a verification stack that no other surface reaches — not the CLI, Python, wasm or C:

  • Streamable compaction with a certificate — compact_streamable rewrites an accretive log into one delivery-ordered segment (leading stream index, trailing offset footer, in-band dictionaries), and compact_and_certify / verify_compaction / compose bind a content projection so that the pre- and post-compaction containers fold to byte-identical canonical content; the verifier recomputes from the pre bytes with its own keyring. Compaction never changes content and mints no content signatures.
  • Merkle mountain range proofs — mmr::{root, prove, verify_proof, prove_file} (gts-mmr-proof-v1), and verify_content_chain, which walks COSE signature → head replay → digest inclusion. There is no transparency log and no signed checkpoint service.
  • OpenPGP transport keys — armored Ed25519 certificates and unencrypted secret keys (parse_transport_key, parse_secret_signing_key) with verify_file_with_keyring for rotation; other algorithms, encrypted keys and v5/v6 packets are rejected, and there is no keyring or revocation store.

Native unit tests cover file media-type mapping and tar/PAX serialization and parsing. Frozen vectors and compaction/certification fixtures exercise additional paths. These checks cover their stated cases; they do not establish complete coverage of the files profile, nested GTS-in-blob reader, trust policies or ULID behavior. The upstream gmeow-gts specification and frozen vectors remain the format authority.

Conformance vectors

GTS conformance is defined against a frozen, language-neutral vector corpus (vectors/ in the repository), shared byte-exact across the sibling GTS engines in other languages. The vectors are never regenerated or “fixed” in this repository — the format is governed in the gmeow-gts project, alongside the specification and the other reference engines.

The C ABI’s star-layer round-trip

The GTS star-layer round-trip of a dataset containing quoted triples / reifier bindings succeeds through the kernel to_gts → read_graph → import_gts_graph path, the same as a star-free round-trip; crates/rdf-capi/tests/abi.rs’s gts_star_roundtrip_preserves_the_statement_layer pins it. See Getting Started: C.