Glossary

📍 Where we are: The back-of-the-book reference for all of Volume 3 — every recurring term of neural representation, in plain words. Keep it open in a second tab and come back whenever a word stops making sense.

Neural representation borrows its vocabulary from linear algebra, geometry, probability, and (because this is a neuro-symbolic series) from the logic of Volume 2, then sharpens every term against one small academic knowledge graph. Below are the volume's recurring terms, in plain language, listed alphabetically so they are easy to find. Each entry is a starting point; the chapter it points to gives the full, exact picture.

Algebraic ceiling — a limitation provable for every parameter setting of a model family, not just for a badly trained instance: DistMult's forced score tie, the ball's lens false positives, and the 1-WL bound on message passing are all ceilings of this kind. (See The Honest Verdict: What Vectors Can and Cannot Hold.)

Attention — the differentiable, content-addressed read: queries and keys decide relevance through a softmax over scaled dot products, values carry content, and the output is a relevance-weighted average of the values. With orthonormal keys and sharp logits it degenerates to an exact dictionary lookup, a symbolic operation recovered as the limit of a differentiable one. (See Attention: Reasoning by Relevance.)

Ball containment criterion — B(c_A, r_A) ⊆ B(c_B, r_B) exactly when ‖c_A − c_B‖ + r_A ≤ r_B: sound by the triangle inequality, exact by the worst-point construction, and the germ of every ELEm loss and of the soundness probe. (See Balls and Cones: Concepts as Regions.)

Basis decomposition — writing every relation matrix of an R-GCN as a mixture W_r = Σ_b a_rb V_b of a small number of shared prototype matrices, so the parameter count grows with the number of bases rather than the number of relations. (See Relational GNNs: R-GCN and Beyond.)

Beta embedding — representing an entity or query as a vector of Beta(α, β) probability densities rather than a hard region; intersection multiplies densities (the parameters add) and negation is the closed-form swap (α, β) → (1/α, 1/β), an involution, so double negation is exactly the identity. (See Beta and Probabilistic Embeddings: Buying Negation with Densities.)

Bilinear model — a score function that multiplies rather than translates: the head vector, a relation vector, and the tail vector are combined term by term and summed, as in DistMult's Σ_i e_h[i] w_r[i] e_t[i]. (See Bilinear Models: DistMult and ComplEx.)

Binding — the vector-symbolic operation that composes a role and a filler into a single vector dissimilar to both inputs, so that "which filler occupies which role" survives being stored; in holographic reduced representations the binding operator is circular convolution. (See Vector Symbolic Architectures: Binding and Superposition.)

Binding problem — the inability of a weighted average to record which filler goes with which role; the question attention leaves open and the binding operator of vector symbolic architectures answers. (See Attention: Reasoning by Relevance.)

Box embedding — representing a concept or a query as an axis-aligned box, a lower and an upper corner per dimension; membership is a coordinatewise between-the-corners test, and two boxes intersect exactly (corner-wise max of lower corners, min of upper). (See Box Embeddings: Query2Box Geometry.)

BoxEL — the ontology-embedding family that renders each EL++ concept as an axis-aligned box trained by per-corner hinge losses; conjunction becomes exact corner arithmetic, and a box can be genuinely empty, giving the bottom concept ⊥ a real denotation the ball never had. (See BoxEL and Box²EL: Faithful Ontology Embeddings.)

Box²EL — BoxEL's successor, which also gives each role a head/tail pair of role boxes and each concept a bump vector added at the source of an existential, so the same role can land differently for different fillers; the ontology-level repair for one-translation-per-role. (See BoxEL and Box²EL: Faithful Ontology Embeddings.)

C² (two-variable counting logic) — first-order logic restricted to two reusable variables plus counting quantifiers ("there exist at least n"); two nodes receive the same stable 1-WL color exactly when they agree on every C² formula, which makes C² the logical name of the message-passing ceiling. (See The Expressiveness Ceiling: 1-WL, C², and Graded Modal Logic.)

Capacity — how much a fixed-width vector can hold before it stops being decodable: as the number of triples stored in one superposition grows relative to the dimension, cleanup accuracy degrades gracefully rather than failing at a cliff. (See Vector Symbolic Architectures: Binding and Superposition.)

Circular convolution (⊛) — the binding operator of holographic reduced representations: a wrap-around convolution of two d-dimensional vectors, computable in O(d log d) through the FFT, whose output resembles neither input. (See Vector Symbolic Architectures: Binding and Superposition.)

Cleanup memory — the stored list of known item vectors against which a noisy unbinding result is matched by nearest cosine; the step that converts an approximate vector answer back into an exact symbol. (See Vector Symbolic Architectures: Binding and Superposition.)

Closure under intersection — the property a region family needs for conjunction to stay in the family: boxes have it (corner arithmetic), balls lack it (two overlapping balls meet in a lens, and a lens is not a ball), and that single geometric fact decides which shape carries the ontology-embedding chapters. (See Boxes versus Balls: An Expressiveness Story.)

ComplEx — the bilinear model over complex-valued vectors, scoring with Re⟨e_h, w_r, conj(e_t)⟩; the complex conjugate on the tail makes the score asymmetric, repairing DistMult's forced tie on antisymmetric relations such as cites. (See Bilinear Models: DistMult and ComplEx.)

DistMult — the diagonal bilinear model s(h, r, t) = Σ_i e_h[i] w_r[i] e_t[i]: elegant and strong, but provably symmetric, assigning s(h, r, t) = s(t, r, h) for every relation, which is fatal for antisymmetric relations. (See Bilinear Models: DistMult and ComplEx.)

Distortion — the disagreement between graph distance and embedding distance, here measured as the scale-fitted mean of |d_emb/(s·d_graph) − 1| over all node pairs; the number hyperbolic space drives down where flat space, by a packing argument, cannot. (See Hyperbolic Embeddings: Hierarchy in Curved Space.)

EL++-closure — the property of a region family (together with its role interpretations) of being closed under intersection, existential projection, and role composition, so every complex concept of the EL++ grammar gets a region built from its parts' regions by construction rather than by luck. (See TransBox and mOWL: Closure and Tooling.)

ELEm (EL Embeddings) — the construction that maps each concept of an EL++ TBox to a ball and each role to a translation vector, trained so each normal-form axiom's geometric constraint holds; conjunction is its confessed weak point, since the intersection of two balls is not a ball. (See EL Embeddings: Geometry Meets Logic.)

Embedding — a learned vector (or region, or density) standing in for a discrete symbol, placed so that geometry encodes the symbol's relationships; the volume's models are transductive, learning one embedding per known entity and relation. (See Link Prediction: Completing the Graph.)

Entailment cone — the region a concept claims in an order embedding: all points coordinatewise at or above its vector. A subconcept's cone nests inside its ancestor's, so the entire is-a hierarchy becomes cone containment; a hyperbolic variant opens the cones away from the origin of the Poincaré ball. (See Balls and Cones: Concepts as Regions.)

EPFO query — an existential positive first-order query: a multi-hop question built from existential variables, conjunction, and disjunction, with no negation; the query family that Query2Box answers geometrically and whose boundary Beta embeddings cross. (See Box Embeddings: Query2Box Geometry.)

Expressiveness ceiling — a limitation that holds for every parameter setting of a model family, not just for a badly trained instance: message-passing GNNs, whatever their weights, can never distinguish two graphs that 1-WL color refinement cannot. (See The Expressiveness Ceiling: 1-WL, C², and Graded Modal Logic.)

Filtered ranking — the evaluation protocol that removes other known-true answers from a query's candidate list before recording the truth's rank, so a model is never penalized for correctly scoring its own knowledge. (See Link Prediction: Completing the Graph.)

Five tensions — exact versus robust, proof versus score, discrete versus differentiable, constructed versus learned, closure versus capacity: the axes on which the volume's verdict is written and on which every integration method of Volume 4 must declare its position. (See The Honest Verdict: What Vectors Can and Cannot Hold.)

GCN (graph convolutional network) — the canonical instantiation of message passing: aggregate neighbors with the symmetrically normalized adjacency D^(−1/2)(A + I)D^(−1/2), then update with a shared linear map and ReLU; the self-loop keeps a node's own state, the normalization stops high-degree blow-up. (See Message Passing: The GNN Blueprint.)

GIN (graph isomorphism network) — the message-passing architecture that reaches the 1-WL ceiling by using sum aggregation, the injective way to summarize a multiset of neighbors, where mean and max collapse genuinely different neighborhoods. (See The Expressiveness Ceiling: 1-WL, C², and Graded Modal Logic.)

Graded modal logic — modal logic with counting, able to say "at least n neighbors satisfy C"; the local fragment of the two-variable counting logic C², and exactly the logical language whose node properties message-passing GNNs can express. (See The Expressiveness Ceiling: 1-WL, C², and Graded Modal Logic.)

Graph neural network (GNN) — a network that computes a vector for each node by repeatedly aggregating messages from the node's neighbors and updating its state; the number of layers sets how many hops of structure each node can see. (See Message Passing: The GNN Blueprint.)

Hits@k — the fraction of ranking queries whose true answer lands in the top k candidates; reported at k = 1, 3, 10 alongside MRR. (See Link Prediction: Completing the Graph.)

Hyperbolic space — negatively curved space in which volume grows exponentially with radius, matching the exponential branching of a tree; hierarchies that distort badly in flat space embed here with low distortion. (See Hyperbolic Embeddings: Hierarchy in Curved Space.)

Inverse relation — the relation r⁻¹ that holds from b to a exactly when r holds from a to b. R-GCNs add one channel per inverse so a directed edge informs both of its endpoints; in evaluation, an unrecognized inverse duplicated across splits is a classic source of leakage. (See Relational GNNs: R-GCN and Beyond.)

KL divergence — the standard asymmetric measure of how one probability distribution differs from another; between Beta distributions it has a closed form in digamma functions, and Beta embeddings use it as the distance from a query's density to an entity's. (See Beta and Probabilistic Embeddings: Buying Negation with Densities.)

Knowledge graph — entities connected by directed, labeled edges, each edge a triple (h, r, t) of head, relation, and tail; the data structure every model in this volume learns from, fixed here as one small academic world. (See Link Prediction: Completing the Graph.)

Lens — the intersection of two overlapping balls: a convex region bounded by two circular arcs meeting at two corners, and not a ball; the concrete witness that balls are not closed under intersection, with the smallest enclosing ball forced to cover provably false positives. (See Boxes versus Balls: An Expressiveness Story.)

Link prediction — knowledge-graph completion reframed as ranking: for a query (h, r, ?) or (?, r, t), score every candidate entity, sort, and measure where the withheld true answer lands. The volume's shared scoreboard. (See Link Prediction: Completing the Graph.)

Logistic loss (softplus) — L = softplus(−y·s) = −ln σ(y·s), the negative log-likelihood of a triple's correct ±1 label under the sigmoid of its score, with derivative −y·σ(−y·s); the loss the bilinear models train on, paired with L2 weight decay. (See Bilinear Models: DistMult and ComplEx.)

Margin ranking loss — the hinge max(0, γ + d_pos − d_neg), which trains a scorer by pushing each true triple at least a margin γ ahead of a corrupted one, demanding relative order rather than absolute scores. (See Translational Models: TransE and Its Family.)

Message passing — the blueprint behind every GNN: in each round, a node aggregates its neighbors' current vectors into a message and updates its own vector from that message and its previous state. (See Message Passing: The GNN Blueprint.)

mOWL — the Python library that standardizes the ontology-embedding pipeline: OWL ingestion, normalization through jcel/ELK, a zoo of geometric models under one loss-dispatch contract, and a shared rank-based evaluator, at the scale of SNOMED CT and the Gene Ontology. (See TransBox and mOWL: Closure and Tooling.)

MRR (mean reciprocal rank) — the average of 1/rank over all ranking queries; top-heavy by design (a rank-1 answer contributes 1, a rank-10 answer only 0.1), with 1.0 perfect. (See Link Prediction: Completing the Graph.)

Multi-head attention — several attention heads run in parallel on learned subspaces, then concatenated and mixed; the capacity to track multiple simultaneous relevance patterns, roughly one relation per head. (See Attention: Reasoning by Relevance.)

Negation (BetaE involution) — the closed-form parameter swap (α, β) → (1/α, 1/β) applied to each Beta component; it is an involution (its own inverse), so double negation is exactly the identity, and it supplies the operator no ball or box can express because their complements leave the family. (See Beta and Probabilistic Embeddings: Buying Negation with Densities.)

Negative sampling — generating training negatives by corrupting a true triple's head or tail with a random entity, while consulting a list of known-true triples so that a genuine fact is never presented as false. (See Link Prediction: Completing the Graph.)

Norm constraint — pulling entity vectors back to the unit sphere between rounds of gradient updates; it removes the degenerate solution in which all embeddings shrink together to satisfy the margin trivially, so training learns structure instead of collapsing. (See Translational Models: TransE and Its Family.)

Normal form (NF1–NF4) — the four axiom shapes Volume 2's normalization reduces any EL++ TBox to: C ⊑ D, C ⊓ D ⊑ E, C ⊑ ∃r.D, and ∃r.C ⊑ D. Ontology embeddings turn each shape into its own geometric loss term, so the normal forms are the interface between the logic and the geometry. (See EL Embeddings: Geometry Meets Logic.)

Order embedding — an embedding in which hierarchy is read directly off coordinates: one concept subsumes another when the more specific vector dominates the more general one in every coordinate, making is-a a component-by-component ≥ test. (See Balls and Cones: Concepts as Regions.)

Permutation equivariance — the guarantee that renumbering a graph's nodes merely renumbers a GNN's outputs in the same way; the network computes on structure, not on the arbitrary order in which nodes happen to be listed. (See Message Passing: The GNN Blueprint.)

Poincaré ball — the model of hyperbolic space inside the unit ball, with distance d(u, v) = arcosh(1 + 2‖u−v‖² / ((1−‖u‖²)(1−‖v‖²))), which blows up near the boundary; general concepts sit near the center, specific ones near the rim, and depth in the hierarchy reads off the norm. (See Hyperbolic Embeddings: Hierarchy in Curved Space.)

Projection (relation) — Query2Box's per-relation operator on regions: following relation r translates the current box by r's offset and widens it by r's softplus-positive size, taking "everything one r-hop away" in a single geometric step that can only grow the region. (See Box Embeddings: Query2Box Geometry.)

Query2Box — the model that embeds a whole multi-hop query as a box: anchor entities are points, each relation hop projects the box, conjunction intersects boxes exactly, disjunction is handled by rewriting into a union of conjunctive branches, and the answers are the entities whose points fall inside or nearest to the final box. (See Box Embeddings: Query2Box Geometry.)

Query2Box distance — dist_outside + α·dist_inside: the L1 distance from an entity's point to the box surface, plus an α-discounted pull toward the box center; a zero outside term means the entity sits inside the box. (See Box Embeddings: Query2Box Geometry.)

RBF kernel — k(p, q) = exp(−‖u_p − u_q‖²/2σ²) on symbol embeddings: identical symbols score exactly 1, near-synonyms score high, unrelated symbols score near 0. The width σ is the tolerance dial, recovering hard equality in the limit σ → 0. (See Soft Unification: Matching Symbols in Vector Space.)

Receptive field — the T-hop neighborhood that a T-layer GNN's output provably depends on, and nothing outside it; the exact statement of message passing's locality, verified bitwise in the companion code. (See Message Passing: The GNN Blueprint.)

R-GCN (relational graph convolutional network) — the GCN extended to typed multigraphs: one weight matrix per relation, plus inverse-relation channels and a self-loop matrix, with each relation's messages averaged by its own in-degree and all channels summed before the nonlinearity. (See Relational GNNs: R-GCN and Beyond.)

Riemannian SGD — gradient descent corrected for curved space: the Euclidean gradient is rescaled by the inverse metric (on the Poincaré ball, by (1 − ‖θ‖²)²/4) before the step, so updates respect hyperbolic distances rather than flat ones. (See Hyperbolic Embeddings: Hierarchy in Curved Space.)

Scaled dot-product attention — A = softmax(QKᵀ/√d_k), O = AV: relevance scores from query–key dot products, divided by √d_k so the logit variance stays at 1 for any key width, then normalized row-wise and used to average the values. (See Attention: Reasoning by Relevance.)

Score function s(h, r, t) — any map from candidate triples to real numbers, higher meaning more plausible; every link-prediction model in the volume is, at bottom, a choice of score function. (See Link Prediction: Completing the Graph.)

Soft unification — replacing symbolic unification's yes-or-no match with a kernel score on the symbols' embedding vectors: identical symbols score exactly 1, similar ones score high, dissimilar ones near 0, so a prover can use facts written in a vocabulary it was never given. (See Soft Unification: Matching Symbols in Vector Space.)

Softmax Jacobian — the derivative matrix diag(p) − p pᵀ of the softmax, derived by the quotient rule; it shrinks toward zero as the distribution saturates toward one-hot, which is why over-sharp attention stops learning. (See Attention: Reasoning by Relevance.)

Soundness probe — an empirical confusion table: test every ordered pair of named concepts for geometric containment, compare against the exact reasoner's entailed subsumptions, and count the true and false positives. A measurement of soundness, never a theorem of it. (See EL Embeddings: Geometry Meets Logic.)

Superposition — storing many bound pairs in one vector by simple addition; each stored item remains approximately recoverable, and the whole academic knowledge graph fits, noisily, in a single d = 1024 vector. (See Vector Symbolic Architectures: Binding and Superposition.)

t-norm composition — scoring a multi-step proof by combining its per-step match scores with a t-norm; soft unification uses min (the Gödel t-norm), so a proof is only as strong as its weakest step, the same weakest-link algebra as Volume 2's confidence lattice. (See Soft Unification: Matching Symbols in Vector Space.)

TransBox — the construction that interprets every role as a box of allowed translations and builds a complex concept's region from its parts by corner arithmetic and Minkowski sums (the set of all pairwise sums of two sets, again a box for boxes), aiming at closure under the full EL++ concept grammar. (See TransBox and mOWL: Closure and Tooling.)

TransE — the minimal embedding model: each entity a point, each relation a translation vector, a triple true to the degree that e_h + w_r lands near e_t, scored by negative distance. (See Translational Models: TransE and Its Family.)

Translation (relation as) — the geometric reading of a relation as a fixed displacement of space: advises is one arrow, added to any adviser's point to reach the advisee's. Simple and trainable, and provably unable to represent symmetric or one-to-many relations exactly, the failure modes TransH, TransR, and RotatE each repair. (See Translational Models: TransE and Its Family.)

Unbinding — applying the binding operator's approximate inverse (for circular convolution, binding with the involution of the cue) to a superposed memory; it recovers a noisy version of the stored partner, which cleanup then sharpens into an exact symbol. (See Vector Symbolic Architectures: Binding and Superposition.)

Vector symbolic architecture (VSA) — an algebra over high-dimensional vectors (bind, superpose, unbind, clean up) that stores and queries symbolic structure entirely inside fixed-width vectors, trading exactness for graceful degradation. (See Vector Symbolic Architectures: Binding and Superposition.)

Weisfeiler-Leman (1-WL) — the color-refinement test: repeatedly recolor every node by its own color together with the multiset of its neighbors' colors, until the coloring stabilizes. Two graphs whose color histograms never diverge are 1-WL-indistinguishable, and no message-passing GNN can tell them apart either. (See The Expressiveness Ceiling: 1-WL, C², and Graded Modal Logic.)

Zero-loss soundness theorem — the guarantee that a geometry with exactly zero training loss is a model of the ontology and therefore satisfies every entailed statement: perfect recall, not perfect precision, and out of reach the moment any loss remains. (See TransBox and mOWL: Closure and Tooling.)

If a term here still feels fuzzy, follow it back into the chapter where it lives, and it will make far more sense in context.