From GQE to BetaE: Points, Boxes, Distributions

📍 Where we are: Part V · Complex Logical Query Answering — Chapter 16. Query Embedding: The Computation DAG built the instrument: the DAG, the exact executor, the easy/hard protocol, and the two reference columns (oracle 1.0000, traversal 0.1491). This chapter puts the first three neural players in front of it.

The previous chapter ended with an empty column: a model that beats traversal's 0.1491 by inferring held-out edges. Query embedding is the field's answer, and its move is radical in one specific way. Instead of executing the query DAG (directed acyclic graph) over sets of entities, embed it: walk the DAG bottom-up exactly as the symbolic executor does, but let each node output a geometric object in a vector space rather than a set, and at the root rank every entity by its distance to the final object. The executor's four set operators become learned geometric operators, and the entire question of what the model can express collapses into one design decision: what kind of object is a query node? Three generations gave three answers. GQE (Graph Query Embedding) says a point [1], and inherits a fast conjunctive executor that provably cannot say "or" or "not" (the two theorems of absence derived below). Query2Box says a box, which buys union, but only by rewriting the query outside the embedding space [2]. BetaE says a vector of Beta distributions, the first geometry closed under the full fragment, negation included [3]. The companion module clqa_models.py miniaturizes all three behind one shared interface, trains them on the same query bank with hand-written gradients, and scores them with the previous chapter's own filtered_mrr. Its committed 14-row table is this chapter's verdict.

The simple version

Imagine three cartographers asked to mark "where the answers live" on a city map. The first uses pushpins: perfect for a single address, workable for "near the intersection of these two streets", but hopeless for "either of these two neighborhoods" (one pin cannot sit in both) and absurd for "everywhere except downtown" (no pin means outside). The second uses rectangles: a neighborhood is a rectangle, overlapping two rectangles is a rectangle, but two separate neighborhoods still need two rectangles drawn side by side, and the outside of a rectangle is not a rectangle, so "except" stays unsayable. The third abandons shapes with edges entirely and paints heat maps: every operation, overlaying two maps, blending them, or inverting hot and cold, produces another heat map, so nothing is unsayable. Inverting is the trick the first two could never do. That ladder, pin to rectangle to heat map, is GQE to Query2Box to BetaE.

What this chapter covers

• The shared skeleton: one embed_query walk over the DAG, six operator slots each model must fill (anchor, projection, intersection, union, negation, distance), and the controlled-experiment discipline carried over from Volume 3: same graph, same bank, same protocol, so the models differ only where they differ.
• GQE, the point baseline: projection as translation, intersection as a permutation-invariant DeepSets pooling, and two theorems-of-absence derived honestly: why no point can represent a union, and why negation is not even approximable.
• Query2Box extended, not re-derived: Volume 3 built the box geometry; what is new here is the disjunctive-normal-form theorem that buys union from outside the space, at the price of multiplying branches.
• BetaE's closure: the reciprocal negation as an exact involution, intersection as weighted parameter interpolation (with the exponent-adding derivation of why Betas stay Betas), and De Morgan union derived in parameter space.
• The controlled comparison: identical training queries, shared filtered negative pools, one margin-loss form (each model keeping its own tuned margin, step size, and epoch count), with hand gradients certified against finite differences to 1.8e-09.
• The committed 14-row table, read in three sweeps: the dash pattern against the structural theory, the shared rows read honestly at toy scale, and the zero-shot rows as a compositional-generalization probe.
• The interpretability bill: what "the query is the geometry" silently gives up relative to the symbolic executor, setting up the fuzzy-set alternative.

One interface, three geometries

An experiment comparing three models is worthless if the models are allowed to differ in more than the thing being compared. So the companion pins everything else down. All three models implement the same two-method contract: embed_query(q) walks a query DAG from the previous chapter's bank bottom-up and returns a list of geometric objects (one per disjunct, for a reason the Query2Box section derives), or raises if the geometry has no operator for one of the query's connectives; score(e, q) returns a scalar, higher meaning "entity $e$ is a more plausible answer", computed as the negated distance from $e$'s embedding to the nearest returned object. Everything downstream is shared and symbolic: the 36-query bank, the easy/hard split, and query_dag.filtered_mrr (query_dag.py lines 391–408) score all three identically. This is the discipline Volume 3's embedding chapters installed, now applied one level up: where the table's dash pattern differs, the only available explanation is the geometry, and where its numeric cells differ, the geometry is the dominant explanation left standing (each model keeps its own tuned hyperparameters, a caveat the training section discloses).

Filling the contract means filling six operator slots beneath the choice of query object (the table's first row), and the whole chapter is this table unpacked:

slot	GQE (point)	Query2Box (box)	BetaE (Beta vectors)
query object	$\mathbf{q} \in \mathbb{R}^{16}$	$(\mathbf{c}, \mathbf{o})$, offset $\mathbf{o} \ge \mathbf{0}$, in $\mathbb{R}^{8}$	$2K$ log-parameters, $K = 4$ Betas
anchor entity	its point	degenerate box, zero offset	its parameter vector
projection $P_r$	translate: $\mathbf{q} + \mathbf{t}_r$	translate and widen	affine map on log-parameters
intersection $\mathcal{I}$	elementwise min (DeepSets)	attention centers, shrunk offsets	weighted parameter sums
union $\vee$	none (raises)	DNF outside the space	DNF (De Morgan also available)
negation $\neg$	none (raises)	none (raises)	$(\alpha, \beta) \mapsto (1/\alpha, 1/\beta)$
distance	$\lVert \mathbf{q} - \mathbf{v} \rVert_2$	outside $+\; 0.2 \cdot$ inside	$\sum_k \mathrm{KL}$

Here $\mathbb{R}^{16}$ is the space of lists of 16 real numbers (16 is GQE's embedding dimension, the count of coordinates each entity and query carries; Query2Box uses 8, matching Volume 3's boxes.py); the symbol $\in$ reads "is a member of", so the first row says the query is one such list, and $\mapsto$ in the negation row reads "maps to": the object on the left is sent to the object on the right. $\mathbf{v}$ is the candidate entity's embedding, $\lVert \cdot \rVert_2$ is the Euclidean norm, DNF abbreviates disjunctive normal form (the union-last rewrite the Query2Box section derives), and $\mathrm{KL}$ is the Kullback–Leibler divergence between probability distributions, defined when we reach BetaE. Three more table symbols deserve naming now rather than at their sections: in the box column, $\mathbf{c}$ is the box's center and $\mathbf{o}$ its per-dimension offset (the half-width in each coordinate); in the negation row, $(\alpha, \beta)$ are the two shape parameters of a Beta distribution, defined in the BetaE section; and the sum $\sum_k \mathrm{KL}$ in the distance row adds one KL term per Beta slot, the index $k$ running over the $K = 4$ slots. The two none entries are not implementation laziness; they are the structural theory this module exists to demonstrate, and the code makes them load-bearing rather than decorative. A model with no operator for a connective raises an exception, and the experiment's dash pattern is declared up front and asserted (clqa_models.py lines 524–532):

# The structural theory this module exists to demonstrate: which of the 14
# types each geometry can express AT ALL.  GQE (conjunctive only) dashes the
# 2 union and 5 negation types; Q2B's DNF buys back the unions; BetaE's
# closed-form complement leaves nothing inexpressible.
DASHES: dict[str, frozenset[str]] = {
    "GQE": frozenset({"2u", "up", "2in", "3in", "inp", "pin", "pni"}),
    "Q2B": frozenset({"2in", "3in", "inp", "pin", "pni"}),
    "BetaE": frozenset(),
}

The harness verifies the pattern in both directions: on a dashed type, embed_query must raise StructuralGap (silently guessing would be a bug), and on a supported type it must not (clqa_models.py lines 633–643, asserted again at lines 681–683). A dash in the final table is therefore a checked mathematical claim about the geometry, not a missing number.

Three answers to "what object is a query node": a point that can only translate and pool, a box that widens and intersects but must handle union outside the space, and a vector of Beta densities closed under intersection and negation; the bottom band is the 7/5/0 dash pattern the committed table verifies. Original diagram by the authors, created with AI assistance.

GQE: the honest poverty of a point

GQE (Graph Query Embedding) embeds every entity and every query node as one point [1]. The companion implements the translation variant: projection, the operator that follows relation $r$ one hop, is the addition of a per-relation vector $\mathbf{t}_r$, exactly Volume 3's TransE move recruited for query nodes; intersection pools branch embeddings elementwise (clqa_models.py lines 222–232):

    def _embed(self, q) -> np.ndarray:
        if isinstance(q, str):
            return self.ent[E_ID[q]].copy()
        if _chain(q):
            v = self._embed(q[0])
            for r in q[1]:
                # P(q, r) = q + t_r  (projection = translation)
                v = v + self.rel[R_ID[r]]
            return v
        # I({q_1..q_n}) = elementwise min over the branch embeddings
        return np.min(np.stack([self._embed(s) for s in q]), axis=0)

The min deserves a pause, because its shape is forced by logic, not chosen by taste. Intersection consumes a set of branch embeddings, and sets have no operand order: $S_1 \cap S_2 = S_2 \cap S_1$, so the operator $\mathcal{I}$ must satisfy $\mathcal{I}(\mathbf{q}_1, \ldots, \mathbf{q}_n) = \mathcal{I}(\mathbf{q}_{\rho(1)}, \ldots, \mathbf{q}_{\rho(n)})$ for every reordering $\rho$ of the indices $1$ through $n$ (here $n$ is the number of branches meeting at the node; the reordering gets the letter $\rho$ because $\sigma$ is reserved for the sigmoid later in this chapter). A function with that property is called permutation-invariant, and the standard recipe for building trainable ones is the DeepSets form, a symmetric pooling of the inputs wrapped in feed-forward layers [4]; GQE's published intersection is precisely that, with an elementwise mean or min as the pooling, the choice made on validation data [1]. The companion fixes the min and drops the wrapping layers (stated plainly in its header, lines 39–58): the elementwise min is permutation-invariant on its own, since the minimum of a set of numbers does not care in which order the numbers arrive. Scoring is the negated Euclidean distance, $s(v, q) = -\lVert \mathbf{q} - \mathbf{v} \rVert_2$ (lines 234–237; the published model scores by cosine similarity, a substitution the same header discloses).

Now the poverty, stated as two theorems of absence rather than as missing features. No point operation can represent union. Fix any query point $\mathbf{q}$ and consider two answers $\mathbf{a}$ and $\mathbf{b}$ sitting in two well-separated clusters, with the midpoint $\mathbf{m} = \tfrac{1}{2}(\mathbf{a} + \mathbf{b})$ a non-answer between them. Write the vector from the midpoint to the query as an average, $\mathbf{q} - \mathbf{m} = \tfrac{1}{2}(\mathbf{q} - \mathbf{a}) + \tfrac{1}{2}(\mathbf{q} - \mathbf{b})$, and apply the triangle inequality (the norm of a sum is at most the sum of the norms):

$$\lVert \mathbf{q} - \mathbf{m} \rVert \;\le\; \tfrac{1}{2}\lVert \mathbf{q} - \mathbf{a} \rVert + \tfrac{1}{2}\lVert \mathbf{q} - \mathbf{b} \rVert \;\le\; \max\big(\lVert \mathbf{q} - \mathbf{a} \rVert,\; \lVert \mathbf{q} - \mathbf{b} \rVert\big),$$

the last step because an average of two numbers never exceeds the larger. So the midpoint scores at least as well as the worse of the two answers, for every possible placement of $\mathbf{q}$. A disjunctive answer set with two separated clusters therefore has no faithful single-point representation: whatever point you pick, some non-answer in the middle ranks at least as high as a true answer, and under the protocol's expected-rank tie rule even the tie costs the answer rank. Negation is worse: not even approximable. Under distance scoring, "the entities a point accepts" is always a ball, the set with $\lVert \mathbf{v} - \mathbf{q} \rVert \le \tau$ for some threshold $\tau$, and a ball is bounded and convex (a set is convex when it contains the whole straight segment between any two of its points). The complement of a near-point set is unbounded, almost everything in the space, and no ball is the complement of a ball; that kills exact representation. Convexity also closes the approximation escape, under one hypothesis that must be stated: the entities negation must exclude (the original answers) sit inside the convex hull of the entities it must include (the complement), where the convex hull of a set of points is the smallest convex set containing them, so any convex set containing the points contains their hull as well. That is exactly the configuration for a near-point answer set with complement entities on all sides of it, and it is fatal for a ball: being convex, a ball cannot contain that surrounding shell without containing the hull's interior too, so any placement of $\mathbf{q}$ that accepts the complement also accepts what it was supposed to reject. The companion enforces both absences with the raises quoted earlier, and its committed run states them out loud:

[3] structural gaps, enforced (embed_query raises, never guesses)
  GQE    on 2u: StructuralGap — GQE has no union operator (one point cannot cover two disjoint answer regions)
  GQE    on 2in: StructuralGap — GQE has no negation operator (the complement of a neighborhood is not a neighborhood)
  Q2B    on 2in: StructuralGap — Q2B has no negation operator (the complement of a box is not a box)
  BetaE  embeds all 36 bank queries (42 DNF disjuncts) — nothing raises

Where GQE does compete, it is genuinely competent: in the committed table (quoted in full below) it posts 0.6667 on 1p, a perfect 1.0000 on 2i, and 0.4167 on the never-trained pi type. The original system answered conjunctive queries over graphs with millions of edges at a fraction of the cost of enumerating subgraphs, and its own scope statement is candid: existential conjunctive queries, disjunction and negation out of scope [1]. The two later generations are best read as paying, one connective at a time, for what the point could not say. Even ip and pi exist for GQE only by composition: the model was trained on 1p and 2i shapes alone, and answers deeper types purely by chaining its two operators, a point the zero-shot sweep of the table returns to.

Query2Box: boxes buy union, from the outside

Volume 3's Box Embeddings chapter already built this geometry in full: a query is an axis-aligned box, a center $\mathbf{c}$ plus a non-negative per-dimension offset $\mathbf{o}$; projection translates the center and widens the offset (following a hop can only grow an answer region); and the point-to-box distance splits into an outside part and a discounted inside part, with every subgradient derived kink by kink in boxes.py lines 122–141. None of that is re-derived here; this suite literally imports it, and pins the discount with an assert (clqa_models.py lines 78 and 84):

from boxes import ALPHA, box_dist, box_dist_grads, sigmoid, softplus  # noqa: E402

assert ALPHA == 0.2  # the Query2Box inside-distance down-weight this suite uses

so the distance every Query2Box cell in the table is computed with is Volume 3's, verbatim (boxes.py lines 105–119): with $\mathbf{a} = \lvert \mathbf{v} - \mathbf{c} \rvert$ the per-dimension gap from the candidate entity to the center,

$$d_{\text{box}}(\mathbf{v};\, \mathbf{c}, \mathbf{o}) \;=\; \big\lVert \max(\mathbf{0},\, \mathbf{a} - \mathbf{o}) \big\rVert_1 \;+\; 0.2 \cdot \big\lVert \min(\mathbf{a},\, \mathbf{o}) \big\rVert_1,$$

where $\lVert \cdot \rVert_1$ sums absolute values across dimensions: the first term is the L1 path from the point to the box surface (zero inside), and the second, down-weighted by $\alpha = 0.2$, orders the points already inside gently toward the center [2]. Two things are new at the query-DAG level. The first is the learned intersection (clqa_models.py lines 326–334):

    def _intersect(self, boxes):
        """Cen_∩ = Σ_i a_i c_i with a = softmax(w·c_i);
        Off_∩ = min_i(o_i) ⊙ σ(g)  — strictly inside the smallest branch."""
        Cm = np.stack([c for c, _ in boxes])
        Om = np.stack([o for _, o in boxes])
        s = Cm @ self.att
        a = np.exp(s - s.max())
        a /= a.sum()
        return a @ Cm, Om.min(axis=0) * sigmoid(self.gate), a

The new center is an attention-weighted average of the branch centers (a softmax, $a_i = e^{s_i} / \sum_j e^{s_j}$, turns the raw scores $s_i = \mathbf{w} \cdot \mathbf{c}_i$ into positive weights summing to one, where $\mathbf{w}$ is a single learned scoring vector shared by every branch, the companion's self.att at line 303, so the result is again permutation-invariant); the new offset is the elementwise minimum of the branch offsets, shrunk by a learned gate $\sigma(\mathbf{g})$, where $\sigma$ is the logistic sigmoid $\sigma(x) = 1/(1 + e^{-x})$, whose output lies strictly between 0 and 1. The result can therefore never be wider than the smallest branch box: its offset is strictly smaller in every dimension whenever the branch offsets are positive, which they always are here because every branch reaching an intersection ends in a projection whose softplus widening is strictly positive (boxes.py lines 92–96), an inductive bias echoing set semantics ($A \cap B \subseteq A$), and the harness checks exactly this strict offset shrink on the trained model (clqa_models.py lines 702–708). Containment of the box itself is not guaranteed, because the attention-weighted center is a convex combination of the branch centers and can drift outside the smallest branch. On the worked 2i query the committed internals read: attention $a = (0.729, 0.271)$, branch offsets of L1 mass $2.323$ and $2.323$, intersection offset $1.154$, strictly smaller than both.

The second new thing is the union, and it is the chapter's most instructive negative-turned-positive result. A box fails union for the same midpoint reason a point does: a box is convex, so any box containing two separated answer clusters also contains the non-answers between them. The Query2Box response is a theorem, not a workaround: every existential positive first-order (EPFO) query, the fragment built from ∃ (read "there exists"), ∧ ("and"), and ∨ ("or") with no negation, can be rewritten into disjunctive normal form (DNF), a union of union-free conjunctive queries, with the union applied as the very last step [2]. The rewrite needs only two identities. Projection distributes over union, $P_r(A \cup B) = P_r(A) \cup P_r(B)$, because "some head in $A \cup B$ has an $r$-edge to $t$" holds exactly when some head in $A$ does or some head in $B$ does. And intersection distributes over union, $(A \cup B) \cap C = (A \cap C) \cup (B \cap C)$. Applying both bottom-up pushes every interior $\cup$ upward until all unions sit at the root. The companion's dnf (lines 175–191) is this rewrite, and its recursion mirrors the two identities line for line; the same function also refuses the one thing the rewrite cannot do (complement does not distribute over union), which the bank never asks of it.

Each union-free disjunct then gets its own box, and the union happens at the score level: $s(v, q) = \max_j \big( -d_{\text{box}}(\mathbf{v};\, \text{box}_j) \big)$, the negated minimum over disjunct distances (lines 336–339). At that final aggregation step the treatment is exact, because membership in a union is precisely an existential over disjuncts: writing $\llbracket q \rrbracket$ for a query's answer set (the previous chapter's denotation brackets), $v \in \bigcup_j \llbracket q_j \rrbracket$ if and only if $v \in \llbracket q_j \rrbracket$ for some $j$, and taking the best disjunct score mirrors that "some" without loss. Whatever error remains lives inside each conjunctive branch's embedding, not in the union itself. The price is multiplicity: the rewrite multiplies branches (the code's product(*...) over intersection operands), in the worst case exponentially in the number of union nodes, and the committed run counts the cost on our bank: the full 36-query bank rewrites to 42 disjuncts (printed on the BetaE line of the structural-gap block, since BetaE is the one model that embeds all of them; Query2Box itself embeds the 25 union-and-conjunction queries of that bank, 31 disjuncts). There is also a theorem explaining why this exile from the embedding space is necessary rather than lazy: keeping union inside the space, one region per query under a distance-threshold reading, forces the embedding dimension to grow linearly with the number of pairwise-disjoint answer sets the space must distinguish, which at knowledge-graph scale means dimension on the order of the entity count [2]. The 2u and up rows of the committed table (0.6000 and 0.2250) exist for Query2Box only through this DNF route; GQE's dashes sit beside them. Negation, however, remains out of reach: the complement of a box is not a box, and the raise quoted in the previous section enforces it.

BetaE: closure, bought with densities

BetaE ends the arms race by changing the object to something with no boundary at all: each entity and each query node becomes a vector of $K$ independent Beta distributions [3]. A Beta distribution $\mathrm{Beta}(\alpha, \beta)$ is a probability density on the unit interval, $p(x) = x^{\alpha - 1}(1 - x)^{\beta - 1} / B(\alpha, \beta)$ for $x$ between 0 and 1, where the two positive shape parameters $\alpha$ and $\beta$ control where the mass piles up and $B(\alpha, \beta)$ is the normalizing constant that makes the density integrate to one. The companion uses $K = 4$ Betas per object, so an embedding is $2K = 8$ numbers, stored as logarithms $(\ln \alpha, \ln \beta)$ so the parameters stay positive no matter what gradient descent does; all of this is Volume 3's Beta and Probabilistic Embeddings machinery (beta.py), imported rather than rebuilt.

The three operators, in the order the DAG meets them. Projection is a per-relation affine map on the log-parameters, $\mathbf{l} \mapsto W_r \mathbf{l} + \mathbf{c}_r$ with $W_r$ a learned $8 \times 8$ matrix and $\mathbf{c}_r$ a learned shift (the published model uses a small per-relation multi-layer perceptron here; the companion linearizes it to the affine map, the same simplification spirit as its frozen attention weights below). Intersection is a weighted interpolation of parameters, and this is where a small derivation shows why the family is closed where balls and boxes were not. Take branch densities $p_i(x) \propto x^{\alpha_i - 1}(1 - x)^{\beta_i - 1}$ (the symbol $\propto$ reads "proportional to": equal up to the normalizing constant) and non-negative weights $w_i$ summing to one. The natural product-of-experts combination is the weighted geometric mean of the densities, and its exponents simply add:

$$\prod_{i=1}^{n} p_i(x)^{w_i} \;\propto\; \prod_{i=1}^{n} x^{w_i(\alpha_i - 1)} (1 - x)^{w_i(\beta_i - 1)} \;=\; x^{\sum_i w_i \alpha_i \,-\, \sum_i w_i} \,(1 - x)^{\sum_i w_i \beta_i \,-\, \sum_i w_i},$$

and since $\sum_i w_i = 1$ the exponents are $\big(\sum_i w_i \alpha_i\big) - 1$ and $\big(\sum_i w_i \beta_i\big) - 1$: an unnormalized $\mathrm{Beta}\big(\sum_i w_i \alpha_i,\; \sum_i w_i \beta_i\big)$. Blending Beta densities this way lands back inside the family, so intersection is one parameter average, no surrogate needed. BetaE learns the weights with an attention network; the companion freezes them uniform at $w_i = 1/n$ (its header says so plainly), which is the same formula with the attention replaced by the mean (clqa_models.py line 455).

Negation is the reason this model ends the Part's algebraic ladder. The map

$$\neg\,\mathrm{Beta}(\alpha, \beta) \;=\; \mathrm{Beta}\!\left(\tfrac{1}{\alpha},\, \tfrac{1}{\beta}\right)$$

swaps the regions of high and low density (parameters above 1 pile mass centrally; their reciprocals, below 1, pile it at the edges), and in log-parameter space it is nothing but a sign flip: $\ln(1/\alpha) = -\ln \alpha$, so $\mathbf{l} \mapsto -\mathbf{l}$ (clqa_models.py line 451). Applying it twice gives $-(-\mathbf{l}) = \mathbf{l}$, the identity, exactly: negation is an involution, as classical complement is ($\neg\neg A = A$). Volume 3 verified this on raw parameters; here the harness re-verifies it at the DAG level, on the trained model, bit for bit (clqa_models.py lines 698–701):

    # -- operator sanity: BetaE's ¬ is an involution at the DAG level (a
    #    chain r·¬·¬ embeds bit-for-bit as the chain r) ...
    assert np.array_equal(models["BetaE"]._embed(("bob", ("advises",))),
                          models["BetaE"]._embed(("bob", ("advises", N, N))))

With intersection and negation both closed, union comes free by De Morgan: $A \vee B = \neg(\neg A \wedge \neg B)$. Work it in parameter space for two branches with uniform weights. Negate each operand, $(\alpha_A, \beta_A) \mapsto (1/\alpha_A, 1/\beta_A)$; intersect, giving the average $\tfrac{1}{2}(1/\alpha_A + 1/\alpha_B)$ per slot; negate again, giving the reciprocal of that average:

$$\alpha_{A \vee B} \;=\; \frac{2}{\dfrac{1}{\alpha_A} + \dfrac{1}{\alpha_B}},$$

the harmonic mean of the operand parameters, and likewise for $\beta$. So BetaE is the first geometry here with an in-space union, no DNF required. Honesty requires the next sentence: in practice the DNF route is more accurate, and the original evaluation offers De Morgan as an alternative mode while reporting DNF as the default [3]; the companion follows the field and routes unions through dnf too (line 441). Closure is an algebraic achievement; it is not automatically an empirical one.

Distance cashes the slot table's promise. The Kullback–Leibler (KL) divergence between two densities $p$ and $q$ is $\mathrm{KL}(p \,\Vert\, q) = \int p(x) \ln \tfrac{p(x)}{q(x)}\, dx$, read: the average extra surprise incurred by modeling data that truly follows $p$ as if it followed $q$; it is zero exactly when the two densities agree and grows as $p$ piles mass where $q$ has little. Swapping the arguments changes the value, so KL is asymmetric, and BetaE exploits the asymmetry. Its distance is the KL summed over the $K$ slots, $\mathrm{Dist}(v; q) = \sum_{k=1}^{K} \mathrm{KL}\big(\mathrm{Beta}(\alpha_{v,k}, \beta_{v,k}) \,\big\Vert\, \mathrm{Beta}(\alpha_{q,k}, \beta_{q,k})\big)$, with the entity as the first argument so that an entity concentrated where the query has no mass is charged heavily [3]. The closed form and its digamma/trigamma partial derivatives were built from scratch in Volume 3 (beta.py lines 86–97 and 100–115); this module imports kl_beta and kl_grads and never re-derives them. One design-space note before the race: Betas are not the only closed geometry. Cone embeddings recover negation with a different object, sector-like cones whose complements are again cones, showing that closure under the full fragment is a property one can engineer into several geometries, not a unique trick of densities [5].

Trained identically: the controlled comparison

The race is only fair if the lanes are. All three models train on the same query bank, built from the observed graph alone: one 1p query per training edge (15 of them, clqa_models.py line 105) and every unordered pair of observed legs sharing a tail as a 2i query (4 of them, lines 109–112). BetaE alone additionally trains 15 2in queries, one per edge with a sampled negated leg (lines 115–130), because it is the only model with a $\neg$ to train. All three draw negatives from the same filtered pools: for each training query $q$, negatives come from $V - \mathrm{ALL}(q)$, the full entity set minus the query's complete answer set on the full graph (lines 136–138), so a held-out hard answer is never pushed away as a false negative; as in Volume 3, that filter is where generalization to hard answers comes from. One honest asterisk on "identical" before the loss: the lanes share the bank, the pools, and the protocol, but not the tuning. Each model keeps its own Volume-3-tuned margin, step size, epoch count, and dimension (GAMMA, LR, EPOCHS, and the two dimensions at clqa_models.py lines 88–92), and because BetaE alone trains the extra 2in queries, its negative draws from the shared pools diverge from the other two models' after the first epoch even though the pools themselves are identical. The dash pattern is independent of every one of these choices; the numeric cells are not, which is why the table-reading section below trusts them differently. All three minimize the same margin ranking loss form per (query, answer, negative) triple, each with its own margin $\gamma$,

$$L \;=\; \max\big(0,\; \gamma + d(\mathbf{v}^{+};\, q) \;-\; d(\mathbf{v}^{-};\, q)\big),$$

read: the true answer must sit closer to the query object than the sampled negative by at least the margin $\gamma$, or the violation is charged. When the hinge is active, $\partial L / \partial \theta = \partial d_{\text{pos}} / \partial \theta - \partial d_{\text{neg}} / \partial \theta$ for every parameter $\theta$ the query touches, where the symbol $\partial$ marks a partial derivative, so $\partial L / \partial \theta$ reads "how fast the loss $L$ changes when the parameter $\theta$ alone moves", and every one of those partials is written by hand: the norm subgradient and the min-mask routing for GQE (the mask is built in _embed_train, clqa_models.py lines 240–254, and applied at lines 256–273), the chain rule through the attention softmax, the offset min, $\sigma$, and softplus for Query2Box, where $\mathrm{softplus}(x) = \ln(1 + e^{x})$ is the smooth positive ramp that keeps the projection's width increments non-negative and the sigmoid $\sigma$ is exactly its derivative (lines 342–409, with the softmax Jacobian worked in the comments at lines 380–390), and the chain through the exponentials and the affine maps for BetaE (lines 467–513). The box-distance subgradients and the Beta-KL partials themselves are Volume 3's, derived in the Box Embeddings and Beta Embeddings chapters and imported here; this chapter does not repeat that work.

Hand gradients earn their keep only if they are checked, and the check here is stronger than re-deriving: the harness perturbs every parameter coordinate of a fresh model by $\pm 10^{-6}$, recomputes the step's own loss, and compares the central finite difference against the update the step actually applied, $(\theta_{\text{before}} - \theta_{\text{after}})/\eta$, where $\eta$ is the learning rate the step used, so the quotient recovers the gradient the step implied (_fd_audit, lines 549–578). Auditing the applied update, rather than a returned formula, catches a subtle class of bug: on a 2i query whose two legs share a relation, updating the shared parameter inside the loop would evaluate the second leg's gradient at an already-moved point. The committed verdict:

[5] gradient audit: each model's APPLIED step vs central finite differences
  ...
  GQE    max |implied − FD| = 5.98e-10   (tolerance 1e-05)
  Q2B    max |implied − FD| = 3.29e-10   (tolerance 1e-05)
  BetaE  max |implied − FD| = 1.80e-09   (tolerance 1e-05)

Training itself is unremarkable by design, which is the point of a controlled comparison. The committed loss traces (mean margin loss per epoch, snapshots at 1, 100, 500, and the final epoch):

[1] training: mean margin loss per epoch snapshot
  GQE    [1] 1.1849  [100] 0.1173  [500] 0.1853  [1000] 0.1372
  Q2B    [1] 1.8404  [100] 0.0209  [500] 0.0000  [1500] 0.0000
  BetaE  [1] 1.0130  [100] 0.0000  [500] 0.0000  [1000] 0.0000

Query2Box and BetaE drive the hinge to zero; GQE settles near 0.14 and stays there, jittering (its per-epoch renormalization of entities to the unit sphere, lines 275–279, keeps the margin loss honest but also keeps it from vanishing). One worked query shows all three generalizing rather than memorizing. The 2i query authored(alice, x) ∧ authored(bob, x) has the single answer p1, and the edge (bob, authored, p1) is held out, so traversal of the training graph returns the empty set; the committed run ranks p1 first for all three models:

[2] one worked 2i query: authored(alice, x) ∧ authored(bob, x) — hard answer p1
  ((bob, authored, p1) is held out: traversal of G_train finds NOTHING here)
  GQE    top-3: p1=-0.475  p2=-1.178  alice=-1.471   (p1 at rank 1 of 13)
  Q2B    top-3: p1=-1.760  p2=-4.052  alice=-4.586   (p1 at rank 1 of 13)
  BetaE  top-3: p1=-0.746  alice=-2.029  p3=-4.703   (p1 at rank 1 of 13)

Each model placed p1 inside the intersection object because its alice leg pulled p1 in during training and the geometry of intersection kept it there when the bob leg was added: the imputation the previous chapter demanded, performed three different ways.

The 14-row verdict

Here is the chapter's centerpiece, the committed payoff table: filtered MRR (Mean Reciprocal Rank, the average of one over each hard answer's rank, with all known answers filtered out of the candidate list) on each type's hard answers, under the previous chapter's exact protocol, with a dash where the geometry cannot express the type. Recall the previous chapter's naming grammar for the type codes: a leading digit counts repetitions, p is one projection hop, i an intersection, u a union, and n a negation attached to one operand of an intersection, letters read in composition order (so pni negates the intersection's projection-chain operand, pin its one-hop operand):

[4] the payoff table: filtered MRR on HARD answers, per query type
  (protocol = query_dag.filtered_mrr: rank vs the non-answers V − ALL, ties at
   expected rank; — marks a type the geometry cannot express)
  type   #hard       GQE      Q2B    BetaE  random(0)
  1p         2    0.6667   0.3333   0.5500     0.2250
  2p         2    0.2381   0.3500   0.3000     0.1010
  3p         2    0.1339   0.3333   0.1500     0.4167
  2i         2    1.0000   1.0000   1.0000     0.1125
  3i         2    1.0000   1.0000   1.0000     0.0909
  pi         2    0.4167   0.5833   0.3000     0.5625
  ip         1    1.0000   1.0000   0.5000     0.1000
  2u         2         —   0.6000   0.1500     0.1056
  up         2         —   0.2250   0.1339     0.2917
  2in        2         —        —   0.1214     0.1964
  3in        1         —        —   1.0000     0.2500
  inp        1         —        —   0.5000     0.0909
  pin        1         —        —   0.1000     0.1250
  pni        1         —        —   0.1429     0.1250
  pooled          0.6085   0.5794   0.4197     0.2128
                    (13)     (17)     (23)       (23)   ← hard answers each pool covers

Read it in three sweeps, in decreasing order of trustworthiness.

Sweep one: the dashes, which are theorems. GQE dashes exactly the two union types and the five negation types, seven cells; Query2Box dashes exactly the five negation types; BetaE dashes nothing. This is DASHES verified cell for cell against enforced raises (lines 633–643 and 681–683), and it is the one part of the table that transfers to any scale unchanged, because it is a statement about geometry, not about this graph: convex objects cannot hold unions in-space, bounded objects cannot hold complements, and reciprocal Betas hold both. The dash pattern is the chapter's thesis, printed by the experiment.

Sweep two: the shared rows, read honestly. Within the rows where models compete, every model beats the frozen random scorer on 1p (asserted, lines 685–688), all three are perfect on 2i and 3i, and the orderings wobble: GQE wins 1p, Query2Box wins 2p, 3p, and pi, and BetaE finishes second on every projection row but last on pi and ip. Each row pools one or two hard answers, so a single rank swing moves an entry by tenths, and the per-model tuning disclosed in the training section rides along inside every cell; random even beats all three models on 3p, and two of the three on pi, which is what an uninformed baseline can do when a row has two answers and thirteen candidates. These cells are a demonstration that the machinery works end to end, not a benchmark. The published anchor at field scale, with millions of queries and thousands of entities, is stable and worth citing as such: BetaE over Query2Box over GQE on average MRR across the EPFO types on the standard benchmarks [3]. The one reading of our negation rows that is fair at toy scale is existence versus absence: BetaE posts a number in all five, beats random on 3in, inp, and pni (computed and printed by the run; the harness asserts only that at least one such win exists, lines 690–696), and loses to random on 2in and pin; nothing else in the table has anything to say there at all.

Sweep three: the zero-shot rows, the real probe. The models trained on 1p and 2i shapes only (BetaE also 2in). Every other row, and especially ip and pi, which compose both trained operators in an order never seen, tests whether the models learned operators or templates: at inference, embed_query simply chains projection into intersection into projection as the DAG dictates, on structures no training example ever exhibited. GQE and Query2Box both hit 1.0000 on ip and post 0.4167 and 0.5833 on pi. That compositional transfer, an operator learned inside one structure working inside another, is the property that separates query embedding from mere query memorization, and it is the design intent of the field's train/evaluate split [2].

The pooled line needs its footnote said out loud, and the committed output says it: the three pools cover different type sets (13, 17, and 23 hard answers for GQE, Query2Box, and BetaE respectively), so the pooled numbers are not comparable across models; GQE's 0.6085 is an average over only the types a point can express. Compare rows, never the pooled line.

What "the query is the geometry" hides

Before Part V celebrates, one cost must be put on the record, because the next chapter's methods exist to pay it differently. The symbolic executor of the previous chapter materializes a set of entities at every internal node: evaluate advises(alice, y), and the node's output is literally the set containing bob, inspectable, auditable, wrong in ways you can point at. The query embedders materialize nothing of the kind. The intermediate object after the alice-leg projection is a point, a box, or eight Beta parameters; no entity set exists at that node, and no procedure in the model recovers one. If the intermediate geometry has drifted, placing the box where no advisee lives, the error is invisible until it surfaces as a wrong final ranking, with no proof and no audit trail identifying which hop failed. The failure is silent by construction: distances degrade gracefully, so a nonsense intermediate still yields confident-looking scores. This is the precise interpretability price of replacing execution with embedding, and it motivates an entire alternative line in which each node's output is a fuzzy set, a vector combined by a t-norm algebra chosen so that the logical laws hold again [6]. In the strongest form of that line the fuzzy vector is indexed by the entities themselves, one membership degree per entity, so every intermediate node once again names entities and the query algebra returns to being logic. That line is the next chapter's subject.

The unsolved part

The suite evaluates 14 query structures after training on two shapes (BetaE, three); each model answers every structure its geometry can express, and only BetaE answers all 14. That sounds like triumphant generalization until you ask what the real systems train on: not 19 or 34 queries but millions, sampled from the benchmark graphs across five to ten structures [3], plausibly because the learned operators do not compose reliably without saturating exposure. Two open problems hide in that number. The first is sample efficiency: an exact executor needs zero training queries, and a method that needs millions to approximate one is spending its budget on something; nobody has a tight account of what. The second is the query distribution itself: the sampling procedure decides which structures, which anchors, and which relations the operators get to practice on, and every reported MRR silently conditions on those choices, the same benchmark-design lesson the previous chapter flagged for the bank and Volume 3 flagged for link prediction. And beneath both sits this Part's most subversive question. Every query in the bank bottoms out in 1-hop atoms; the hard answers exist because single edges are missing. What if the only thing worth learning is a 1-hop link predictor, with all composition done by search over the DAG at inference time, no query training at all? If that works even competitively, then the millions of sampled queries were training a component that logic could have supplied for free. The next chapter runs exactly that provocation.

Why it matters

This chapter is Volume 4's cleanest specimen of a theme the whole book has been building: the algebra you can express is decided by the representation you choose, before any training happens. Points, boxes, and Betas are not three quality tiers; they are three different fragments of first-order logic wearing geometry, and the committed dash pattern is that statement made executable and asserted. For the two pillars, this is the neural mirror of Volume 2's expressiveness ladders (EL++ versus more expressive description logics: what you can say determines what you can decide); for Volume 5, the silent-intermediate problem named above is a trust problem in miniature, the first clear case in this book of a system whose answers can be scored but whose reasoning cannot be inspected, the exact gap that faithfulness and verification research on large-scale reasoners must close. When a future system claims to "reason over a knowledge graph in vector space", the questions this chapter installs are: which connectives does its geometry close, where do its dashes fall, and what would an audit of its intermediate nodes even look at?

Key terms

• Query embedding — answering a query DAG by embedding it bottom-up, each node's output a geometric object, and ranking entities by distance to the root object; the executor's set operators replaced by learned geometric ones.
• GQE (Graph Query Embedding) — the point generation: projection is a per-relation transform (here a translation), intersection a permutation-invariant DeepSets pooling, scoring by similarity to entity embeddings (cosine in the paper; the companion uses negative Euclidean distance); conjunctive fragment only [1].
• Permutation invariance / DeepSets — the requirement $\mathcal{I}(\mathbf{q}_1, \ldots, \mathbf{q}_n) = \mathcal{I}(\mathbf{q}_{\rho(1)}, \ldots, \mathbf{q}_{\rho(n)})$ for every reordering $\rho$, forced on intersection because set operands have no order, and the symmetric-pooling architecture family that satisfies it.
• Query2Box distance — $d_{\text{box}} = \lVert \max(\mathbf{0}, \mathbf{a} - \mathbf{o}) \rVert_1 + \alpha \lVert \min(\mathbf{a}, \mathbf{o}) \rVert_1$ with $\alpha = 0.2$: an outside term to the box surface plus a discounted inside term; imported verbatim from Volume 3's boxes.py.
• DNF rewrite — the theorem that every EPFO query rewrites so unions apply last, via $P_r(A \cup B) = P_r(A) \cup P_r(B)$ and distributivity; each conjunctive disjunct is embedded separately and the best disjunct score wins, exact at the aggregation step, at the price of multiplying branches (36 bank queries become 42 disjuncts) [2].
• BetaE negation — $(\alpha, \beta) \mapsto (1/\alpha, 1/\beta)$, a sign flip of log-parameters and an exact involution ($\neg\neg = \mathrm{id}$, asserted bit for bit on the trained model); the first closed negation in the ladder [3].
• Parameter-interpolation intersection — the weighted geometric mean of Beta densities is again Beta, with parameters $\sum_i w_i \alpha_i$ and $\sum_i w_i \beta_i$ (the exponent-adding derivation); with negation it yields an in-space De Morgan union, the harmonic mean of parameters.
• StructuralGap / dash pattern — the enforced raise when a geometry has no operator for a connective; the committed 7/5/0 pattern across GQE, Query2Box, and BetaE, asserted to match the structural theory cell for cell.
• Zero-shot structure — a query type never seen in training (here everything beyond 1p, 2i, 2in), answered purely by composing trained operators along the DAG; the field's probe for operator-level rather than template-level learning.

Where this leads

BetaE closed the algebra, but it closed it inside the black box: intermediate nodes are parameters, not entities, and the training bill runs to millions of sampled queries. The next chapter, Fuzzy and Training-Free CLQA, attacks both costs at once. Fuzzy-set methods make every intermediate node a vector of per-entity membership degrees combined by the t-norms of Part II, so the executor's inspectable sets return in soft form; and the training-free provocation planted above gets its experiment: take one trained 1-hop predictor, compose it symbolically over the query DAG with continuous optimization, and see how far down this chapter's table you can get without ever training on a single multi-hop query.

---

Companion code: examples/integration/clqa_models.py implements this entire chapter behind one interface: the shared banks and filtered negative pools (lines 97–138), the DNF rewrite (lines 175–191), GQE with its enforced raises (lines 197–279), Query2Box reusing Volume 3's boxes.py distance (lines 285–413), BetaE reusing Volume 3's beta.py KL machinery (lines 419–517), the declared-and-asserted dash pattern (lines 524–532, 681–683), the finite-difference audit of every hand gradient (lines 549–578), and the payoff table (lines 626–651). Run python3 examples/integration/clqa_models.py to reproduce every number byte for byte; the run ends with SUMMARY clqa_models: 1p gqe=0.6667 q2b=0.3333 betae=0.5500 random=0.2250 | dashes gqe=7 q2b=5 betae=0 | betae_neg_wins=3in/inp/pni | pooled gqe=0.6085 q2b=0.5794 betae=0.4197 | fd_max=1.8e-09.