TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs

Diffusion language models write by cleaning up a field of noise rather than producing one word after another. Their reasoning changes at every cleaning round, so truth-checking tools that only scan the finished text miss subtle factual mistakes.

We track how words connect to each other throughout the whole unmasking routine, catching hallucinations as they emerge, drift, or lock into place.

Why this was hard

In diffusion language models, factual errors are not static across decoding steps. A model may draft nonsense, drift into plausible falsehood, or repair itself just before the final output. Single-snapshot detectors read only the final text or one isolated middle step, so they cannot separate temporary incoherence that resolves into truth from a confident error that solidifies early. Output-based measures such as semantic entropy need several full generations to measure agreement, which multiplies latency fivefold on an iterative model. Catching this properly needs a method that reads the whole trajectory in one pass and isolates how token dependencies evolve.

How it works

TDGNet is a temporal dynamic graph architecture that monitors factuality across the reverse diffusion process with no extra sampling. At each diffusion step \(t\), we take the bidirectional attention matrix from the final transformer layer and drop every edge below a sparsity threshold \(\tau\). That forms an attributed graph snapshot \(G^{(t)} = (V, \mathcal{E}^{(t)}, X_V^{(t)}, X_E^{(t)})\), where vertices are sequence tokens and directed edges are attention weights above the threshold. Node features \(h_i^{(t)}\) come from projecting the model's 4,096-dimensional hidden state down to 128.

A message passing network aggregates neighbour features at each step:

\[\bar{m}_i^{(t)} = \frac{1}{|\mathcal{N}^{(t)}(i)|} \sum_{j \in \mathcal{N}^{(t)}(i)} \psi(h_j^{(t)}, h_i^{(t)}, e_{ji}^{(t)})\]

where \(\psi(\cdot)\) is a learnable MLP and \(e_{ji}^{(t)}\) the scalar attention weight. Each token's incoming message and its prior memory \(s_i^{(t+1)}\) go into a gated recurrent unit, whose running memory \(s_i^{(t)}\) records whether token relationships stabilise or drift as denoising runs from \(t = T\) down to zero.

Rather than weighting every diffusion stage equally, a temporal attention mechanism computes a trajectory representation \(z_i = \sum_{t=0}^{T} \alpha_i^{(t)} \cdot \text{Linear}(s_i^{(t)})\), with \(\alpha_i^{(t)}\) a softmax over a learnable query vector \(q\). For efficiency the recurrence is evaluated at keyframes \(\{T, \lfloor T/2 \rfloor, \lfloor T/4 \rfloor, 0\}\). Response-level classification mean-pools all token vectors and trains under binary cross-entropy; span localisation sends individual \(z_i\) straight to a token head.

We tested on LLaDA-8B-Instruct and Dream-7B-Instruct across five question-answering benchmarks including CommonsenseQA, HotpotQA and TriviaQA. On LLaDA-8B the detector reaches 0.68 AUROC at response level, beating latent baselines such as TSV by up to 8.5% in a single pass with 160ms of overhead per query. On token-level localisation over Natural Questions it reaches 0.87 AUROC, outperforming static graph methods by pinpointing the exact hallucinated span.

What it does not do

The detector struggles when a model locks into a confident misconception at the very first step and never explores an alternative. In those static failures the attention graph shows no drift at all, so detection falls back on subtle attention misalignment against the prompt rather than any dynamic signal. Removing graph connectivity entirely and relying on isolated token activations collapses performance to near chance on every benchmark, which is the clearest evidence that the structure is doing the work.

Accepted to the EMNLP 2026 main conference. Preprint on arXiv.