Process View

All the other views look at a trained model and ask “what mechanisms are in here?” The process view asks a different question: “how did these mechanisms get here?” It treats the training trajectory as part of the mechanism’s description — when it formed, what had to form first, whether it appeared gradually or as a sudden phase transition.

This matters because the same final circuit can arise from different formation processes, and the formation process tells you things the final state does not.

Thesis

A mechanism is a process that forms over training, individuated in part by its formation history: when it forms, what it depends on, what must precede it.

What it explains

Grokking. The generalization mechanism exists in the weights long before it appears in behavior. The phase transition from memorization to generalization is a formation event — one that static views looking only at the final checkpoint cannot see. The process view makes the delayed generalization into a core part of what the mechanism IS, not just a historical curiosity about how it was found (Nanda et al., 2023).

Induction head emergence. Induction heads appear suddenly at a specific training step, not gradually. This suggests a phase transition rather than gradual construction. The formation has a prerequisite: previous-token heads must form first. These developmental dependencies — mechanism B cannot form until mechanism A exists — are invisible from any static view (Olsson et al., 2022).

Why the same final circuit can be different mechanisms. Two models might converge to the same circuit (same subspace, same gauge orbit) by different training paths. The process view counts these as different mechanisms because the formation process, not the final circuit, is the object it individuates. The disagreement with the static views is over which object is the mechanism, and no further experiment settles it — a declaration does.

What this view says

The mechanism at time $t$ is the state of the process at $t$ , not the static object it converges to. What “exists” under this view includes checkpoint trajectories, formation events, and developmental prerequisites.

Two mechanisms are the same if they share the same formation process: same formation event type, same developmental prerequisites, same formation criterion timing. Cross-seed identity requires that the formation event occurs at the same relative training step, in the same order relative to other events, with the same prerequisites.

The process view pairs readily with a static view (usually subspace or structural) — the static view says what the mechanism is at any given checkpoint, and the process view says how it came to be. It is a view in its own right, with its own ontology and its own identity criterion, and it constitutes a family of its own in the paper’s partition by failure mode.

When it works and when it doesn’t

The process view is needed when the phenomenon is inherently dynamic: phase transitions, developmental prerequisites, formation order, delayed generalization. For these cases, no static view can express the relevant claims.

Computationally expensive. Requires checkpoint analysis across many training runs with many saved checkpoints. This is impractical for large models where each checkpoint is tens of gigabytes.

Formation criterion ambiguity. Different criteria — behavioral threshold, AGOP (average gradient outer product — a matrix summarizing the gradient structure of the learned function) convergence, weight-space structure — can give different formation times for the same mechanism. The formation criterion must be stated explicitly, and the choice changes the claim.

Overreach. Not every mechanism needs a developmental explanation. Describing the training trajectory is informative when the phenomenon is dynamic, but adding “it formed at step 5000” to a mechanism description does not always add explanatory power. The process view is needed when the phenomenon is inherently dynamic, not by default.

Technical details

Evidence

Checkpoint analysis: subspace estimates (DAS or AGOP) measured across training steps
AGOP trajectories: convergence to eventual causal subspace — empirically observed to precede behavioral detection in some cases (Nanda et al., grokking), but whether this generalizes to arbitrary mechanisms is not established
Phase transitions: sudden changes in IIA, composition score, or behavior
Knockout retraining: train a model with a proposed prerequisite component permanently ablated (a training-time intervention, not inference-time ablation) — if the target mechanism then fails to form, the prerequisite is established; if it forms anyway via a different route, the prerequisite claim is falsified

What it lets you prove

Formation order: mechanism A forms before mechanism B across seeds
Developmental prerequisite: ablating component X during training prevents mechanism Y from forming
Lead time: AGOP convergence precedes behavioral detection by $\Delta t$ steps

Formalism

Dynamical systems on weight space, AGOP, training-time subspace estimation. The formation event corresponds to a trajectory crossing a boundary in mechanism space $\mathcal{M}$ .