What are we missing?
Section titled “What are we missing?”Mechanistic interpretability has produced detailed accounts of induction heads, indirect object identification, greater-than comparison, and a handful of other circuits. How much of a model’s behavior does this cover? The Apollo Research “complete model understanding” project (#31, #32) calls the unexplained fraction “dark matter” — model behavior that has no known mechanistic explanation. The honest answer is that known circuits explain a small fraction of what even GPT-2 Small does.
But “how much do we understand?” is not a well-posed question without specifying resolution. At a coarse resolution, you might say “we understand that layer 0 does token embedding, layers 1-5 build contextual representations, and layers 6-11 do task-specific computation.” That’s nearly 100% coverage at the resolution of “what does each layer group do?” Zoom in and ask “what does each attention head do?” and coverage drops. Zoom in further — “what does each feature in each head do on each input?” — and coverage is effectively zero. Both “we understand most of it” and “we understand almost none of it” are correct at different resolutions.
This is compounded by the distinction between capability and propensity. Steinhardt (2026) argues that the field measures capabilities (what the model can do at peak performance) but not propensities (what the model tends to do across a distribution of inputs). A model might have the capability to answer truthfully (there exists a mechanism for truthful recall) while having a propensity to confabulate (the mechanism fires inconsistently). Circuit analysis that shows “this circuit does truthful recall when active” is a capability finding. Whether the circuit actually fires when it should is a propensity finding — and almost no current work measures propensity.
The view territory
Section titled “The view territory”Resolution-dependence is the defining feature of the Stratified view. The Stratified view says: mechanisms exist at multiple resolutions simultaneously, and different resolutions reveal different structure. An attention head is a mechanism at one resolution. The QK circuit within that head is a mechanism at a finer resolution. The individual feature interactions within the QK circuit are mechanisms at a finer resolution still. Completeness is resolution-relative — always “complete at resolution X,” never just “complete.”
The capability-propensity distinction maps onto the Instrumental vs. Role boundary. Capability findings are Instrumental: this mechanism can produce this behavior. Propensity findings require at least Role-level evidence: this mechanism is the one that typically produces this behavior in the model’s normal operation, not just when experimentally activated.
Mechanistic validity impact
Section titled “Mechanistic validity impact”| Criterion | Current status | What’s missing |
|---|---|---|
| V5 Scope declaration | Unknown — no systematic measure of how much behavior is explained | Requires resolution specification: “complete at which granularity?” |
| I3 Minimality | Incomplete — mechanism boundaries depend on resolution | The same mechanism has different boundaries at different resolutions; current work doesn’t specify which |
| C1 Falsifiability | Confused — capability and propensity are conflated | ”This circuit does X” conflates “can do X” with “tends to do X” — different operationalizations with different validity requirements |
| I4 Specificity | Untested at most resolutions | A mechanism that is specific at one resolution may be non-specific at another |
| I3 Minimality | Resolution-dependent — finer resolution means less parsimonious | There’s a natural Pareto frontier between resolution and parsimony that the field hasn’t characterized |
Resolution status: Scoped
Section titled “Resolution status: Scoped”“How much do we understand?” is ill-posed without specifying resolution. A published circuit can recover most of a task metric while leaving the computation underdetermined at finer grain. This is Stratified-view territory: the interesting objects are the stable strata and the structure that survives coarse-graining. The capability-propensity distinction (what the model can do vs. what it tends to do) maps onto the Instrumental vs. Role boundary and requires view-specific evidence at each level.
Sources
Section titled “Sources”- Apollo Research (2024) #31, #32: Complete model understanding, “dark matter” (45+ MI Projects)
- Steinhardt (2026): The case for evaluating model behaviors
- Sharkey “Sparsify” (2024): Hierarchical abstraction, decompilation fidelity (Sparsify agenda)
- Sharkey et al. (2025) §3.5: Microscope AI — extracting latent knowledge (arXiv:2501.16496)
- Nanda (2022) §9: Learned features — functional types, universality, completeness (200 Open Problems)