Attention Sinks Perform Two Distinct Algorithms Requiring Complementary Interventions
Attention sinks—where attention concentrates on a single token in transformers—are not a monolithic phenomenon but reflect two fundamentally different computational mechanisms: adaptive nop (suppressing updates via null routing) and broadcast (aggregating and redistributing global information). Existing interventions like gating and register tokens implicitly target only one mechanism each, explaining why neither alone is sufficient; combining them yields complementary performance gains. Diagnostic signatures (negligible value norms for nop, low-rank outputs for broadcast) enable practitioners to identify which mechanism is operating in their models.