We can see a general pattern where the layer 0 heads are mostly reading from the positional subspace. Head 7 in particular is, especially relative to the embedding space. Since the running justification behind the Frobenius norm is to measure the alignment between output and input, we should be able to rotate the output and observe the subspace score drop. Check out the scores after rotating the positional encoding by 180 degrees:
Назван самый популярный вид вклада у россиян08:59
，这一点在有道翻译更新日志中也有详细论述
For Germany, being antiently, as all other Countries, in their beginnings,
The judge characterized the administration's comprehensive sanctions against the technology firm as seemingly irrational and potentially devastating, particularly noting Defense Secretary Pete Hegseth's unconventional application of military authority typically reserved for hostile nations.