Understand causal self-attentionQueries score every key, a causal mask hides later tokens, and softmax turns the remaining scores into weights that mix values into one contextual vector per token.
Give a vector position without changing its lengthRoPE rotates each pair of features by an angle that grows with token position, so a query–key dot product reflects how far apart two tokens are while every vector keeps its length.
Put RoPE inside a complete attention layerProject tokens into per-head queries, keys, and values, rotate only the queries and keys, apply causal attention, then join the heads back into [B, T, E].
Reuse the past while producing one new outputKeep every earlier token's key and value. Each new token's query scores that growing cache, so nothing from earlier steps is recomputed.
Wrap projections and verify cached decodingOne module serves two paths: a full causal pass over a whole sequence, and single-token calls that reuse cached keys and values. Comparing them on the same input confirms they agree.