Yeah, I know they exist in lower layers. Though layers being mostly deterministic (hardware glitches aside) I think they are relatively easy to rely on. Whereas LLMs seem to have an element of intentional randomness built into every prompt response.