The architectures are high level concepts and the mechanics usually have enough detail for you to try and implement.

Transformers are very "mendable" in that you can permute the architecture in crazy or random ways, and still basically always end up with get a coherent LLM. The difference comes down to training efficiency, inference efficiency, and usually minor differences in performance.

Hyperparams and stuff, I mean it's standard to do a sweep anyway.