Simple languages are not necessarily easy languages to understand. See: brainfuck, APL, K, etc.

I think the problem with assemblers in particular is that the canonical definition is the byte code, not the human readable text. x86 is particularly annoying because no one agrees on the syntax of that text, there are hundreds of mnemonics, things are constantly being updated, and the practicing assembly programmer cares deeply about the execution semantics of the microarchitecture more than the specific sequence of instructions. Some of the language is also completely foreign to higher level programmers, like instruction pipelines, uops, instruction latency, and so on.

Rarely do you sit down and write this assembler by hand, you compile some C code and poke at it with vtune/uprof to measure hot sections of the code, break those down, and implement faster versions. It's fundamentally an iterative, experimental process.