ld writes to the GOT. The executable segment where .text lives is not written to (it's position independent code in dynamic libraries).

ASLR is not an obstacle -- the same exact code can be mapped into different base addresses in different processes, so they can be backed by the same actual memory.