Mathematics is the FORTRAN of the real source. Closer to a real source is probably "real" things like atoms and other universal things.

If I remember correctly, Stargate-SG1 at one point had some ideas about this sort of universal language, that multiple species could use for communication, as any sufficiently intelligent specie probably been able to see atoms and so on, but may have completely other way of doing "math-like" stuff.