Maybe there are issues I'm not aware of but using dockcross has made cross-compilation quite easy in my experience.

https://github.com/dockcross/dockcross

How does it handle .so version differences and glibc version differences between the container and the target system?