> Also why can't I have my memory back when it's not in use in tightly packed systems?

That is changing as we speak:

[1] https://openjdk.org/jeps/8359211

[2] https://openjdk.org/jeps/546

[3] https://openjdk.org/jeps/8350152