Depends on your risk tolerance.
I personally wouldn't trust a machine if a container was exploited on it, you don't know if there were any successful container escapes, kernel exploits, etc. Even if they escaped with user permissions, that can fill your box with boobytraps if they have container-granted capabilities.
I'd just prefer to nuke the VPS entirely and start over than worry if the server and the rest of my services are okay.
Yea I feel that too.
there are some well respected compute providers as well which you can use and for very low amount, you can sort of offload this worry to someone else.
That being said, VM themselves are good enough security box too. I consider running VM's even on your home server with public facing strategies usually allowable