Yes I think you have to, to a extent the same also applies to dedicated servers. Even if you own a server that you place in a Colo, they can still pull your drives or plug in a KVM.

If you're data is sensitive encrypt it locally and send it. The reality is most people are running something like a website, API or a SAAS and basically just have to have a provider they trust somewhat and take reasonable security precautions themselves. Beyond that it's probably not as secure as it could be unless it's in a facility you own or control access to.

That's correct. I wouldn't think of it as a VM (a container) though but rather as a server which happens to be virtual. Yes, that's literally just a different word for the same thing but the different emphasis affects thought patterns. For all intents and purposes, from the buyer's perspective, a VPS is a small server, not a different type of thing.

It's true you shouldn't put super sensitive data on a VPS because the host could access it. Regular sensitive is fine - your host will be in a world of trouble if they access your data without permission, so you can generally trust them not to read your emails or open your synced nudes. But if your data is so sensitive that the host would risk everything to read it, or would avoid getting in trouble at all (e.g. national security stuff) then absolutely don't use a VPS. For that level of paranoia you'd need at least a dedicated server which makes it unlikely the host has a live backdoor into the system, ideally your own server so you know they don't, and for super duper stuper paranoid situations, one with a chassis intrusion switch linked to a bag of thermite (that's a real thing).