Rather than "on/off" I think we need to distinguish between at least four things:
1. Connected naively to the internet.
2. Behind a hardened VPN endpoint which is on the internet.
3. Has a separate physical private network.
4. Requires physical access.
I think it's obvious that #1 should be prohibited in favor of #2. After that point we need to ask what the impact is of a Denial of Service attack that prevents anyone from remotely accessing the system.
The difference between #2 and #3 may depend on whether things could be Very Bad if the system is disconnected at a time of the attacker's choosing. For example, disabling access to flood-control valves during a hurricane.
The big question for 2 is what devices have access. If it’s a bunch of employees from loosely managed general use laptops, bad. If it’s a few computers at HQ that are totally locked down and without internet access, probably ok.