I don't know much about militaries or nuclear reactors, but I know that reactors are used in some submarines and in some aircraft carriers -- situations where you want a vessel to to remain at sea for long periods of time without refueling, and weight is not a primary concern.
That's pretty niche, though. Think about trucks, tanks, aircraft, generators for outposts, etc. It might be cool if you could safely package a zillion nuclear reactors for those use cases, Terminator style, but I'd guess that reactors are a better fit for centralized, permanent power generation.
The Aircraft Reactor Experiment https://en.wikipedia.org/wiki/Aircraft_Reactor_Experiment, yielding 2.5 megawatts, was about two meters tall and one meter in diameter; the fuel was 15kg of U-235, but I think the reactor as a whole may have weighed several hundred kg. (It couldn't have been more than about 40 tonnes, just because no material would be dense enough, but I think it was much lighter than that.
Oh, here it is: https://media.githubusercontent.com/media/openmsr/msr-archiv... page 24/139 budgets 48400 "pounds" for "reactor and reactor shield", which is 22 tonnes, about 110kW/tonne.
The smallest nuclear submarine was NR-1, which had a total displacement of 400 "tons": https://en.wikipedia.org/wiki/American_submarine_NR-1 so the reactor must have weighed less than that.
The 10MW version of SSTAR was supposed to weigh 200 tonnes, 50kW/tonne, while the 100MW version was 500 tonnes, thud 200kW/tonne: https://en.wikipedia.org/wiki/Small,_sealed,_transportable,_...
A 4.95-kg americium thermal reactor design outline has been published: https://www.researchgate.net/publication/239521070_The_Small...
Arleigh Burke class destroyers have 80MW of propulsion and displace 8300 to 9700 tons, compared to which the SSTAR's 500 tonnes is almost insignificant. So weight isn't an issue for ships, and weight doesn't get ridiculously high until you're down below the megawatt scale.
So, you may be right that existing proven reactors won't scale down to a single truck or tank. There isn't a known physical reason it's impossible, or even impossible to do safely, but it hasn't been achieved.
Probably you are right that many small reactors would be more dangerous, but warships exist so that they can go into dangerous situations. You have to weigh the risk of a reactor problem against the risk of being unable to fight because you have no fuel. And we've certainly seen that many militaries have little concern for sailor safety.
Despite all this, no navy has switched all their ships to nuclear fuel. The only explanation I can come up with is that it's unsustainably expensive.