Hard disks used to come with a paper label listing their bad sectors. (They still come with a list, it's just now it is hidden in the firmware like you describe.)

I do some retro-computing every now and then and of these old drives that still work (about a 50% chance), when I format them assuming all sectors are good, they will inevitably report the same bad sectors as is on the label.

I don't really know why "bad sectors are still bad after 40 years" is so surprising to me, but there you go.

There's a step which "flashes" the magnetic structure on the disks during manufacturing which also allows drives to navigate the disk surface and "see" individual sectors. The bad sectors list from factory shows the points where this process has failed due to bad coating or other anomalies.

So they are not worn out sectors, they were bad (or not there) from the start. You can't rub them out, because there is not enough magnetic flux or material to store data at that address.

I remember someone writing a rant against having the list in firmware and managed automatically by the drive. If the list was on the host side, your software could route around the bad sectors and minimize the total amount of seek time during a program run. But with the drive managing bad sector redirection, you could never predict read latency since any read could get re-routed to a spare sector by the drive, adding latency outside your control. Everything still became automatic though.

You could buy disk packs with no bad sectors back then if you wanted that ("bit error free"). They cost extra but the premium wasn't all that high. Of course they could still develop bad sectors later.

And the firmware remaps them, as far as I know. That's very different from a firmware exposing a bad sector list to the host, which is the case for some flash systems, I believe.

Disk firmware follows sectors which are hard to write, flags them as "current pending sector". Then reallocates that sectors data to one of the reserves when it has time and increases "Reallocated Sector Count", and if it deems the sector dead or unusable, it's get blacklisted and "Offline Uncorrectable" count is increased as well, from what I remember.

This is all transparent from the host, maybe an occasional "wait a little, I'm kinda busy" signal to the controller. Same for flash, but it's way faster and more transparent, since the real map inside the flash controller is never linear, anyway.