IIUC, that's what sci-fi LED panels of really old computers were. They showed all the internal statuses of the CPU as well as CPU-RAM bus. And the toggle switches allowed individual bit overrides.

The operator sets a CPU RESET switch to RESET, then powers on the machine, and start toggling RAM address and data switches, like HHLL HLLH HHHL LLLL. The operator then press and release the STEP push switch. The address 0b 1100 1001 is now set to 0b 1110 0000. This is repeated until the desired program or a bootloader is all complete. The operator finally sets CPU RESET to Normal, and CLOCK dial to RUN.

The CPU exits reset state, initializes program counter with reset vector, e.g. 0b1000, and start executing instruction at PC++. 1000, 1001, 1010, so on. Then oh no, the EXCEPTION indicator comes on, the LED shows 0b 1110 0000. That's divide r0 by 0, etc.

They didn't actually spend every half a day toggling those switches. They loaded their equivalents of bare minimum BIOS recovery code, then the rest wad loaded from magnetic or mechanical tapes. Only when computers were booted up blank slate or crashed and in need of debugging, the users resorted to that interface.

If they had the CPU-RAM main bus split into ROM and RAM address ranges in such ways that setting address to reset vector will yield the first byte of a BIOS program lithographically etched into the ROM chip, then simply powering on the machine will do the same thing as loading the BIOS manually.

There were also things like magnetic core memories. They didn't require lithography to fabricate, and there were both ROM and RAM kinds of those.

Of course, if you have sufficiently simple input devices that could do DMA, then you can do something e.g. IBM 1401 did:

    When the LOAD button on the 1402 Card Read-Punch is pressed, a
    card is read into memory locations 001–080, a word mark is set
    in location 001 to indicate that it is an executable instruction,
    the word marks in locations 002-080 (if any) are cleared, and
    execution starts with the instruction at location 001. [...] To
    read subsequent cards, an explicit Read command (opcode 1) must
    be executed as the last instruction on every card to get the new
    card's contents into locations 001–080.
I imagine the additional wiring on that LOAD button must have been pretty small: the READ functionality already exists in the 1402 device, it also has an output signal that tells when the read is finished (so the 1401 Processing Unit knows when the Read command is done), so you just need to tie that signal into resetting the PC to 1 and then starting the clock.

The pictures of workers weaving a program into core rope memory is peak steampunk vibes.

Weaving and computing are closely intertwined