It's just that at some point when it's all physically wired up in hardware as opposed to being stored in some form of memory I have difficulty thinking of it as code or a program. By the time you're rearranging wires to enter a "program" aren't you actually refactoring the CPU itself?

Anyway I feel like the answer to the chicken and egg problem originally posed is to point out that things used to be different. Tools such as text editors and compilers are merely modern syntactic sugar.