The tax office can decide that the company is, in fact, a German company if the founder lives and operates in Germany. Now the company paperwork has to be done in two countries. I don't know what the tax implications are.
Applies to all EU countries btw, you can't just choose a different jurisdiction for your new company risk-free.
The tax office WILL 100% decide you have to pay corporate taxes in Germany - and it does not need to decide that in the first place, you will have to report it proactively on your own to the tax office and file for taxes.