Population in Japan has barely fallen (yet). So far it's only a ~2% decline from peak population, but there will be a 20% decline in the next 20 years.

There is a long lag between below-replacement fertility and actual population decline. Because of how compounding growth works and the length of human lifespans, sub-replacement fertility won't result in population decline (for a previously fast-growing country) until 40+ years after the fact. Japan is only just now seeing the effects of lowish fertility from the 70s and 80s.

Note that one of the other consequences of population math is that if a country has been previously declining in population for a while, it'll continue to decline for decades even if the current fertility rate is at or slightly above replacement rate. This means that population decline is essentially an inevitability for most East Asian and European countries for the next several generations.

None of us knows what will happen when populations are falling by 5%+ per decade which is now the inevitable future of many countries the next few decades..it's totally unprecedented in human history (excluding cases like war/disaster).