The problem source is that the "cost" of tokens are taken at face value from business that are losing money at record speeds. E.g. https://artificialanalysis.ai says "doing task A costed us $10 using OpenAI", and that is the "cost" the OP used as basis for "tokens are cheap". Meanwhile OpenAI is losing $19 for each $1 in revenue... So right now OpenAI should be charging around $200 to do task A just to break even, but that would mean their use base would collapse.
There are two different things occurring here.
One is how much does it cost OpenAI to train the model.
The other is, if I stole OpenAI's model how much would it cost for me to run it?
R&D costs versus operational costs. Operational costs are very likely profitable. R&D is catastrophically expensive currently.
There is also the cost of the inference hardware that gets ignored because they already have it from training.
The main problem with the scenario of just doing inference is it relies on nobody else training models better than yours. As long as people are training private models that are better than yours, just inference isn't a viable business model.