How will they make money on their product exactly? To the tune of being worth nearly a trillion dollars? There is no guarantee that inference will go down, we’ve seen some improvement with cheap models, but they aren’t what people want, and otherwise models stay expensive to run and use
Inference is already profitable (training is not)
So what. In a highly competitive industry they can't keep selling inference unless they continually train better models. It's like saying my airline is profitable if you don't count the cost of buying new airplanes.
[citation needed]
OpenAI have made this claim and maybe it is with API pay-per-use (there's also good evidence eveb that is not if you dive into how much a rack of B200s cost to operate), but I'd be very sceptical that the free, $20 or $200 a month plans are profitable.
Then the questions are if the market will bear the real cost and if so how competitive OpenAI are with Google when Google can do what Microsoft did to Netscape and subsidize inference for far longer than OpenAI can.