But only as long as training actually improves models significantly. As soon as those improvements stay below a certain threshold, the better move is to invest your R&D money into other things like harnesses or new tricks one can play with existing models and the immense cost of training is just not worth it to be 0.5% ahead.
I'm absolutely certain that we will reach that point, just not when. Could come sooner than we think though.
Don't get me wrong I still want the models intelligence to improve, but for all practical purposes we are already there this is why many people are already moving to cheaper/open source. There is still a case though for the 1% of queries that demand SOTA
At that point they would lose all advantage stemming from their ability to boil the ocean though.