The proposed CAHSR business plan calls for 2 peak hourly trains plus 2 additional trains per day stopping nowhere between the Bay Area and Los Angeles. 2 of these are suggested to be actual non-stop SF to LA Union Station, even though bypassing San Jose is pretty crazy. But anyway you are right: it's no problem at all.
There are also plenty of population centers between SF and LA which is why trains are going to stop in Fresno and Bakersfield (combined pop: 2.2 million). Also Palmdale. If Palmdale is nowhere then Brightline West is also a train to nowhere.
Alephnerd is making the same mistake that many nerds have made. They are arguing about the existing passenger, while the point of the project is to serve the next ten million Californians.