I'm worried that the middle could fall out of the computing market across the board. If you can afford to keep up with the upgrade treadmill, you'll get private, local inference capabilities. If you can't afford to stay on the treadmill, you'll be stuck with whatever cloudshit malware Silicon Valley wants to foist on you.
I acknowledge that this is already the case, to some extent. The cheapest laptops at Best Buy are crammed with the most preinstalled malware. That's been the case for, what, 25 years? But you've always been able to wipe that cheap laptop and make it into a much more capable, trustworthy machine.
Well, assuming LLMs do become a pervasive part of the computing experience, what happens to the cheap laptops? Do all computers get more expensive to accommodate local inference? Does the rift between the everyday user's experience and the savvy user's experience grow even wider than it already is? Neither outcome seems good for the average joe who just needs to check his email.
Remember how in like 1999/2000 Sun was trying to predict that everyone's computer would be some form of thin terminal in the future? Turns out they were very wrong on the part about it running on Sun server back-end infrastructure, but that same general purpose has now been accomplished through other methods where a lot of people do basically EVERYTHING inside a web browser tab to some external cloud service.
Now add the need for external inference because very few random consumers are going to buy a $3000 laptop when they can get the $600 laptop at Best Buy, and that trend further escalates.
This has always been the case, forever. You have to pay for a product or service. How you do so can be with cash or your data/body/vote/eyeballs/indirect discretionary purchases.
The amount of work that can be done funded by foundations and free work is nowhere close to what people want.
Oh give me a break. This argument that people's objections to advertising comes from some Pollyanna naivety over things being free is nonsense. Tell me the last time you have ever seen a company be upfront and explicitly offer a free tier where they tell you upfront exactly what they are harvesting about you and for what purpose (and no, "improving user experience" isn't being upfront) but also offer you a paid version where they explicitly promise not to do that.
"If the product is free then you are the product" is supposed to be a cautionary observation, not an axiomatic proscription.
Does anyone actually trust companies that offer paid services to not harvest their data? Whenever I see this trope I think to myself that whatever lack of regulation, oversight and enforcement lead to that being okay would equally allow for them to both take my money and harvest all my of data anyways.
This is akin to people who criticise socialised medicine or services and pejoratively characterize supporters as just wanting "free stuff", as if the concept of collective payment is naive or something