> "they can just come eat your lunch once it looks yummy enough. Tread carefully"
True, and worse, they're hungry because it's increasingly seeming like "hosting LLMs and charging by the token" is not terribly profitable.
I don't really see a path for the major players that isn't "Sherlock everything that achieves traction".
Thanks for introducing me to the verb Sherlock! I'm one of today's lucky 10,000.
> In the computing verb sense, refers to the software Sherlock, which in 2002 came to replicate some of the features of an earlier complementary program called Watson.[1]
[1] https://en.wiktionary.org/wiki/Sherlock
But what’s the future in terms of profitability of LLM providers?
As long as features like Study Mode are little more than creative prompting, any provider will eventually be able to offer them and offer token-based charging.
I think a few points worth making here:
- From what I can see many products are rapidly getting past "just prompt engineering the base API". So even though a lot of these things were/are primitive, I don't think it's necessarily a good bet that they will remain so. Though agree in principle - thin API wrappers will be out-competed both by cheaper thin wrappers, or products that are more sophisticated/better than thin wrappers.
- This is, oddly enough, a scenario that is way easier to navigate than the rest of the LLM industry. We know consumer apps, we know consumer apps that do relatively basic (or at least, well understood) things. Success/failure then is way less about technical prowess and more about classical factors like distribution, marketing, integrations, etc.
A good example here is the lasting success of paid email providers. Multiple vendors (MSFT, GOOG, etc.) make huge amounts of money hosting people's email, despite it being a mature product that, at the basic level, is pretty solved, and where the core product can be replicated fairly easily.
The presence of open source/commodity commercial offerings hasn't really driven the price of the service to the floor, though the commodity offerings do provide some pricing pressure.
Email is pretty difficult to reliably self-host though, and typically a PITA to manage. And you really don’t ever want to lose your email address or the associated data. Fewer people could say they properly secure, manage and administer a VPS on which they can host the email server they eventually setup, over say a 10yr period.
Most people I saw offer self-hosted emails for groups (student groups etc), it ended up a mess. Compare all that to say ollama, which makes self-hosting LLMs trivial, and they’re stateless.
So I’m not sure email is a good example of commodity not bringing price to the floor.
We can assume that OpenAI/Anthropic offerings are going to be better long term simply because they have more human capital, though, right? If it turns out that what really matters in the AI race is study mode, then OpenAI goes "ok let's pivot the hundreds of genius level, well-paid engineers to that issue. AND our engineers can use every tool we offer for free without limits, even experimental models". It's tough for the small AI startup to compete with that, the best hope is to be bought like Windsurf