An acquaintance of mine has a start-up in this space and uses OpenAI to do essentially the same thing. This must look like, and may well be, the guillotine for him...
It's my primary fear building anything on these models, they can just come eat your lunch once it looks yummy enough. Tread carefully
No disrespect to your acquaintance, but when I heard about this, I didn't think "oh a lot of startups are gonna go under", I thought "OAI added an option to use a hard-coded system prompt and they're calling it a 'mode'??"
This is step 1. Like they say in the post, they will learn from the data.
> "they can just come eat your lunch once it looks yummy enough. Tread carefully"
True, and worse, they're hungry because it's increasingly seeming like "hosting LLMs and charging by the token" is not terribly profitable.
I don't really see a path for the major players that isn't "Sherlock everything that achieves traction".
Thanks for introducing me to the verb Sherlock! I'm one of today's lucky 10,000.
> In the computing verb sense, refers to the software Sherlock, which in 2002 came to replicate some of the features of an earlier complementary program called Watson.[1]
[1] https://en.wiktionary.org/wiki/Sherlock
But what’s the future in terms of profitability of LLM providers?
As long as features like Study Mode are little more than creative prompting, any provider will eventually be able to offer them and offer token-based charging.
I think a few points worth making here:
- From what I can see many products are rapidly getting past "just prompt engineering the base API". So even though a lot of these things were/are primitive, I don't think it's necessarily a good bet that they will remain so. Though agree in principle - thin API wrappers will be out-competed both by cheaper thin wrappers, or products that are more sophisticated/better than thin wrappers.
- This is, oddly enough, a scenario that is way easier to navigate than the rest of the LLM industry. We know consumer apps, we know consumer apps that do relatively basic (or at least, well understood) things. Success/failure then is way less about technical prowess and more about classical factors like distribution, marketing, integrations, etc.
A good example here is the lasting success of paid email providers. Multiple vendors (MSFT, GOOG, etc.) make huge amounts of money hosting people's email, despite it being a mature product that, at the basic level, is pretty solved, and where the core product can be replicated fairly easily.
The presence of open source/commodity commercial offerings hasn't really driven the price of the service to the floor, though the commodity offerings do provide some pricing pressure.
Email is pretty difficult to reliably self-host though, and typically a PITA to manage. And you really don’t ever want to lose your email address or the associated data. Fewer people could say they properly secure, manage and administer a VPS on which they can host the email server they eventually setup, over say a 10yr period.
Most people I saw offer self-hosted emails for groups (student groups etc), it ended up a mess. Compare all that to say ollama, which makes self-hosting LLMs trivial, and they’re stateless.
So I’m not sure email is a good example of commodity not bringing price to the floor.
We can assume that OpenAI/Anthropic offerings are going to be better long term simply because they have more human capital, though, right? If it turns out that what really matters in the AI race is study mode, then OpenAI goes "ok let's pivot the hundreds of genius level, well-paid engineers to that issue. AND our engineers can use every tool we offer for free without limits, even experimental models". It's tough for the small AI startup to compete with that, the best hope is to be bought like Windsurf
I'm too young to have experienced this, but I'm sure others here aren't.
During the early days of tech, was there prevailing wisdom that software companies would never be able to compete with hardware companies because the hardware companies would always be able to copy them and ship the software with the hardware?
Because I think it's basically the analogous situation. People assume that the foundation model providers have some massive advantage over the people building on top of them, but I don't really see any evidence for this.
Does https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked... count? (Edit: Missed I wasn't the first to post this in a sibling.)
Claude Code and Gemini-CLI are able to offer much more value compared to startups (like Cursor) that need to pay for model access, largely due to the immense costs involved.
Yes, any LLM-adjacent application developer should be concerned. Even if they don't do 100% of what your product does, their market reach and capitalization is scary. Any model/tooling improvements that just happen to encroach in your domain will put you on the clock...
This is actually a public validation for your friend's startup.
A proper learning tool will have history of conversation with the student, understand their knowledge level, have handcrafted curricula (to match whatever the student is supposed to learn), and be less susceptible to hallucination.
OpenAI have a bunch of other things to worry about and won't just pivot to this space.
This risk should be priced in on day one.
All bigco today, not only foundational model providers but also in media and other vertical, tend to be a platform for end user. They don’t want middle man.
If you are trying to be a middle man, you should be prepared.
I used to work for copy.ai and this happened to them. Investors always asked if the founders were worried about OpenAI competing with their consumer product. Then ChatGPT released. Turns out that was a reasonable concern.
These days they’ve pivoted to a more enterprise product and are still chugging along.
How can't these founders see this happening, too? From the start OpenAI has been getting into more markets than just "LLM provider"
There’s a case for a start up to capture enough market that LLM providers would just buy it out. Think of CharacterAI case.
Character AI was never acquired, it remains independent.
They originally claimed they wouldn’t as to not compete with their API users…
[citation needed]
Ah, I don't know. Of course there is risk involved no matter what we do (see the IDE/Cursor space), but we need to be somewhat critical of the value we add.
If you want to try and make a quick buck, fine, be quick and go for whatever. If you plan on building a long term business, don't do the most obvious, low effort low hanging fruit stuff.
yeah, if you want to stick around you need some kind of moat