Maybe Zuck should double down on his "spoiler" role with models rather than compete head-to-head.
He doesn't have to match Anthropic or OpenAI model revenue if he can deflate theirs by 99%.
All he has to do is keep spending a few billion dollars developing frontier models, release them as open weights, and turn coding models into a commodity. He also needs a good OSS reference harness to match. Very few people are in a position to do this and for it to make business sense.
That's quite likely where things are headed regardless, and he could speed it up significantly.
We should all hope models move from proprietary products to commodities the way compilers did.
This may be one of the best things Zuck could do for the world.
> keep spending a few billion dollars developing frontier models, release them as open weights, and turn coding models into a commodity
the muse family is no longer open-source. it is still priced very "cheaply", but a significant shift in approach.
If he deflates their revenues, who is going to rent the compute from Meta?
The goal is not for meta to take their market, the goal would be for meta to damage their competitors.
If meta releases an open-weight LLM that is not Chinese made, cheaper to run than the SOTA premiums, etc, it would lower the number of people paying for frontier labs models. We saw with with early LLAMA models, but they didn’t keep up in the race with v4.
Meta would benefit from this, not from increased revenue at the hands of open LLMs, but from reduced competition. Meta competes with Google for ad spend, and lowering the Google revenue (or increasing costs) from AI reduces the competitive advantage. OpenAI wants to build an ad engine, so same thing will apply there too - make it less-revenue-generating to compete. Meanwhile G, OpenAI, and Anthropic are huge talent sinks that they have to compete with, especially for ML talent which is core to Metas business goals (ads). Finally, Meta needs lots of GPUs to train their ad engine models. By reducing the revenue-per-GPU of these labs, they’re reducing demand on a core revenue generating supply they have to compete for.
The problem is that Meta jumped headfirst into the circular financing of datacenter build out, so they're just as screwed if the infrastructure becomes less valuable.
If it's good enough, they don't need to sell to Anthropic/OpenAI.
Commoditize your complements.
Coding models are not the destination. Coding models are just part of the bootstrapping process towards general intelligence.
Software has several unique properties on both ends of its production process that make assertions of progress based on the software use case invalid.
Software is easy to define as “working”: just run it. But - useful software requires an absolute truck worth of code - 100k loc before you’re talking about a real product, or else dozens of iterations of a toy you make for yourself before it’s useful enough to quit toying with and just use for what you wanted it for.
Sure, the success of software is hard to anticipate and what “good” UX is is hard to pin down - that’s not what I’m talking about. I’m talking just making the code and having no lint errors. That shit is a slog but it’s a slog with a clear goal amenable to hill climbing.
Through that lens software is mostly pattern matching. It’s very rare that an activity in software construction is out of distribution because even if the core of the thing is novel it needs a massive blanket of UI and a tech stack and a production environment to run in and observability and and and and. You get it I hope.
Meanwhile most work out there is a mess of undocumented, un-codifiable detail with no objective criteria for success, only a very wide gradient of “job well done” to “what is this garbage go and fix it”.
We are solving the easy parts of software and soon all that’s left will be the parts that are just like other work. And then we engineers will also be doing mostly squishy subjective judgment stuff.
He tried that with llama?
the way he could really be the spoiler king is to release an their training dataset to open source… doubt he’d go that far.
if they release training dataset, they will be in trouble for copyright reasons
all he has to do is that prove builing these inst hard anymore. because the whole moat these companies have is the perception that building models at frontier is really hard .
Hitherto he's pretty much proved the opposite.
I guess we'll see how Meta did this time.
they've failed at every hard thing they tried to do. so if meta did it then it must be easy.