This is why Anthropic and OpenAI are pushing for more “regulation”. You can’t show ROI to investors by cutting inference costs.
Regulation might inconvenience them a little, but the upside is huge if it means open models out of China get banned.
This is why Anthropic and OpenAI are pushing for more “regulation”. You can’t show ROI to investors by cutting inference costs.
Regulation might inconvenience them a little, but the upside is huge if it means open models out of China get banned.
That would be like regulating who gets access to the best food on a sinking ship. The USA could allocate capital and attention toward much better things, for the country and for their people. I just hope the next couple administrations are wise enough to not experiment with the idea that ASI can get the country out of this 'fall of Rome' slow motion decline.
Ah yeah, our lovely free market, always asking for tariffs, blocking competition, to build cartels and monopolies.
Free markets don't exist to help individual companies, they are explicitly meant to hinder them.
They are meant to help the consumer.
Unfortunately neither of the two parties in the US even pay lip service to free markets anymore.
Free markets help the big producers much more than consumers.
The Peter principle doesn't apply to consumers, but it does to producers. And if nobody regulates how big you can get, you eventually end up with a nothing but ruthless monopolies in every industry. Regulated markets tend not to generate as many monopolies. And of course, for natural monopolies, the state is the least bad option, because consumers have at least some say in how its operated through voting, and there is no profit motive.
I wouldn't say 'help' the consumer as that makes it seem like things are biased.
The idealistic perfectly competitive markets entail exhausting all gains from trade.
TINSTAAFM. There is no such thing as a free market under capitalism, as the first winners - which there are, even in a perfectly free market - will put their resources into making it not-free, as we're seeing here. These advantages accumulate and you end up with a few massive monopolies which the market exerts little control over. Not zero, but little, much less than most economists claimed for close to a century.
Keeping a market free would require magnitudes more regulation than right now - the opposite of what free market proponents advocate for.
> You can’t show ROI to investors by cutting inference costs
You can as long as you can saturate hardware with profit that makes up for training costs (which are also in free fall).
Anthropic is most at risk since they haven't bought any hardware (afaik), however OpenAi is building custom silicon now. The rest of Mag7 are primarily hardware vendors
Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
MiMo RL'd their latest models for $3.5M (million, not billion). It's hardware, software, and algos
https://mimo.xiaomi.com/rl/
Also look to the history of stable diffusion
I believe that’s post-training, not pre-training.
> I believe that’s post-training, not pre-training
Aren't AI companies already butting against the limits of usable training data?
Outside of undiscovered techniques, additional costly pre-training will likely yield diminishing returns if you've already scraped the whole internet and destructively scanned as many 20th century books as you could lay your hands on.
yes, just post training, which is the most expensive phase now as I understand it
Training costs are not in free fall, and not every "post training" process is equal, Everyone does this differently. The costs certainly haven't gone down, to train a new 10T param model from scratch, like at all.
You cannot point at Xiaomis tiny model and compare it to training a 10T param model. The levels of dunning krueger are getting out of hand.
You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
> You also arent calculating the salaries of the people involved, of which there's only a few thousand worldwide in your hiring pool.
There were probably a few dozen 20 years ago, and the existing pool grew from somewhere. AI (nee machine learning) is not rocket science - but it is a science, so it rewards those who are literate, curious, can hypothesize, experiment and learn from results to gain an intuition. I bet most statistics-adjacent professionals (millions, worldwide) can move laterally into AI research and excel at it. Nevermind AI has been among the most-in-demand undergrad programs since ChatGPT dropped. The only limited pool is for celebrity AI talent.
You should start a recruiting outfit then if its that easy, because you could be making 40-60k per hire!!!
Your average Stats/machine learning expertise has little to do with standing up training infra and pipelines across $50 million in gpus.
you haven't cited any examples, nor acknowledged this is how everything in computing has gone, DK is a two way street I suppose
stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy, many more people will be able to train up what we have today, especially as more of the pipeline is open sourced and the knowhow distills
>stable diffusion is an excellent example, the price to train today's or yesterday's frontier is the analogy,
Dont use llms to draft your responses to people, extermely cringe. That explains the llm induced Dunning Krueger.
What examples... There are no public numbers regarding how much it costs to train a 10T model, atleast nothing concrete.
You should just use common sense and think about how you have to fit tensors in memory to train, and the larger the model is the more GPUs you need. Then you could also use common sense to think about how GPUs have gone up in price, not down.
> Dont use llms to draft your responses to people, extermely cringe
You are throwing around unfounded and incorrect accusations. Why? It's a weak ass rebuttal mate
common sense is using things like history to guide expectations instead just saying things without evidence that are contrary to the patterns in computing history
I would think that access to power is more critical than compute
> This is why Anthropic and OpenAI are pushing for more “regulation”
Given everything happening with Black October[0], don't you think regulating models may be reasonable?
[0] https://en.wikipedia.org/wiki/Black_October_(information_sec...
I too would like security issues to stay unpatched because only three letter agencies have access to uncrippled tools for finding the bugs.
Your point is important enough to say without sarcasm.
The AI companies didn't create the vulnerability. It has always been there.
The tools to detect these vulnerabilities will not go away. The only question is who will be allowed to have access to them?
I think that the frontier labs knew what they were doing when they invested in cybersecurity starting in the spring (or maybe earlier). They knew it would create hype. They knew it fit with their narrative that they are building a dangerous thing (something they believe because it’s fun to believe in doom, because it’s a great story for investors, and because it’s not inconsistent with a moat - in fact it makes a moat via regulatory capture).
“Black October” sounds a lot like Red Scare.
Well, OpenAI is at the root of Black October, so maybe they are the problem
If they are the problem, shouldn't they be regulated?
You remind me of those politicians who try to scapegoat people who point out security holes. https://www.theregister.com/security/2022/02/15/missouri-end...
Regulate the bad actors, not the neutral tools. Lockpicks aren't illegal, but breaking and entering is. Imagine if MasterLock used a lockpick to break into a government building and steal stuff, then tried to lobby to make lockpicks illegal. That's what OpenAI is doing, and you're falling for it.
This is the goal of all government regulations and it's always resulted in higher prices and near monopolies dominating the market.
Yet, so many Democrats support it.
In the hands of 4th party system Progressive Republicans and 5th party system New Deal Democrats, government was an extremely powerful force for trust-busting.
In the hands of 6th party system Republicans, namely Ronald Reagan and Robert Bork, the government worked hard to usher in a a new era of Robber Barons. We live in the culmination of this project.
History says the result depends on the hands, not the tool. 6th party system Democrats have largely failed to undo the Reagan/Bork policies, but that's an act of omission not comission and even then Lina Khan was a big step in the right direction. This issue was on the ballot in 2024 and will be again in 2028. Hopefully we make better choices in 2028.
If you are going to limit the 5th Party system to the first half of the long 1930s-1990s period of overlapping realignments (which is pretty reasonable), then the 6th Party system should be the second half of that period, and the period from about the end of the Clinton Administration on should be the 7th Party system; you can call now the 6th Party system if you call the entire long realignment period the 5th (which makes some sense, too, though there is a pretty significant difference between the New Deal dominant period and the Civil Rights dominant period of the realignment), but to split it somewhere in the middle but then lump the period after with it makes little sense.
If you want to call Obama and Biden 7th party system, I don't really agree but I also don't think it matters for my argument: we can just re-label them and move on.
It is pretty funny you refer to these periods because they are the quintessential example of big business pushing for regulations to push out their competitors and become monopolies. And the "progressives" enthusiastically supported such monopolization.
Regulated monopolies are much better than unregulated ones. Free markets generate the latter, and they have created most of the big issues we have today: inequality, climate change, the military industrial complex.
If you take away all regulation you’d probably end up with a monopoly anyway, as companies would no longer be prevented from merging with their competitors or using underhand tactics to gain market share.
It's wild how businessmen will pitch you the 10 ways they will abuse the rules of the game to obtain a monopoly and extract rent, then they will turn around and say that the game rules are perfect and our only failing is not following them hard enough.
Goomba fallacy
Nope, same guy. Seen it a dozen times. Unenlightened self interest is a helluva drug.
This is the goal of some, but very few overall, government regulations. Not even close to "all".
That's wrong, but why let reality get in the way of a pithy ideology?
Government regulations exist to regulate activity, to prevent, punish, and/or remediate harms.
Environmental regulations exist because businesses wouldn't clean up their own messes. Financial regulations exist almost exclusively because of harms and abuses committed by members of the financial industry. Building codes are updated to reflect lessons learned about people died, to prevent more people from dying the same way.
If AI regulations are passed, it will be because the AI industry decided not to self-regulate itself and stop illegally hacking computer systems, generating CSAM, or various other illegal activities that should easily be preventable if their systems were actually intelligent.
“All” is doing a lot of work there.
Also there are real trade offs. Food would be cheaper if, for instance, it were legal to cut it with any old cheap substance regardless of health effects.
But I totally agree that some regulations are dumb or are regulatory capture monopoly schemes.
Unfortunately like many other things there is no absolute simple rule to save us from thinking. Each case must be evaluated in depth hopefully with domain expertise.