I don't know what the solution to this is, but I find it somewhat unfair that I pay money to Anthropic, and I pay money to OpenAI, and neither of them will let me use their best models for securing the software I work on.
Admittedly Opus 4.8 xhigh does a good job, but are my customers not entitled to have more security from a Fable/Mythos or GPT-5.5-Cyber audit over the codebase? Or I guess the inverse question: why aren't they allowed that audit?
(Fable/Mythos being unavailable notwithstanding.)
It seems OpenAI will at least let me do this narrowly, at greater cost, by using one of their partners. But I already pay them money!
It is unfair and not useful at all.
If a criminal organisation (include here some countries) want to be deterred. We should all have access so we can improve security of our products.
Because the people that want to do evil, will do it anyway. They will build a myths, fables, and cyber clone.
I dislike the hypocrisy of it. Oeh ah it’s too dangerous, criminals can make use of this. But at the same time they themselves stole a whole lot of data to be trained on.
The problem is even worse than that. OpenAI and Anthropic have your source code and superior knowledge of its vulnerabilities. All you can do is hope that they won't one day use it against you.
or accidentally hand the information over to someone who will
But they will! Or the government or the xyz agency !
To what end?
I think using open weight models will solve this. I believe they are nearly caught up and much of the gains are in the harnesses or properly orchestration of subqueries. (I'm no expert, just my opinion).
When the open weight models catch up, if they don't get lobbied and banned by OpenAi and Anthropic, then you'll be able to use them to properly secure your software.
Pretty sure the secret sauce is in the summarised thinking. Maybe better though process… But I have a feeling it’s server side tools and a scratch space to prepare the reply.
Sometimes the summarised thoughts include stuff that makes no sense unless it’s got a workspace on the server. Stuff like “I am now writing x to file y”.
Not championing it, but this is where something like OpenClaw comes into play, right? The harness around the model, the ability to call tools, etc.
I'm no cyber expert, maybe one can weigh in.
Are there zero days that only a true genius can discover? Or can a smart-enough model, run over the codebase for enough time, discover them all?
Like as we get smarter and smarter models do we expect each new generation to keep finding vulnerabilities, or to plateaue?
A large part of vulnerability analysis is just having the time to crunch through enough possibilities. Expertise and smarts definitely speed this up but there's a lot of just turning the crank until something falls out. Even a relatively dumb model with some good prompting will find vulnerabilities if you ask it to and give it the time and resources to do so.
Completely agree. Its all about time spent.
Been in the security industry a long time as a software engineer. Security research is no different than any other engineering discipline. It is down to the time you are willing to invest and where in the abstraction you focus.
All of this pearl clutching and hand wringing over the capabilities of the models is silly to me. It has much less to do with some magical cybersecurity ability and much more to do with increasing ability of models to stay on task for long horizons. Any passionate engineer will recognize this - if you grind 10,000 hours you will find the solution to most problems, the problem is most people lack the motivation to even start, and are too risk averse to play hacker.
The NSAs claim that all government systems were hacked by mythos and they were shocked by that is farcical. They have been hacked over and over and over by many who took the risk and tried.
It's like they hired a competent red teamer to do internal pen testing for the first time, which we know is absolutely not the case. They have been doing it for years, and almost certainly surfacing the exact same kinds of findings each time, but they haven't been honest with the public about it and can scapegoat mythos now.
> Any passionate engineer will recognize this - if you grind 10,000 hours you will find the solution to most problems, the problem is most people lack the motivation to even start, and are too risk averse to play hacker.
This. I'd love to spend my whole day hacking stuff, but I need to pay my bills.
Now with AI tooling my late night/weekend hobby hacking stuff is at least getting done. I'm definitely progressing with things that I began 2 years ago and I had to stop as other life priorities took over.
That entirely depends on whether a “smart enough” model is a genius or where that cutoff is.
To your second question, a clear plateau would be a piece of software that is 100% secure, without vulnerabilities. Since that’s impossible for anything more than a trivially simple program, particularly when you consider an ecosystem, I think there won’t be a plateau. If you use model A to secure program Dog, smarter model B could find a vulnerability in Dog or just skip to attacking Dog’s OS, firmware, etc.
It is super unfair
It creates a two tier system - those who have access and those who do not. Worse, it’s some corpo making the decision
Remember when access for all used to be a high priority for AI? I do. I noticed they don't talk about that anymore
Surely what's coming is them offering to fix your vulnerabilities via higher-margin professional services?
At least on their benchmark, the regular, public GPT-5.5 is basically at Mythos level already. (2% difference on CyberGym)
They didn't test Opus 4.8, but it probably isn't very far behind.
take a look at this bug and the chain required to exploit it:
https://projectzero.google/2021/12/a-deep-dive-into-nso-zero...
https://projectzero.google/2022/03/forcedentry-sandbox-escap...
exploiting vulnerabilities on hardened targets isn't just in a different league from finding them, it is a different sport altogether.
put simply, it's the difference between an integer overflow leading to a sandbox escaping RCE and one that leads to a crash.
Codex Security and 5.5/5.6 are still very good finding vulnerable code -- they will identify and fix unsafe behavior, but they will refuse to help you with exploitation -- they will actively prevent you from taking any steps to weaponize the unsafe behavior that are not required to remediate it. they will err conservative here, but for the most part they will still let you discover and address a wide range and depth of vulnerabilities. you can verify yourself to turn off the most basic safeguards and sign up through a more rigorous process for a spectrum of TAC options.
obviously there is a balance here -- openai wants to empower defenders while at the same time not exposing capabilities to the adversaries that would overwhelm defenders. there is no "right" answer. it is a work in progress. this is an intentional and deliberate decision to provide defenders with a (temporary, dwindling) advantage.
the example i chose was pretty extreme, but the underlying principle -- enable visibility discovery and remediation, but make it difficult to weaponize and defeat countermeasures makes sense given the bigger picture, IMO.
this calm before the storm is not going to last for very long, and defenders need every advantage they can get to get their houses in order before these capabilities are widely commoditized.
I have Verified/Trusted Access on both of them and I don't even work in Cyber.
I filled it out as an individual using my own Org ID and I got accepted to both of them, lol.
So I got turned off the OpenAI form because it’s pretty heavily geared towards “enterprise” which I’m not. But I’ll have a stab at it later.
What’s the equivalent form for Anthropic please? The closest I got from Google was Claude Security’s “contact sales.”
Equivalent of Trusted Access for Anthropic is "CVP" (Cyber Verification Program)
https://support.claude.com/en/articles/14604842-real-time-cy...
Why is it unfair? Are you entitled to them? They released a product and you are paying for it. If you don't like the product, don't pay for it and don't use it.
At least, pay them for the inferior intelligence until Donald Trump says you can.
Because while you may be a good actor, there are just as many bad actors out there.
How does Anthropic or OpenAI differentiate between the two?
Once you solve that, you can get access to Mythos ;)
More importantly who gets to decide good or bad?
Remember all of these models are based on unimaginable levels of copyright infringement. Is OpenAI a bad actor, that they use their models to infringe on the rights of others?
This isn't a moral argument. This is all about power and money, not good or bad. That includes the Mythos ban. Good vs bad actors is political theater designed to distract from what's actually going on.
> unimaginable levels of copyright infringement
This isn't how copyright works. The models don't wholesale encode literal information from original works and are substantive transformations. Now, you yourself as a user can use the models and weights to infringe on a copyright.
There have been some US cases about this, but it isn't generally settled internationally. "Fair use" is a US specific thing. Even in the US there are ongoing cases.
Paper about how weights are a derivative work of the training data: https://arxiv.org/abs/2407.13493
Currently in progress law suits about AI copyright: https://informationisbeautiful.net/visualizations/the-rise-o...
Yeah, I'm familiar with that argument re derivative work, but weights aren't really what's being shipped or sold, and I think it's reasonable to argue that the generated tokens aren't derivative but substantively transformed.
That said, I would prefer a situation where hyper-scalers make an effort to compensate sources of good data, e.g. newspapers and so on.
Like it or not, Bartz v. Anthropic established that as fair use. So it isn't legally copyright infringement as currently understood under the law. This may change but it isn't obviously wrong.
I think parent poster was referring to the open secret that the early models were trained on massive collections of pirated novels and textbooks.
> How does Anthropic or OpenAI differentiate between the two?
So if they can't why do some companies still get access today? Just 1s much bigger than "us".
It's the equivalent of saying a company like Amazon or Cloudflare should block access to web hosting or "illegal hosting". The argument back then was they aren't gatekeepers? But now they are?
This is really odd taking two completely different things and trying to apply law against them. Hosting was somewhat protected by previous rulings, selling AI services is not.
> This is really odd taking two completely different things and trying to apply law against them. Hosting was somewhat protected by previous rulings, selling AI services is not.
What's different? What's not protected? And what's "hosting"? Where do you draw the line with "managed services"?
So if you use "AI" to hack a computer it is different to using "hosting" to put "illegal content"?
Are you implying 1 of them is legal? But both are for the judge to decide.
OR if this is about the provider -- who's selling AI services? It's LLM. Just running software on GPUs. There's no AI. There, done. Same.
While I appreciate the desire to have the best:
> Or I guess the inverse question: why aren't they allowed that audit?
There's undeniably a lot of unsecured software in the world.
Given that ID verification is hard and these companies are clearly new at it (or don't understand the implications of it, cough Worldcoin's eye-scanning orbs cough), which is worse:
(1) sufficiently good AI* is released to everyone: critical infrastructure and open source projects gets better hacking tools to white-hack their own code at exactly the same time as black hat hackers
(2) sufficiently good AI* is released to critical infrastructure and open source projects first: everyone else, the average paying customer has to wait but so too do the black hats
Because (2) is either the status quo or better depending on if you have access or not; and because (1) seems to me to lead to an acceleration of zero-days, I lean towards (1) being the worse.
* having no experience of pen-testing, I take no position on if this is "it" or not
1 assumes that some "private entity" gets to decide what is crucial infrastructure and what is not, what gets the opportunity to be patched and what doesn't.
I'm not ok with that and don't know why anyone would be.
Because it's their property. Now you can try to make an argument that it's stolen IP and that matters in some way, but that's just more likely to ensure no one has access.
Even more so they are getting push back from the government (good job electing idiots) that said models are a security risks.
But until then the company can charge/give access to whoever they want for however much they want except in the cases the law says no.
And if you don't like it raise a trillion dollars and make your own.
The owner of a thing deciding to whom they wish to provide access to the thing they own, is a necessary consequence of the concept of private property.
The only two alternative to a private entity making this decision are a government making this decision, or nobody making this decision, the latter of which is equivalent to both government and a private entity making the decision to do (1).
That's a complete false dichotomy.
We create all kinds of consortiums to manage complex decisions that shouldn't be owned by one entity.
Soon, very soon, if you will need something useful, like medical advice, financial advice, you will be told that, well, ok, but you need to pay for an "extended license" that gonna be in thousands of dollars per month, otherwise you need to hire someone who paid that money.
The only hope are Chinese models, as Chinese commies are playing a different game as long as they are behind the flagship models (but it will change soon, like with cheap Chinese cars) and maybe, finally, Europe will start working on their solutions, instead of regulations.
That sounds too dire. My suspicion is building a model either as a derivative or brand new is a solved problem. There are indeed capital constraints today but would wager that over the horizon those go down. If one business is restricting access to something great over the medium term other companies will step up.
I think every business wants to bill on value not usage. That's where the real money is made. If a diagnosis is worth $100 and takes $1 worth of tokens you want to bill as close to $100 as you can. Right now they're billing $1 and barely making money.
What makes the diagnosis "worth" $100? Right now it requires a highly paid human which sets a floor on the cost.
If there's competition from LLMs it's going to drive down the cost.
>What makes the diagnosis "worth" $100?
The opinion of the customer paying for it
If you can buy a gun from a weapons manufacturer it doesn't mean they should also allow you to buy a rocket launcher.
First of all, these products are "legal". I think the point is more that we pay for your top subscription but you've decided that a handful of companies that you pick get access to the best of the best now and everyone else has to wait and perhaps they may be allowed access at some point...if deemed worthy.
You want the few leading AI companies in the U.S. to work under the model where they (and potentially the U.S. gov't) gets to decide who gets access to what compute? If you are fine paying into that model, then good for you...just a matter of time before they cut you off and you have no ramifications.
Since when did US markets ever guarantee unlimited access to everyone. I don’t understand this line of thinking that has cropped up around AI companies.
I haven't seen anyone suggest that a company can't do this, they absolutely can within certain frameworks of the law. That doesn't mean consumers have to like it and continue to business with companies such as these.
[flagged]
I'm not sure I follow your logic. Paying for a service does not mean you get access to all potential services a provider offers. Providers can choose to keep some services internal.
Silly example: I pay Netflix for their most basic plan, so I get ads. Just because I already pay them money, doesn't mean I have a right to no ads! It also doesn't mean I have a right to 8k streaming; maybe Netflix reserves that for their internal cinema.
Both companies offer "MAX" or "PRO" plans - and the best models were available to those customers. This new wave of "It's too dangerous for the public" is a new initiative from both companies.
I agree with your overall sentiment. Paying for "Claude Mini" doesn't get you "Claude Maximos".
However, the overall precedent that the companies have set is that if you pay for the top tier subscription, you get the top tier model. That's not true any more.
This is so similar to people arguing for plan tokens to be used with third party tools. It does not jive with my understanding of the world. Do people really expect that paying for a top plan actually gets you guaranteed access to everything? It’s great when it works but at the end of the day why build that false expectation.
Just like when you buy a top of the line camera or car, and then they release a new one, you are entitled to the now-top-of-the-line camera or car.
What the heck come on.
Buying a camera or car is different from paying a subscription, right? Different expectations
No, I have never bought a subscription and then expected it to get arbitrarily upgraded any time a higher tier was introduced.
But that's the exact standard that was set by the LLM providers, right? My ChatGPT 3.5 sub became a 4o sub, which became a 5.1 sub and so on
You have the right to complain and ask for more though
When Netflix launched, you got the service without ads. That has changed. That's what's known as a rug-pull.
> Paying for a service does not mean you get access to all potential services a provider offers. Providers can choose to keep some services internal.
The problem most people have is "the logic".
Sure you can keep it internal. Sure you can not offer it to everyone.
No then it is not for "world security", "world peace" or some other "explanation".
I pay United airlines the price of an economy ticket, it's so unfair I don't get to fly in first class
If you want this analogy be correct, then you need to rewrite it to a world where you are not allowed to buy first class tickets at any price. Only people who work for a giant corporation that is blessed are allowed to buy first class, well you are only sold a coach seat. That doesn't sound very fair to me
You can pay anthropic enough money to be part of project glasswing. Some people here just can't accept that there's something coding related they can't afford.
How much does it cost?
Somebody's never flown private