Looks like a DevOps guy is going to take the blame for this: "Binance's initial investigation suggests that the security incident may involve a compromised DevOps engineer’s laptop, which allowed an attacker to gain access to a service container and submit unauthorized withdrawal transactions." https://x.com/binance/status/2115993361517891592
If a compromised laptop was the entry point and that allowed someone to steal 6 billion dollars, it’s not really the engineers fault, right?
"A loss of X dollars is always the responsibility of an executive whose financial responsibility exceeds X dollars." - Weinberg’s 'First Principle of Financial Management' and 'Second Rule of Failure Prevention’ - 'First-Order Measurement', Quality Software Management, Volume 2, Gerald Weinberg
Well it is engineers fault. But not that engineers. Some engineer designed a system without extreme oversight that should be involved in the process.
If you follow that trail far enough it's whoever designed cryptocurrency, right? Because banks can and do reverse transactions as last resort.
Banks can't reverse wire transfers. Unless there is some super-level intervention that is very rarely used. But they can trace where it went.
No, because this is explicitly the design of Bitcoin. If you give me a foo machine but it makes a bar, that's your fault. If you give me a foo machine and the foo hurts me, that's my fault.
lol I guess you just settled a hundred years of product liability jurisprudence
Bitcoin isn't a product.
Agreed.
Bitcoin isn't a store of value (its too volatile), it isn't a currency (too slow and expensive for small transactions), and it isn't even a product in its own right.
- You can use it as a store of value. - You can make transactions.
If you don’t like it or think it’s too slow doesn’t change these facts.
I agree that it’s not a product though. It is a distributed system that nobody owns.
My point is thdt your aren't really storing value when price is highly volatile, you're gambling. If I put $100 into a store of value, it would continue to be worth $100. If I put $100 in and it may be $30 or $300 later, that isn't value stored.
And yes you can functionally make transactions, but based on current transaction fees it looks like I'd pay $0.22 in fees so I could wait 10-20 minutes for a transaction to likely clear. I'm not doing that at a gas pump, grocery store, etc.
Broken by design?
https://news.ycombinator.com/item?id=50301221
Yep exactly. Broken from design stage. So that engineer were talking about is... Satoshi!?
not such a genius after all /s
This kind of low bar reminds me of FTX
If nothing else it shows a complete disregard for insider threat.
Nope, that engineer had far too much access. I'm confident this is the case in pretty much all organizations to some degree, but a lot of engineers are also the only ones that can deny themselves the access so in a roundabout way it's their fault anyway.
But a lot (generalizing here) don't like giving up access "in case they need it". I haven't worked backend in ages, but years ago there was one developer that went to the ops guys and said "can I just ssh to production I need to check some env vars to debug my software" (or something to that degree). Ops team was good and it was a hard no - "just add some logging, you can deploy a new release quickly enough".
> access "in case they need it"
A good solution to this is to have a "break glass" system. Ie. access permissions get granted to the developer directly in an emergency by going through a set of steps available beforehand. All requests by that developer are then allowed, limited to 10 kilobytes of results per minute and logged.
For example, imagine you run a bank, and a rare code bug means that at exactly midnight on new years eve all bank balances have $1B added to them.
Your oncall takes down the production servers to to reduce losses and uses the break glass system, by getting approval from n other developers, allowing them direct database access. They use that access to count(accounts) with a balance around $1B or more, see it is everyone, and subtract $1B from them all, before bringing production back up with an older software release. Downtime = 2 hours.
In the morning, all the developers actions are reviewed.
If the developer turned evil and trying to dump the database, the n approvers and 10 kilobytes per minute would reduce data loss. But the break glass system still lets the developer make fairly major emergency fixes to reduce losses or downtime.
It's not a access thing, it's a security control.
For example at most good crypto companies that hold wallets it's not possible to ssh at all, even if someone asked and go permission. It just simply isn't possible short of having a AWS root key or physical access to a computer in a DC.
There is no login, no accounts, no ssh keys or similar, they are immutable containers running from images with a signed SHA. The SHA can only be built by a build system that has it's own set of strict security controls.
Without details, I don't know why a comproised laptop would make any difference unless it submitted code through a PR and had it approved by multiple people.
Not only. I'd say the fact that something was stolen might be the engineers fault (depending on how exactly the laptop was compromised). But the amount that could be stolen like that is just ludicrous.
If you download a typosquatted package because you don't check what you type or your AI assistant got fooled I'd consider it my fault in such a case.
If the engineer, without being hacked, could take 5 billion dollars from your company and flee the country to go buy an extradition-free island, you have already gravely fucked up as an enterprise. The only way this happens is if your system is designed to allow this to happen.
This is how virtually all "crypto" exchanges work -- especially the closer you get to "shitcoin casino". Proper role and system isolation is a challenge if you support just one cryptocurrency. It would be incredibly hard for dozens, assuming service supporting dozens bothered.
I agree it's a grave fuckup but its a common one. You should just assume that when you deposit coins somewhere there are dozens of people who could just walk off with everything on their own. It's remarkable that it doesn't happen more often.
I recommend everyone to move your plantext secrets from ~/.kube/config, ~/.ssh etc into a password-manager. You can tell Claude to adjust the config files to get the secrets from 1Password when they are needed with TouchID requirements etc.
These days I’m getting so many 1Password prompts for agents pulling env variables out of it and with very little context of what is being accessed by who (The dialog 1Password shows is pretty bad) that it would be very easy for something to slip in there.
> I’m getting so many 1Password prompts for agents pulling env variables out of it and with very little context of what is being accessed by who
This is a one-sentence horror story. I almost can't believe someone would let this happen, unless the 1Password account is not important (highly unlikely).
Permission fatigue is real.
I know it sounds bad, but it’s the reality for most people, just like permission prompts at every corner and after each macOS / app update (“Your screenshot app is watching your screen”, no shit, that’s why I installed it).
I have been using Yubikeys for years, according to this guide: https://github.com/drduh/yubikey-guide
The nice thing is that using the private key requires physical touch. That means that AI itself (or a remote attacker, for that matter) can't do things unless I confirm.
if $5.6B is protected by your index finger then you're gonna need kidnap insurance, and a really good pair of gloves.
It's not protected by the index finger at all. There is no fingerprint reader in a Yubikey. The only protection is that it requires physical access and can't be triggered from software.
If you're not buying the biometric ones, sure. Now you don't even need the gloves. Maybe kneecap armor instead.
How would that help when your workstation is compromised? They can just steal your authentication the moment you log in through the password manager
It does help in that the secrets are not in plain-text, readable for any script you run. With 1Password, a process asks for the password and it gets transferred not through the filesystem but through IPC. 1Password shows a dialog: "Do you want iTerm to get your SSH key ABC123?"
In my thinking, this is more secure than plaintext in filesystem
If your system is compromised, it doesn't matter what you are using: the attacker will get the password for whatever secure thing you store your secrets in.
Sure, but is your system compromised because someone left a key in plaintext? Attack surface is finite and can be meaningfully reduced.
My secret is `4`. Is that ok? Do I still need to move it??
4 is fine, but only if you’re a certificate authority.
as for ssh keys, you can generate your keypair in secure enclave and reuiqre touchId for every new ssh session. The private key never leaves the secure enclave.
Probably the better option instead of relying on a password manager.
It is a tradeoff. When using secure enclave, you need a second backup keypair in cold storage that needs to be secured.
Isn’t the password manager a prime target too?
you know most password managers have open sockets, or other channels to other local processes to interact with and allow stealing entries?
you know most tools like gcloud, bitwarden, etc store auth keys just in dotfiles too?
Local compromise is KO now, and we can't do shit on platforms like MacOs didn't research Linux recently
The fact a single devops guy has the access to do this is the actual issue
They claim the attack wasn't just "compromised laptop ==> drain wallets" but had more steps in-between, with the attackers using image registries as an entry point into their systems.
One person's key probably should not be enough to move billions no matter who it is.