I do not work in the space at all, but it seems like Cloudflare has been having more network disruptions lately than they used to. To anyone who deals with this sort of thing, is that just recency bias?
I do not work in the space at all, but it seems like Cloudflare has been having more network disruptions lately than they used to. To anyone who deals with this sort of thing, is that just recency bias?
It is not. They went about 5 years without one of these, and had a handful over the last 6 months. They're really going to need to figure out what's going wrong and clean up shop.
Engineers have been vibe coding a lot recently...
The featured blog post where one of their senior engineering PMs presented an allegedly "production grade" Matrix implementation, in which authentication was stubbed out as a TODO, says it all really. I'm glad a quarter of the internet is in such responsible hands.
It's spreading and only going to get worse.
Management thinks AI tools should make everyone 10x as productive, so they're all trying to run lean teams and load up the remaining engineers with all the work. This will end about as well as the great offshoring of the early 2000s.
there was also a post here where an engineer was parading around a vibe-coded oauth library he'd made as a demonstration of how great LLMs were
at which point the CVEs started to fly in
Matrix doesn't actually define how one should do authentication though... every homeserver software is free to implement it however they want.
the main bit of auth which was left unimplemented on matrix-workers was the critical logic which authorizes traffic over federation: https://spec.matrix.org/latest/server-server-api/#authorizat...
Auth for clients is also specified in the spec - there is some scope for homeservers to freestyle, but nowadays they have to implement OIDC: https://spec.matrix.org/latest/client-server-api/#client-aut...
Thats a classic claude move, even the new sonnet 4.6 still does this.
It’s almost as classic as just short circuiting tests in lightly obfuscated ways.
I could be quite the kernel developer if making the test green was the only criteria.
[dead]
No joke. In my company we "sabotaged" the AI initiative led by the CTO. We used LLMs to deliver features as requested by the CTO, but we introduced a couple of bugs here and there (intentionally). As a result, the quarter ended up with more time allocated to fix bugs and tons of customer claims. The CTO is now undoing his initiative. We all have now some time more to keep our jobs.
Thats actively malicious. I understand not going out of your way to catch the LLMs' bugs so as to show the folly of the initiative, but actively sabotaging it is legitimately dangerous behavior. Its acting in bad faith. And i say this as someone who would mostly oppose such an initiative myself
I would go so far as to say that you shouldnt be employed in the industry. Malicious actors like you will contribute to an erosion of trust thatll make everything worse
Might be but sometimes you don’t have another choice when employers are enforcing AIs which have no „feeling“ for context of all business processes involved created by human workers in the years before. Those who spent a lot of love and energy for them mostly. And who are now forced to work against an inferior but overpowered workforce.
Don’t stop sabotaging AI efforts.
Honestly i kinda like the aesthetic of cyberanarchism, but its not for me. It erodes trust
Forcing developers to use unsafe LLM tools is also malicious. This is completely ethical to me. Not commenting on legality.
I dont like it either but its not malicious. The LLM isnt accessing your homeserver, its accessing corporate information. Your employer can order you to be reckless with their information, thats not malicious, its not your information. You should CYA and not do anything illegal even if your asked. But using LLMs isnt illegal. This is bad faith argument
I see someone is not familiar with the joys of the current job market.
That's extremely unethical. You're being paid to do something and you deliberately broke it which not only cost your employer additional time and money, but it also cost your customers time and money. If I were you, I'd probably just quit and find another profession.
That's not "sabotaged", that's sabotaged, if you intentionally introduced the bugs. Be very careful admitting something like that publicly unless you're absolutely completely sure nobody could map your HN username to your real identity.
Typo: "shop", should have been with an 'el'.
(: phonetically, because 'l's are hard to read.
Launching a new service every 5 minutes is obviously stretching their resources.
been at cf for 7 yrs but thinking of gtfo soon. the ceo is a manchild, new cto is an idiot, rest of leadership was replaced by yes-men, and the push for AI-first is being a disaster. c levels pretend they care about reliability but pressure teams to constantly ship, cto vibe codes terraform changes without warning anyone, and it's overall a bigger and bigger mess
even the blog, that used to be a respected source of technical content, has morphed into a garbage fire of slop and vaporware announcements since jgc left.
Do you feel that Matthew Prince is still technically active/informed? I've interacted with him in the past and he seemed relatively technically grounded, but that doesn't seem as true these days.
https://xcancel.com/eastdakota/status/2025215495142564177
https://xcancel.com/eastdakota/status/2025221270061580453
Rather than be driven by something rational like building a great product or making lots of money he is apparently driven by a desperate fear of being a dinosaur.
Regardless of how competent he is or isn’t as a technologist, a leader leading with fear is a recipe for disaster.
I’ve had a lot of problems lately. Basic things are failing and it’s like product isn’t involved at all in the dash. What’s worse? The support.. the chat is the buggiest thing I’ve ever seen.
don't worry, if it gets much worse the ceo will just throw all of support under the bus again. it will surely get better.
How about accurate billing info. The ux can’t even figure out we’re annually not monthly. Maybe the AI slop will continue to miscount resources and cost you revenue or piss off a customer when the dashboards they been using don’t match the invoice
You know what they say, shit rolls downhill. I don't personally know the CEO, but the feeling I have got from their public fits on social media doesn't instill confidence.
If I was a CF customer I would be migrating off now.
GSD! GSD!! ship! ship! ship!
**everything breaks**
...
**everything breaks again**
oh fuck! Code Orange! I repeat, Code Orange! we need to rebuild trust(R)(TM)! we've let our customers down!
...
**everything breaks again**
Code Orangier! I repeat, Code Orangier!
exactly. recently "if the cto is shipping more than you, you're doing something wrong"
cto can't even articulate a sentence without passing it through an LLM, and instead of doing his job he's posting the stupidest shit to his personal bootlicking chat channel. I cringe every time at the brown-nosers that inhabit that hovel.
no words for what the product org is becoming too. they should take their own advice a bit further and just replace all the leadership with an LLM, it would be cheaper and it's the same shit in practice
amazing how my comment was flagged in 30 seconds... keep bootlicking
It has been roughly speaking five and a half years since the IPO. The original CTO (John Graham-Cumming) left about a year ago.
They coasted on momentum for half a year. I don't even think it says anything negative about the current CTO, but more of what an exception JGC is relative to what is normal. A CTO leaving would never show up the next day in the stats, the position is strategic after all. But you'd expect to see the effect after a while, 6 months is longer than I would have expected, but short enough that cause and effect are undeniable.
Even so, it is a strong reminder not to rely on any one vendor for critical stuff, in case that wasn't clear enough yet.
You can coast for quite some time (5-10 years?) if you really lean into it (95% of the knowledge of maintaining and scaling the stack is there in the minds of hundreds of developers).
Seems like Matthew Prince didn't choose that route.
I wondered what happened to him?
I am reading HN.
What is your opinion on the recent Cloudflare outages?
He's on a yacht somewhere
For real
Cloudflare Outages are as predictable, as the Sun coming up tomorrow. Its their engineering culture.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
Wait till you see the drama around their horrible terraform provider update/rewrite:
https://github.com/cloudflare/terraform-provider-cloudflare/...