I saw something very similar a few months ago. It was a web app vibe coded by a surgeon. It worked, but they did not have an index .html file in the root web directory and they would routinely zip up all of the source code which contained all the database connection strings, API credentials, AWS credentials, etc.) and place the backup in the root web directory. They would also dump the database to that folder (for backup). So web browsers that went to https://example.com/ could see and download all the backups.
The quick fix was a simple, empty index.html file (or setting the -Indexes option in the apache config). The surgeon had no idea what this meant or why it was important. And the AI bots didn't either.
The odd part of this to me was that the AI had made good choices (strong password hashes, reasonable DB schema, etc.) and the app itself worked well. Honestly, it was impressive. But at the same time, they made some very basic deployment/security mistakes that were trivial. They just needed a bit of guidance from an experienced devops security guy to make it Internet worthy, but no one bothered to do that.
Edit: I do not recommend backing up web apps on the web server itself. That's another basic mistake. But they (or the AI) decided to do that and no one with experience was consulted.
interesting, so the ai got the hard stuff right. password hashing, schema design, fine. it fumbled the stuff that isn't really "coding" knowledge, feels more like an operational intuition? backup folder sitting in web root isn't a security question, it's a "have you ever been burned before" question, and surgeon hadn't. so they didn't ask and the model didn't cover it, imo that's the actual pattern. the model secures exactly what you ask about and has no way of knowing what you didn't think to ask. an experienced dev brings a whole graveyard of past mistakes into every project. vibe coders bring the prompt
The competence profile of any LLM-based AI is extremely spiky - whether it does a particular task well or not is pretty independent of the (subjective) difficulty of the task. This is very different from our experience with humans.
This is what I’m noticing. At my workplace, we have 3 or 4 non-devs “writing” code. One was trying to integrate their application with the UPS API.
They got the application right, and began stumbling with the integration - created a developer account, got the API key, but in place of the applications URL, the had input “localhost:5345” and couldn’t get that to work, so they gave up. They never asked the tech team what was wrong, never figured out that they needed to host the application. Some of the fundamental computer literacy is the missing piece here.
I think (maybe hopeful) people will either level up to the point where they understand that stuff, or they will just give up. Also possible that the tools get good enough to explain that stuff, so they don’t have to. But tech is wide and deep and not having an understanding of the basic systems is… IMO making it a non-starter for certain things.
Maybe this is what's missing in the prompt? We've learned years ago to tell the AI they're the expert principal 100x software developer ninja, but maybe we should also honestly disclose our own level of expertise in the task.
A simple "I'm a professional surgeon, but sadly know nothing about making software" would definitely make the conversation play out differently. How? Needs to be seen. But in an idealized scenario (which could easily become real if models are trained for it), the model would coach the (self-stated) non-expert users on the topics it would ordinarily assume the (implicitly self-stated) expert already knows.
The fix is to not let users download the credentials. In fact, ideally the web server wouldn't have access to files containing credentials, it would handle serving and caching static content and offloading requests for dynamic content to the web application's code.
Disabling auto-indexing just makes it harder to spot the issue. (To clarify, also not a bad idea in principle, just not _the_ solution.) If the file is still there and can be downloaded, that's strictly something which should not be possible in the first place.
Agent-Native DevOps tools are probably necessary. There should be no reason they would do it manually.
How I see it happening: agents like CC have in built skills for deployment and uses building blocks from either AWS or other simpler providers. Payment through OAuth and seamless checkout.
This should be standardised