Interesting, but... wrong tool, wrong job. And by tool, I mean web based chat interface, not LLMs in general. (Maybe wrong delivery mechanism, if you like.)
Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these.
An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution.
As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something.
Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away.
(Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
The whole point is that the friend wouldn’t know what an agent is. The friend would have heard that AI is changing the world and she would have went into ChatGPT or maybe Claude or Gemini. Presuming that the ideal is that anyone should be able to use these tools for anything (which I do not agree with) the chatbot needs to say “you need to use my agent mode.”
This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
We're not talking about the friend. I'm talking to the person I responded to: I'm saying they should have recommended a better tool to their friend.
My HR friend found Lovable off her own bat. I imagine she probably Googled or asked ChatGPT something like, "How do I build a website to do BLAH without knowing anything about building websites?"
The point is people talk, they ask questions, they Google, they talk to ChatGPT, and if they have a problem to solve they're often quite motivated to find a solution to that problem off their own backs.
If someone asks me for advice on how to get something built then I'm going to recommend a tool that suits them and their situation, whatever that may be. In this specific situation, if they know nothing about building software, I'm certainly not going to sit them down and have them try to follow the most jank-ass way imaginable of building software with an LLM when I know much better tools exist that are built with people like them in mind.
Seriously, what is with the overly narrow assumptions in the replies I'm getting this morning? You're the third person who's tried to set this same fraying paper tiger on me. Can we all just wake up and think about the issues a bit more in the round, please?
That would be poisoning the test with outside knowledge. Essentially boosting their skills with some expertise, exactly what was undesirable.
You are not wrong, just that was not the intent of what the person was trying to test for.
What makes this anecdote hard to believe is that seemingly two things happened simultaneously:
1. The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas.
2. The user was the sycophant in this relationship, never steering the session(-s) into producing something tangible.
The premise supposes that somehow the session(-s) never even tangentially touched implementation/deployment ideas and the user has never typed something like "that's enough, how to make this appear in my browser?". While not impossible, the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Indeed. It reads like software engineer employment cope by appealing to the lowest common denominator possible. As if solutions for that type of end user weren't already solved for over a decade ago, most of which now have agents built in.
It just is not believable or interesting. Even if it did happen, the reality is it just doesn't matter.
It reads like a discussion between two people sharing detailed experiences of concrete things they've observed, in response to an article where the author did the same. It could be that they're lying to stave off employment anxiety, I suppose, even though nobody involved seemed particularly anxious and everyone involved seems pretty familiar with AI tooling.
It could also be that you've seen a lot of social media memes regarding "cope", observed their ability to provoke strong emotions, and confused this for meaningful insight. I see a lot of AI commentary these days that is clearly being spread for its virality rather than its truth value.
> What makes this anecdote hard to believe is that seemingly two things happened simultaneously
What i described is what happened. I don’t appreciate the undertones where you are insinuating that i’m lying for whatever reason.
> The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas
You call it steering. I would call it falling into that grove. Probably tiny things in the initial message made the first response more likely to be a clarifying/ideating type. And once that happened the conversation was gaining momentum in that direction and neither participant was trying to guide it in a different one.
> the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Exactly. The LLM itself sidetracked her. They were just talking about cool features they could add, and at no point did she put down her feet and say “stop asking more questions and just write the code”.
It can be a combination of many things. Attitude (some people hate to be rude, and not answering a question feels a bit rude). It can also be that she enjoyed the process of unpacking and elaborating on the idea.
The meaning of the story is not that no lay person can possibly develop using AI. That would be silly, and untrue. I know clear counter examples. The point is that if you don’t know what you don’t know it is harder to steer the AI in the direction you could very easily with the right lingo.
Circular dependency detected. You must first know the thing to be able to know the thing.
No, not really. My HR friend went out and found Lovable on her own. You're acting like people can't Google, or even ask an LLM via a Chatbot interface for recommendations for tools that would help them build an application as an absolute beginner. They can and do. People have agency, which is exactly why this person's friend asked them for help.
This: https://xkcd.com/2501/ 1 billion people use chatgpt, but only 10M use codex, which is even more impressive when you consider the chatgpt windows app now comes with codex a click away. This means everything that is super obvious to us, is not for the average person
They were literally in front of an LLM. Did they ask for tools?
edit: Lovable is a web app too, not an agent.
This comment - and many others - misses the OP's original point: what are tools? Do you mean my laptop, and a pen and paper? What's an agent? I.D.E.? none of this is part of the vocabulary or context of someone who has never built things with a computer, let alone AI. Years ago I had a neighbor who was trying to learn how to use their computer; they complained that the book they were using was only telling them all the DOS and none of the DON'TS. This is were most people start.
You are very much mistaken.
Chatgpt can absolutely do the things here: it can give you files, it can integrate and show web pages that you develop, etc.
There is no need for an "agent", the chat version works just fine and is actually easier for novices.
Isn’t that kinda the point, though? If you already lack the required knowledge to ask a web-based chat UI (which, as an aside, is almost exclusively referring to ChatGPT Free for the average user) to build a HTML page, how could you possibly know to go search for Lovable, let alone figure out agents?
I feel as though the gap between the theoretical power of LLMs and what the average user knows of them and their capability have already widened so far that it’s irreconcilable.
> how could you possibly know to go search for Lovable, let alone figure out agents?
Is this an actually serious question? Am I losing my mind here?
I didn't tell my HR friend about Lovable: she found it on her own. Of course someone's not going to Google for Lovable if they've never heard of it, but they might Google for "how do I build a website without knowing anything about it?", or ask ChatGPT the same question.
They're also, most likely, getting endless ads for AI services that help you build various kinds of software shoved into their faces all the time - these ads may not couch the value ad in exactly these terms, but that's fundamentally what they're advertising.
Not everybody is like this but there are plenty of people in the world who, when they have a problem, are quite motivated to find ways to solve it off their own backs.
> Is this an actually serious question?
Yes, it is. I just Googled that exact question (and I promise I'm not trying to be obstinate when I say I'm Googling the exact phrase!) and it's automated AI response was to use WiX or Squarespace. Prompting it further with "What if I want it to do bespoke things that Squarespace can't do?" it responded with using Figma to design the website UI and then pass it along to either Framer or a "professional developer".
I do genuinely think that this is a discoverability issue. Of course, if you prompt it further with "Could I use AI to do this?" it dutifully responds that it can help with generating HTML, but that's three layers of difficulty to eventually get whatever default model Gemini has for signed-out Google searches to even suggest HTML.