What's easier to use as inspiration for coming up with a list of requirements?:
1. A blank sheet of paper
2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
What's easier to use as inspiration for coming up with a list of requirements?:
1. A blank sheet of paper
2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
Definitely this. I've had plenty of useless meetings with no helpful outcomes that lasted an hour with 6 or more people in them, and a few of them will have had more than $4 spent on snacks, let alone everyone's wages.
That said, poor communication will always get you no matter what: had one day spent going back and forth with a client, they kept saying they wanted a button "wider" each time around, I ended up making it full screen width and they still said "wider", asked them to draw a picture and they finally said "wider vertically". That may or may not have been faster with an LLM, depending on if the LLM ever thought to ask the client to draw what they meant, but fundamentally would have been exactly the same problem.
It goes beyond this, the job is to understand the underlying need. When a clients asks for a button to be bigger, what's in their head is they fear it's not visible enough. Many times the client was happy when I implemented something unrelated but that solved the issue, like changing the colours to increase the contrast between the background and the button, adding space around, etc. so that it pops out more.
> 2. A one-shotted POC in the span of an hour with $4 of tokens that is clearly wrong in 20 different ways?
A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time...
> A meeting between all the stakeholders to agree on requirements easily runs $1,000/hour. Taking your $4 functional PoC into that meeting is going to save so much time..
It is though! Because previously your have the meeting last 4x as long to end up with a list of notes that you already knew.
Now you can generate the PoC, showcase it during the meeting and leave with actionable changes in project direction.
I had to re-read parent's comment over and over, because I simply was not sure if he was being ironic, facetious, or just had trouble expressing his point.. and, honestly, I am still not sure.
Not everyone in meeting will be the same person you are. Visuals help. Prototypes help. They certainly can save us from vague ideas and unclear language, because at least someone can point to the big button at the bottom and say:
"Can I have this in cornflower blue?"
It is useful, simply because it makes it more concrete. Will it change? Is the pope catholic?
I was being entirely serious. Even at $400 of tokens it would probably still be worth it to take a functioning prototype into the discussion. At $4 it is a steal
Ahh, thank you. I initially read it as $4 not being worthy of the time of combined $1000/h meeting. Clearly, more coffee is needed.
I used to spend an hour throwing a PoC together with PHP (or sometimes in MS Access) because hey you can throw shit together real fast, and because I used either a shitty language or a toy tool they were treated as throwaway.
Now LLMs make that easier to do with a larger scope.
I use PowerPoint for that.
Draw outlines, or use graphics, wire up clickable areas, and interaction flows, and you’ve got a presentable “live” demo that doesn’t run afoul of issues presenting demo sites to non-techies (placeholder text, ugly buttons, test domains, colours, etc).
I used Balsamiq for that. The “pencil” render of the shapes make it obvious to not discuss aesthetics issues and only focus on flows.
Obviously "a blank sheet of paper" here is meaning thinking about the problem deeply. I'm surprised how dismissive people are to using brainpower whatsoever. I get that juxtaposing paper to LLMs might have triggered some people though.
As a counterpoint, bringing a one-shotted POC without thought may actually do have a shock and awe sort of effect, dazzling the stakeholders to go down the wrong path since the AI prototype was so easy.
> I'm surprised how dismissive people are to using brainpower whatsoever.
Scrooge McDuck lived by the credo: "Work smarter; not harder."
But for many people, smarter is harder.
A one-shotted POC can also put you straight into the very wrong direction, with a very high degree of certainty.
I also prefer doing a POC, but it's not without risk, especially if your stakeholders are just a bit too eager. Another problem I've seen is that a POC can hide complexity, so a lot of weight is on you as the author to do the due diligence before and bring that along.
This is an often underrated problem. I used to work at a company where we would hold demos every other friday, with drinks and nibbles. We would put a lot of thoughts into what we were going to present, and were really careful about the messaging, because execs and sales were watching.
The last thing we wanted was for an "Isn't it neat?!" hacked-together demo to be mistaken for something shippable, if workable at all.
"A one-shotted POC"
As long as someone doesn't go - "Why do we need requirements, you've built the system already!"....
Long ago I have read somewhere a good method to prevent this:
When doing a proof of concept, always use ugly black-and-white buttons and icons. That makes it obvious even to the non-technical people that it is not finished yet, and still requires some work to do.
Sure, the concept of a _Vomit Draft_ is not new, but the ability to come up with one quickly is.
It plays into the generalisation that people aren't good about saying what is the right way to do something, but much more open to saying what something is doing the wrong way.
In a previous job we used to do prototypes in languages like Perl because we couldn't ship them and knocking something non-production ready was easy. We were able to quickly come up with the PoC, iterate on it, and then plan and write the whole thing in $LANGUAGE_OF_CHOICE.
People are much better than saying what they don’t like than what they do.
Using the "let run multiple agents run amok" approach you surely get a lot of "i don't like that part"-feature requests, yes.
In my opinion, a good engineer is not good because he understands every single obscure technical detail, but because he can build solutions that the people using his technology actually want. The list of things they don't want on the other hand is endless.
Personally at some points (like frontend design) I just ask the agent for 16 options with a convenient comparison system and just yeet designs until I’m happy, guiding it along in a very uncommitted way. “Quantity has its own quality”
That's because good design is very hard to identify, while almost everyone can immediately identify a bad design.
Theyre also inclined to jump into talking about solutions rather than their actual problem which inhibits analysis of the problem.
What if there’s many more POCs they don’t like than those they do?
The latter, clearly? It is far more valuable to have a (badly) working system you can prod and probe than work from fantasy.
What's the most soul crushing?
If me planning the requirements takes >1h the the POC is certainly the cheaper option
"Oh no problem, I'll just run it again telling the LLM to not do those 20 things, and then we'll have the right answer." /s-mostly
It was pretty surprising to me when I realised I could put 5 different bug reports into the same message and it would fix all of them.
Not quite as shocking as the original InstructGPT video, but surprising.
What's even more surprising is that it can be given a grafana API key and reason about bugs from the hints of patterns in the logs even with pretty sketchy and inconsistent logging in places.
It can then raise, investigate and resolve those problems while also patching up and rationalising logging to improve future work.
The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Such low level grunt work is what AI is good for, no one denies that.
What some of us are saying is that AI is poor at figuring out what to build, how best to architecture it, and what to avoid.
Again, no, "what to build" is the humans input, but it's not just grunt work that modern AI tooling is good at... Architecture and what to avoid are also strengths of AI.
> Architecture and what to avoid are also strengths of AI.
Absolutely not at all. Every single day that I work with Claude, there is abundant evidence that AI sucks at these. Ask me for examples if you need them.
Yes, please provide examples.
See these comments of mine, which I added exactly for cases like these where someone asks for evidence:
https://news.ycombinator.com/item?id=50220976
https://news.ycombinator.com/item?id=50165791
https://news.ycombinator.com/item?id=50159827
The first links shows a basic failure of reasoning, which would have gone unnoticed if I was just vibe-coding.
The second link shows weak architectural sense and an inability to see the big picture.
The last link in particular is telling: the agents worked with me (over weeks to months) to implement a db abstraction layer without ever bringing up the fact that as it stands, that would have never worked with any indexes at all. I did not constrain them to my choices, but they never stopped me with that design. It took my eagle-eyed attention to their offhand comments to notice this problem, which then required a lot of rework to fix.
Unfortunately, all three comments have apparently been flagged by somebody for some reason and I can't read them. This is my least favorite aspect of hacker news. It's exhausting, honestly.
Like a sibling comment replied, you can set "showdead" to "yes" on your profile and click update" so see flagged comments. Not sure why they were flagged.
I have read that comments which appear to have been written by an LLM are auto-filtered. As these appear to mostly consist of quotations of LLM output, I expect this would be why.
I suggest putting them on a blog or something and linking to that, rather than HN comments.
Go to your profile, and you can set a flag 'showdead' as = yes. Then you'll be able to see flagged/dead comments.
For flagged threads, you can go to:
https://news.ycombinator.com/active
I'm old enough to remember when things such as this were not considered "low level grunt work".
I'm writing a unique MUD, and It was surprising to me when I posted some bug reports I'd made and it identified things I didn't complain about. "User had trouble with X" and then suggested mitigations for those problems. I ended up rejecting a lot of them because I knew I had different mitigations coming eventually, or it was working that way for a reason...
But it was pretty eye-opening to see how good it was at watching a user struggle and identifying possible fixes.
> The AI deniers have their heads in the sands at this point and are demonstrating they simply haven't tried a modern agent. ( Or perhaps are forced onto copilot? )
Could be either, but also the free version of ChatGPT seems rather braindead compared to the paid version, or to the free version of Claude.