As someone who is searching for a new programming contract right now, reading all of the incredible abilities here is pretty intimidating. Especially since I get almost all of my projects from Upwork which is an outsourcing site.
I believe I am competing directly with these frontier models in some circumstances. Like there are a ton of programmers who previously would be outsourcing work to that site, but now they assign that same work to AI agents.
Ever since November 2022 when ChatGPT blew up, I have been focusing on agents in order to try to get ahead of the curve. But I haven't managed to get an agent business off the ground and have been doing poorly paid agentic projects from that site instead.
But now everyone is building agents, and this crazy list of accomplishments makes it look like we are close to the point where the agents are building agents.
In fact the next time I get an Upwork contract for another agent, I actually should run it through my agent and see how far it can get. What I'm seeing a lot of now is requests to automate as much of a business as possible.
Anyway the point is these models are just about capable of doing the entire job of analyzing a small business and building out all the agents and iterating on them with the business owner.
That's actually what I should build is a SaaS that does that. Which I would if I wasn't basically desperate to get another contract this week.
And I know Upwork is bad but I have not had much success with other options on short notice.
> Anyway the point is these models are just about capable of doing the entire job of analyzing a small business and building out all the agents and iterating on them with the business owner.
If you search for LLM benchmarks focused on real-world tasks, you'll quickly learn this ain't the case. No point in telling business owners about that though, they need to see/learn by themselves.
> they need to see/learn by themselves
The cleanup pay is mighty generous as well.
I just finished two solid months of my life cleaning up a vibe coded app.
Has anyone gotten a position as a post-vibe janitor yet?
You got problems there too, but I don't do shitters. That's Engineering.
he is the real world. he is telling you that it is the case. when will you people wake up. this shit is real and happening. people who would have reached out to you asking for help are now not doing that. people to whom you would otherwise have gone for help are no longer hearing from you or never met you in the first place.
If only LLM benchmarks could benchmark it in the first day!
Still no Artificial Analysis benchmark yet. Or benchmark for Laguna S 2.1 or Meituan models or lots of other models.
According to a few tasks from my little personal coding benchmark it's very good at coding and kinda bad at web design. (Also excellent at "draw me a picture" one-shot prompts, for whatever that's worth)
On a sneaky one that involved parsing MIME headers and dealing with character encodings it did better than Kimi K3 at Max and for 38% lower cost.
Interestingly it seems noticeably better than the qwen3.8-max-preview model they offered just a few weeks ago.
What does the benchmark even mean when people are using AI to make real world things that solve real world problems?
I see people, and my self making amazing things with AI and fixing old projects and having real world impact at the fraction of the cost it would take me to hire people, or hours spent on my own coding.
I have built tools and systems with AI that have allowed me to build windows drivers, android apps, web apps, iOS apps, vm occultation, custom block drivers, custom file systems and more. To the point where entire products have been created.
Not trying to be a doomsday, but yes. It seems as though with the right infrastructure we are at the point where businesses owners can go from idea to product very fast and not need or hire much external talent.
What amazing windows drivers have you sold?
Idk about sold. But it’s loaded on all the windows machines in a fairly big company that solves a real world problem.
It allows us to apply custom ACLs to AI agents and the child process spawned by AI agents. Giving us the ability to control what files an AI agent can read or write to, while still being in the calling users context. It allows us to force all ai derived processes to use a transparent MITM proxy so we can then also apply robust access rules to remote host allow or deny access to specific urls and not others. It also allows us to monitor access to windows Credential Manager with rules ti allow specific singed binaries to access some credentials but not others. It give us complete control of what AI agents on windows can see or not see or access.
Windows native sandboxing is lacking. You have some stuff in WSL that completely are broken once you call a windows native app. Or you have app containers which are too restrictive and result in applying expensive file system ACL to all files the app containers would access, which can take hours when dealing with million of files, and would be required to be applied every time you chains your app container (there are some workarounds, for them but they still have a one time cost a long with a fairly flaky maintenance process). You can get the network part done by running commands as a different user but that would result in the same file system ACL nightmare that app containers has.
Result is we get seatbelt level sandboxing in windows native, and can apply dynamic rules like preventing access to .aws folders regardless of the OS level ACLs, using glob rules like */.aws, so we don’t have to be aware of the exact path ahead of time.
It also has registry tree ACLs and, can prevent process and process trees from gaining administrative access, the list of features goes on and on.
so you decided to vibe code a sandbox because windows sandboxing is lacking, instead of moving to linux where sandboxing is robust?
Some products and companies require windows.
It might please you to know that this driver was vibe coded from a linux vm based sandbox.(that was also vibe coded)
A better question is which products/services have you avoided purchasing.
Suppose I wanted you or someone else on Upwork or Fiverr to port a Rails 4 app to Rails 8 (or React or HTMX or anything up to date and maintainable).
Assume the business logic and all edge cases work in the legacy app. The app is "done", just too old to work on or run on modern hosts. Hence the project.
Would/could you use AI to deliver the project at 10x the speed? Or at 1/10 the price? Or charge the same amount as ever?
I have a feeling the lowest-cost developers on freelance marketplaces are not very good at using the AI tools (and whatever related acumen) and need a lot of hand-holding, the same amount of effort almost as if I decided to guide AI to do it myself.
IMO, there's value you can sell in being someone that (even developers) can hand off a project to, to do end to end, if it's fully specified. Even though the buyer could do it themselves, the developer would be taking on managing it and having AI do it until it's correct, and all the little things, few as they may be.
AI should be giving contract developers superpowers. If it's not, then the limited set of contractors with the AI superpowers will be able to charge 10x as much for their time, and the actual cost of the project remains the same even if it's done 10x as fast/easily.
Curious if you see/feel this, or if there's a marketplace for "please vibe code this for me, make sure it works and is what I want".
There is still some use to having someone else supervise it and/or test it etc. But most of that gap could probably be closed by setting up your harness so that it can efficiently do the end to end testing including UI itself. Recent model releases seem to know how to set up playwright or whatever and test the live page on its own. But I still see some improvement to be made when I look at it myself.
I would say 90% of outsourced developers are going to use AI just like someone making top US dollar. Actually for the majority of projects on Upwork, the need to generate with AI is much greater. And I think it's nonsense the idea that an overseas programmer can't figure out how to make adequate requests to a frontier model.
The models are smart enough that someone could literally just say "convert to Rails 8 like you are real real good best Railsdev" and follow up with a few "now test and make the bestest code Base" and if they are repeating that maybe with a few different leading models.. will a real Rails dev be able to find some fault? Sure. Would they have also found issues with the work of another human Rails dev? Yes probably.I just don't think it takes as much actual skill at this point. And certainly there is no secret ability that people in San Francisco have but not in Bangladesh or only certain neighborhoods or whatever.
One guy I was talking to said he was frustrated because his developer was not using AI enough. I came to the conclusion based on his requirements and relatively generous project timeline that that developer was deliberately avoiding using AI or admitting that he was using AI because he knew if he did then the main project phase would be finished in less than a month.
It does help dealing with what previously were often impossible timelines. Recently my Upwork profile has focused on AI agents so I have not been seeing many of those typical line of business type jobs.
But also more than half of those more typical business web apps are now starting with "I have already generated a prototype" and they want it to be made. "production ready" which I think contractors are taking advantage of how vague that is especially since the majority of these projects have pretty modest actual usage. I actually don't like that because it's much less concrete than delivering actual features.
Im sorry to hear about your situation. Have you blogged about it? I’m curious about how the volume of your type of work has been changing over the past 2-3 years.
> Im sorry to hear about your situation. Have you blogged about it?
I don't know why but this made me laugh out loud. I know you're trying to help but just a funny jump
I stopped blogging years ago when I realized that only bots were reading the posts.
I think the volume for custom agents is probably higher than ever but so is the competition for that work.
That working is as not common as college dropouts becoming successful entrepreneurs. But both become stereotypes. But I know you might also just want to read about it.
PS. Not everyone (in fact the rare few) write as if no one is reading. For most, literally no one ever reads :D
As someone else who is an IT consultant.
You need to run away from any client who thinks your primary purpose is to write code. You must run away from the business if YOU think your primary purpose is to write code.
You should be able to write code and do it well, But the AIs, and I'm not even talking about the ones on the frontier, have been able to write code faster than I have for quite some time.
What you need to explain to your customers is all the things around writing code: software architecture, performance, and so on. Also make sure you have some understanding of the customers business, so you can suggest additional ways to make their lives better/make more money.
Software developers have a chance. Code monkeys? None.
The problem is Claude Fable is now better than most programmers I know at software architecture and performance optimization as well.
bingo. to give an analogy in terms of law firms - you can either be a partner who brings in new business or the associate who does the grunt work of reviewing/writing contracts.
guess one which is valuable ?
That's a popular type of AI cope.
First, I am aware they have been able to write code for some time. I made my first LLM coding agent experiment the day after ChatGPT first blew up in November 2022.
I haven't been a "code monkey" in the last say 15 or 20 years of my programming. I would argue also that code money was never really a thing, it was just an excuse that pretentious overpaid developers used for the existence of underpaid exploited programmers. 95% of programmers have to do requirements analysis and design regardless of their pay grade. I learned a long time ago how important requirements analysis and good design is. And the jobs I am applying to largely specify architecture and requirements analysis in the description. Employers are aware of code generation tools.
But also, the premise that writing the code is not the hard part is just nonsense. Sure requirements analysis is harder than people realize, good design is hard and so is understanding things about proper and closed iteration. But that stuff is not, in the aggregate, harder than the code.
And also, frontier models can absolutely do requirements analysis and architecture. And the sheer speed of implementation means that there is a huge built-in advantage for iterating more effectively.
Two points of disagreement:
"Code monkey" absolutely was and still is a thing. I was hired by Big Name Co originally to be staff but a reorg had me do typical senior position duties for around a year, and it was hell. Every decision was made over our heads, feedback was unwelcome because the feature had already gone through design and gotten layers of sign-offs from different areas of the org, etc. My sole purpose was to write code and review peer code. Staying there a full year was way too much.
Second point, requirements analysis is hard because:
* Different stakeholders have conflicting goals, sometimes mutually exclusive * Some stakeholders have ideas that are actively harmful to users, and you must flex political capital to nip it in the bud * Some stakeholders think they know what they want with a far greater degree of clarity than they actually do, and lack the time to sit down and suss out the nuances, because you're a side-show in the grander scheme of their own priorities.
Writing code is a heck of a lot easier than dealing with people at the edges.
Those are good points, but that mostly comes down to lack of organizational status or office politics etc. which is independent of capability level and largely orthogonal to whether it's a human programmer or an AI agent. Unless you are redefining "software engineer" to be more like a political operator. But that goes for any human group and any job. And anyway after a certain level, not providing the senior engineers and or project managers with sufficient political sway to realize projects goals is a management failure. Unless again you are defining software engineer as someone who is a master social manipulator in the context of their job.
A big part of an engineering performing requirements analysis is owning the technical understanding of the requirements and identifying areas where the requirements are not technically feasible, can be done with less expense with adjustments, or can be better achieved with a different solution.
AI will help you draft the communication of that knowledge, but it won't help you actually deliver the knowledge or ensure that the knowledge was appropriately received. As you say, it is a fundamentally human problem, which is why I maintain that writing code is easier.
I thought this comment was going to be about leveraging agents in ways your existing clientele demographic cannot or will not (yet|ever),
but then you totally jumped the shark!
Can you give me a hint about wtf you are talking about?
To say that you're building agents, is based on a way of viewing things that isn't at all pervasive. Some might say you're building a customer support chatbot. To talk of building agents as a common activity makes sense if you use LangChain I guess, where its title is "Open Source AI Agent Framework | Build Agents Faster". But for instance, YC just released qm, and in its README it talks about customizing the agent, which is quite a bit different from just casually dropping the phrase "I'm building an agent". https://github.com/yc-software/qm
Not sure what you are trying to say but thanks for reminding me about qm, it has some cool features.
My MindRoot framework had some of qm's main features awhile ago though. I do try to use it to build solutions by customizing rather than from scratch when possible. But a lot of clients or potential clients don't like that idea even though it's MIT. They want some that has clearly been invented by them as a new programming project.
I actually turned down a project last year from someone who wanted me to start over in LangGraph -- he had already decided on it before he knew about my thing.
The most recent project I have been using MindRoot and building up the voice capabilities so I can now handle voice agents end to end including SIP.
But on that website now I try not to emphasize my framework at all for most proposals. You don't have leverage on Upwork. And also with code generation it isn't a critical point necessarily.
Why are you looking for projects on upwork if you know they don’t pay well and that you don’t have leverage? That sounds like a pretty bad place where to find contracting projects