Copying from another post. I’m very puzzled on why people don’t talk more about essential complexity of specifying systems any more:
In No Silver Bullet, Fred Brooks argues that the hard part of software engineering lies in essential complexity - understanding, specifying, and modeling the problem space - while accidental complexity like tool limitations is secondary. His point was that no tool or methodology would "magically" eliminate the difficulty of software development because the core challenge is conceptual, not syntactic.
Fast forward to today: there's a lot of talk about AI agents replacing engineers by writing entire codebases from natural language prompts. But that seems to assume the specification problem is somehow solved or simplified. In reality, turning vague ideas into detailed, robust systems still feels like the core job of engineers.
If someone provides detailed specs and iteratively works with an AI to build software, aren’t they just using AI to eliminate accidental complexity—like how we moved from assembly to high-level languages? That doesn’t replace engineers; it boosts our productivity. If anything, it should increase opportunities by lowering the cost of iteration and scaling our impact.
So how do we reconcile this? If an agent writes a product from a prompt, that only works because someone else has already fully specified the system—implicitly or explicitly. And if we’re just using AI to replicate existing products, then we’re not solving technical problems anymore; we’re just competing on distribution or cost. That’s not an engineering disruption—it’s a business one.
I think the crux is that specification has been neglected since even before AI.
Stakeholders (client, managers) have been "vibe coding" all along. They send some vague descriptions and someone magically gives back a solution. Does the solution completely work? No one knows. It kinda works, but no one knows for sure.
Most of the time, it's actually the programmers' understanding of the domain that fills out the details (we all know what a correct form submission webpage looks like).
Now the other end has become AI, it remains to be seen whether this can be replicated.
This is spot-on, and a comment I wish I could pin for others to see.
GenAI is such a boon at present because it occasionally delivers acceptable mediocrity to PMs and Stakeholders who will accept said mediocrity because they have no real clue what they (or their customers) actually want. It’s a system of roles and output that delivers based on patterns in incomprehensively large data sets, provided to humans who cannot hope to validate that information is accurate or legitimate (and not just random patterns in a large enough data set), but passed along as gospel from digital machinegods. To a withering empire obsessed with nostalgia and whose institutions are crumbling beneath them, GenAI appears as a savior; to those confident in their position in the world order, it is merely a thin tool in a larger toolbox, to be used toward greater ends rather than middling output.
Those who understand the specification problems are in a position to capitalize off such monumental shifts, while those blindly chasing get-rich-quick schemes, grifts, and fads will be left behind.
The split between essential and incidental complexity is a really key insight for thinking about how far AI can be pushed into software development. I think it's likely the detail many developers are feeling intuitively but not able to articulate, in regards to why they won't be replaced just yet.
It's certainly how actually using AI in earnest feels, I have been doing my best to get agents like Claude to work through problems in a complex codebase defined by enormous amounts of outside business logic. This lack of ability to truly intuit the business requirements and deep context requirements means it cannot make business related code changes. But it can help with very small context code changes, ie incidental complexity unrelated to the core role of a good developer, which is translating real world requirements into a system.
But I will add that it shouldn't be underestimated how many of us are actually solving the distribution problem, not technical problems. I still would not feel confident replacing a junior with AI, the core issue being lack of self-correction. But certainly people will try, and businesses built around AI development will be real and undercut established businesses. Whether that's net good or bad will probably not matter to those who lose their jobs in the scuffle.
A terrifyingly large percentage of people employed to write software cannot write software. Not even a little. These are the people that can be easily replaced.
In my prior line of work I wrote JavaScript for a living. There were people doing amazing, just jaw dropping astounding, things. Those people were almost exclusively hobbyists. At work most people struggled to do little more than copy/paste in a struggle just to put text on screen. Sadly, that is not an exaggeration.
Some people did what they considered to be advanced engineering against these colossal frameworks, but the result is just the same: little more than copy/paste and struggle to put text on screen. Yes, they might be solving for advanced complexity, but it is almost always completely unnecessary and frequently related to code vanity.
Virtually none of those people could write original applications, measure anything, write documentation, or do just about anything else practical.
> So how do we reconcile this?
Alienate your workforce by setting high standards, like a bar exam to become a lawyer. Fire those people that fail to rise to the occasion. Moving forward employ people who cannot meet the high standards only as juniors or apprentices, so that the next generation of developers have the opportunity to learn the craft without rewarding failure.
> Alienate your workforce by setting high standards, like a bar exam to become a lawyer ...
This would work if the world was willing to pay for software. So at the very least you'd have to outlaw the ad-based business model, or do what lawyers do: things that are absolutely critical for software development (think "program needs to be approved or it won't execute", that deep) that normal people aren't allowed ... and unable ... to do.
An easy answer for what's missing is that the industry isn't run by people who read "No Silver Bullet".
- Articles about tricky nature of tech debt aren't written by people who call the shots on whether the team can spend the entire next week on something that a customer can't see.
- Articles about systems architecture aren't written by people who decide how much each piece of work was significant for business.
- Books on methodologies are optional read for engineers, not essential for their job, and adoption happens only when they push it upwards.
Most of the buzz about AI replacing coding is coming from people who don't see a difference between generating a working MVP of an app, and evolving an app codebase for a decade and fixing ancient poor design choices in a spaceship which is already in flight.
I've even seen a manager who proposed allocating 33% time every day on 3 projects, and engineers had to push back. Such old and common knowledge that it doesn't work is apparently still not a job requirement in 2025. Despite that organizing and structuring project time allocation is management competency and not an engineering skill, it is de-facto entirely up to engineers to make sure it's done right. The same managers are now proud to demonstrate their "customer focus" by proposing to ask AI to resolve all the tech debt and write all the missing tests so that engineers could focus on business requests, and same engineers have to figure how to explain why it didn't just work when they tried.
To talk about complexity is to repeat the same old mistake. I am sure most engineers already know and I am yet to see an experienced engineer who believes their job will be taken by simple prompts. The problem we should be talking about should be titled something like,
"Software Engineering Has Poor Management Problem, and AI is Amplifying It"
People (especially people who don't have a lot of hands on tech experience, or students who also aren't into building things) get the sense that writing software requires learning a lot of arcane tools. And the idea is to promise that anyone who can write a specification should be able to make software (yes, handwaving away the learning to specify well, which is a real skill with many dependent skills). This was the promise of no-code, and then they realized that the no-code system (in addition to usually being limited in power) is actually complex and requires specialized learning, and more of that the more powerful the system is. The LLM will replace SWEs approach is another take on that, because you don't need to learn a system, you prompt in natural language, and the model knows how to interface with the underlying system so you don't have to. In that sense, vibe coding is already the culmination of this goal (despite weaknesses such as maintainability issues).
I've seen it written that the main reason managers tend to want to get rid of SWEs is because they don't understand how to interface with them. Using an LLM solves that problem, because you don't need a nerd to operate it.
> I've seen it written that the main reason managers tend to want to get rid of SWEs is because they don't understand how to interface with them
That’s because software is nebulous enough that you can get away with promising the moon to customers/boss, but in the next meeting, you’re given a reality check by the SWEs. And then you realize the mess you’re thrown everyone in.
Managers knows how to interface with SWEs well (people interface with professionals all the time). Most just hates going back to the engineers to get real answers when they fancy themselves as products owners.
> they don't understand how to interface with them
In other words, they are not good managers. A good manager should be able to work with all kinds of people with different backgrounds and personalities.
> Using an LLM solves that problem, because you don't need a nerd to operate it.
Until you do. LLMs are great ar building prototypes but at some point if you don’t know what you’ll doing you’ll end up with an unmaintainable mess and you won’t have anyone to fix it.
I mean LLMs perhaps are capable of doing that too but they still need to be guided by people who are capable of understanding their output.
Being able to reduce the number of engineers that you need by e.g. 80% would still be a great deal though.
Kind of, but the models also output really awful code, even if it appears to work, and people (especially juniors) will push that awful code into PRs and people eventually approve of it because there's engineers who don't care about the craft, only collecting a paycheck. Then when things break or slow down to a crawl, nobody has any idea how to fix it because its all AI generated goop.
I'm not sure what the answer is - but I will say that LLMs do help me wrangle with essential complexity / real-world issues too.
Most problems businesses face have been seen by other businesses; perhaps some knowledge is in the training set or perhaps some problems are so easy to reason through that a LLM can do the "reasoning" more-or-less from first principles and your problem description.
I am speculating that AI will help with both sides of the No Silver Bullet dichotomy?
So in other words it's helping you race to the median. It can give your business the advantage of moving always in a direction that's average and uninteresting. Nobody will need to lead anymore, so nobody will have the skill of a leader anymore.
It sounds to me like a corporate equivalent of a drug-fueled rager. They want everything good now while deferring all the expenses to tomorrow.
Yeah, I give it about two years until we get to "Hey AI, what should we do today?" "Hi, I've noticed an increase in users struggling with transactions across individual accounts that they own. It appears some aspect of multitenancy would be warmly received by a significant fraction of our userbase. I have compiled a report on the different approaches taken by medium and large tech companies in this regard, and created a summary of user feedback that I've found on each. Based on this, and with the nuance of our industry, current userbase, the future markets we want to explore, and the ability to fit it most naturally into our existing infrastructure, I have boiled it down to one of these three options. Here are detailed design docs for each, that includes all downstream services affected, all data schema changes, lists out any concerns about backwards compatibility, user interface nuances, and has all the new operational and adoption metrics that we will want to monitor. Please read these through and let me know which one to start, and if you have any questions or suggestions I'll be more than happy to take them. For the first option, I've already prepared a list of PRs that I'm ready to commit and deploy in the designated order, and have tested e2e in a test cluster of all affected services, and it is up and running in a test cluster currently if you would like to explore it. It will take me a couple hours to do the same with the other two options if you'd like. If I get the green light today, I can sequence the deployments so that they don't conflict with other projects and have it in production by the end of the week, along with communication and optional training to the users I feel would find the feature most useful. Of course any of this can be changed, postponed, or dropped if you have concerns, would like to take a different approach, or think the feature should not be pursued."
I think AI also introduces a new form of accidental complexity. When using Copilot, I often find myself telling it something to the effect of, “this seems needlessly complex and confusing, is there a better way to do this, or is this level of complexity justified?” It almost always apologizes and comes back with a solution I find much more pleasing, though on rare occasions it does justify its solution as a form of the original accidental complexity you mention. The more we lean on these tools, the more this accidental complexity from the model itself compounds.
Actually, you're not missing anything. The thing is, hype cycles are just that, cycles. They come around with a mix of genuine amnesia, convenient amnesia, and junior enthusiasm, because cycles require a society (and/or industry) both able and willing to repeat exploration and decisions, whether they end up in wins or losses. Some people start to get get the pattern after a while but they are seen as cynics. After all, the show must go on, "what if this or the next cycle is the one that leads us to tech nirvana?"
Software engineering for any non-trivial problem means a baseline level of essential complexity that isn't going away, no matter the tool, not even if we someday "code" directly from our minds in some almost-free way via parallel programming thought diffusion. That's because (1) depth and breadth of choice; and (2) coordination/socials, mostly due but not uniquely related to (1) are the real bottlenecks.
Sure, accidental complexity can shrink, if you design in a way that's aligned with the tools, but even then, the gains are often overhyped. These kinds of "developer accelerators" (IDEs, low-code platforms, etc.) are always oversold in depth and scope, LLMs included.
The promise of the "10x engineer" is always there, but the reality is more mundane. For example, IDEs and LSPs are helpful, but not really transformative. Up to a point that people are being payed right now and don't use them at all, and they still deliver in a "economically justifiable" (by someone) way.
Today it's LLMs. Tomorrow it'll be LISP Machines v2.
There is a lot of truth in No Silver Bullet and I had the same idea in my mind.
Downside is there is much more non essential busy work because of which people had their jobs and now loads of those people will lose the job.
People who do work on real essential complexity of systems are far and between. People who say things like "proper professionals will always have work" are utter assholes thinking mostly that "they are those proper professionals".
In reality AI will be like F1 racing team not needing pit workers and have only drivers, how many drivers are there like 20 so it is 10 teams each having 2 drivers. Each team has 300-1000 people that do all the other things.
If you go to corporate level let's say 1 person working on essential complexity requires 10-20 people doing all kinds of non essential stuff that will be taken over by AI, or to be realistic instead of 10-20 people that person will need headcount of 5.
That is still 15 people out of job - are those people able to take over some essential complexity in a different company or different area of the same company, some would but it is also if they would like to do it. So those people will be pushed around or end up jobless, bitter whatever.
What you are saying is true. In a way, programmers still need to think about and wrestle with architectural complexity. And I agree the biggest overall gain is adding another layer of abstraction. But combine those two things and suddenly you have junior engineers that can very quickly learn how to architect systems. Because that will be the bulk of the job and they will be doing it every day.
Once you remove all the roadblocks with syntax and language blindspots, the high cost of refactoring, the tedium of adding validation and tests, the challenges of integrating systems... suddenly, the work becomes more pure. Yes, you need to know how to do advanced structural things. But you don't need to learn very much about all the rest of it.
And we very quickly get to a point where someone who can break down problems into tidy Jira tickets is effectively programming. Programming was never really about learning languages but making computers do things, which is a transferrable skill and why so many engineers know so many languages.
Even the simplest tickets sometimes that end up requiring a one line change can require hours of investigation to fully understand/stamp the effects of that change.
And perhaps I havent used the greatest or latest, but in my experience LLMs break down hard st anything sufficiently large. They make changes and introduce new errors, they end up changing the feature, or worst case just ouright break everything.
Id never trust it unless you have an extensive amount of good tests for validation
> aren’t they just using AI to eliminate accidental complexity
After using Claude Code to vibe code some stuff, it seems to me that AI doesn't eliminate accidental complexity, it just creates more of it and takes away some of the pain of really bad APIs.
You first start using a hammer well, and then internalizing when to use a hammer. Most are now getting excited about the new shinny hammer. Few knows hammer is not for everything. Some will never know. It has always been the case. Microservice, NoSQL, kubernetes, crypto, web3, now LLM. They range from useful some of the time to completely useless. But they surely appeared to be panacea at some time to some people.
You're missing the part where building a modern website is a huge amount of dev time for largely UI work. Also modern deployment is 100x more complicated than in Brook's day. I'd say 90% of my projects are on these two parts which really shows how productivity has gone down (and AI can fix)
This is mostly self inflicted though. We create complex deployments with the promise that the incremental savings will overtake the upfront costs when they rarely do (and the hidden complexity costs).
So it seems AI will just let us stretch further and make more accidentally complex systems.
Modern development is more complex, not more complicated. We’re still using the same categories of tools. What’s changed is the tower of abstraction we put between ourselves and the problem.
Back when IE was king and IE6 was still 10% of users, I did frontend web work. I remember sitting next to our designer with multiple browsers open playing with pixel offsets to get the design as close as practically possible to the mockups for most users and good enough for every one else. This isn't something LLMs do without a model of what looks good.
I agree on the essential complexity, but I think there is a missing piece that we don't really have good mental tools how to operate (compose) the SW systems with uncertainty. Something like fuzzy logic?
I think there is a promise of that in AI and LLMs (but I remain skeptical, because I it needs a formal and not ad hoc definition). The idea you can build the systems using a fuzzy human language and the things will somehow work out.
> it should increase opportunities by lowering the cost of iteration and scaling our impact
This is very much up for debate and the weakest point of the argument I think. If developers are now 2-3x (remains to be seen..) as productive what will happen to the job market?
I suppose it depends on how much potential software is there that's not currently viable due to cost? How much would scope be increased on current products?
you're spot on. Building software is first and foremost making a team of person understand a problem. The fact that part of it is solved by writing code is almost a byproduct of that understanding, and certainly does not come before.
on this topic I suggest everybody who works in our industry to read Peter Naur's "Programming as Theory Building"[1] and a nice corollary from Baldur Bjarnson: "Theory-building and why employee churn is lethal to software companies"[2]
i think the difference is that now someone with no coding knowledge could start describing software and make the agent build that software iteratively. so for example a mechanical engineer wants to build some simulation tool. you still need to define those requirements and understand what you want to do but the work could be (and this is the big if still, if agents become good enough for this sort of work) done by the agent not a humand programmer. i do not see that happening at the moment but still this does change the dynamic. you are right in that it is not a silver bullet and a lot of the complexity is impossible to get rid of. but i wonder if for a lot of use cases there will not be a software engineer in the loop. for bigger systems, for sure, but for a lot of smaller business software?
A mechanical engineer has a job to do. They can't all spend their time yak shaving with an AI agent building software that they then use to do their actual job. The whole point of building software is it's more efficient to build it once then use it many times. Why would a "someone with no coding knowledge" do this when someone with coding knowledge could do it?
Small businesses often understand domain less, not more, because they cannot invest as much as big businesses in building expertise. They may achieve something within that limited understanding, but the outcome will limit their growth. Of course, AI can help with discovery, but it may overcomplicate things. Product discovery is an art of figuring out what to do without doing too much or not enough, which AI has not mastered yet.
This totally breaks due to complexity. This "works", except that AI destroys earlier functionality when adding new things. Also there is a very low level of complexity where AI just blows up and won't work anymore, not even a little bit.
(and certainly Google's newest AI model is actually a step backwards on this front)
Add to that that nothing changes for difficult work. Writing a driver still requires hardware knowledge ... of the actual hardware. Which AI doesn't have ... and doesn't make any attempt to acquire.
I've seen articles that this part can actually be fixed. If you create a loop where the AI is forced to slowly build up low-level knowledge it can actually find this sort of things. But you need 10 expert AI researchers to do anything like this.
(frankly I don't see how this part could be fixed by better AI models)
What's in danger is the coding job consisting of "write me a form with these 20 fields". The 100% repetitive ones.
Making any non trivial system with AI only highlights this problem. My repo is littered with specs the AI has to refer to to build the system. But the specs are unclear files that have been added to and grown outdated over time, so now we often end up gling back and forth without much progress.
I don't think anyone thinks engineers are going away. But companies will hire less employees to do the same amount of work, and they won't pay engineers more.
… because programming languages are the right level of precision for specifying a program you want. Natural language isn’t it. Of course you need to review and edit what it generates. Of course it’s often easier to make the change yourself instead of describing how to make the change.
I wonder if the independent studies that show Copilot increasing the rate of errors in software have anything to do with this less bold attitude. Most people selling AI are predicting the obsolescence of human authors.
Transformers can be used to automate testing, create deeper and broader specification, accelerate greenfield projects, rapidly and precisely expand a developer's knowledge as needed, navigate unfamiliar APIs without relying on reference, build out initial features, do code review and so much more.
Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
> Most people selling AI are predicting the obsolescence of human authors.
It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step.
That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.
> Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
> It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality…
It is not a reality since it has not happen. In the real world it has not happened.
There is no reason to believe that the current rate of progress will continue. Intelligence is not like the weaving machines. A software engineer is not a human calculator.
> That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom
They didn’t just see layoffs. There were the constant wars with Napoleon and the War of 1812 causing significant economic instability along with highly variable capital investments in textile production at the time. They we’re looking at huge wealth disparity and losing their jobs for most meant losing everything.
What many Luddite supporters were asking for in many parts of England were: better working conditions, a raise to minimum wage, abolishment of child labour, etc. Sabotage was a means to make such demands from a class that held almost all of the power.
Many of those protestors were shot. Those who survived and were laid off were forced into workhouses.
The capitalists won and got to write the history and the myths. They made it about the technology and not the conditions. They told us that the displaced workers found new, better jobs elsewhere.
Programmers, while part of the labour class, have so far enjoyed a much better bargaining position and have been compensated in kind. Many of us also complain about the quality of output from AI as the textile workers complained about the poor quality of the lace. But fortunately the workhouses were shut down. Although poor quality code tends to result in people losing their life’s savings, having their identities stolen, etc. Higher stakes than cheap lace.
> That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
See, this is the kind of conception of a programmer I find completely befuddling. Programming isn't like those jobs at all. There's a reason people who are overly attached to code and see their job as "writing code" are pejoratively called "code monkeys." Did CAD kill the engineer? No. It didn't. The idea is ridiculous.
I don't disagree exactly, but the AI that fully replaces all the programmers is essentially a superhuman one. It's matching human output, but will obviously be able to do some tasks like calculations much faster, and won't need a lunch break.
At that point it's less "programmers will be out of work" as "most work may cease to exist".
Doesn't AI have diminishing returns on it's pseudo creativity? Throw all the training output of LLM into a circle. If all input comes from other LLM output, the circle never grows. Humans constantly step outside the circle.
Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.
I think you're either imagining the circle too small or overestimating how often humans step outside it. The typical programming job involves lots and lots of work, and yet none of it creating wholly original computer science. Current LLMs can customize well known UI/IO/CRUD/REST patterns with little difficulty, and these make up the vast majority of commercial software development.
I wasn’t in the industry to see it first hand, but was this same criticism levied against higher level languages when they first hit the scene? Something to the effect of high level languages not being the right level of precision, because the programmer wanted to directly manipulate what was happening in memory, and the high level languages are not the right level of precision for that?
The issue with natural language isn’t that it’s impossible to be precise, it’s that most people aren’t, or they are precise about what they want it to do for them, but not what the computer needs to do to make it happen. This leads to a lot of guessing by engineerings as they try to translate the business requirements into code. Now the LLM is doing that guessing, often with less context about the broader business objectives, or an understanding of the people writing those requirements.
> was this same criticism levied against higher level languages when they first hit the scene?
No.
Some were concerned that the output of compilers couldn’t match the quality of what could be done by a competent programmer at the time. That was true for a time. Then compilers got better.
Nobody was concerned that compilers were going to be used by capitalists to lay them off and seize the means of producing programs by turning it into property.
> Of course you need to review and edit what it generates.
Treating LLMs as a scaffolding tool yields better results at least for me personally. I just brain dump what I'm thinking of building and ask for it to give me models, and basic controllers using said models, then I just worry about the views and business logic.
In my experience the best use of AI is to stay in the flow state when you get blocked by an API you don't understand or a feature you don't want to implement for whatever reason.
AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.
Here's a fresh example that I stumbled upon just a few hours ago. I needed to refactor some code that first computes the size of a popup, and then separately, the top left corner.
For brevity, one part used an "if", while the other one had a "switch":
I wanted the LLM to refactor it to store the position rather than applying it immediately. Turns out, it just could not handle different things (if vs. switch) doing a similar thing. I tried several variations of prompts, but it very strongly leaning to either have two ifs, or two switches, despite rather explicit instructions not to do so.
It sort of makes sense: once the model has "completed" an if, and then encounters the need for a similar thing, it will pick an "if" again, because, well, it is completing the previous tokens.
Harmless here, but in many slightly less trivial examples, it would just steamroll over nuance and produce code that appears good, but fails in weird ways.
That said, splitting tasks into smaller parts devoid of such ambiguities works really well. Way easier to say "store size in m_StateStorage and apply on render" than manually editing 5 different points in the code. Especially with stuff like Cerebras, that can chew through complex code at several kilobytes per second, expanding simple thoughts faster than you could physically type them.
> AI can very efficiently apply common patterns to vast amounts of code, but it has no inherent "idea" of what it's doing.
AI stands for Artificial Intelligence. There are no inherent limits around what AI can and can't do or comprehend. What you are specifically critiquing is the capability of today's popular models, specifically transformer models, and accompanying tooling. This is a rapidly evolving landscape, and your assertions might no longer be relevant in a month, much less a year or five years. In fact, your criticism might not even be relevant between current models. It's one thing to speak about idiosyncrasies between models, but any broad conclusions drawn outside of a comprehensive multi-model review with strict procedure and controls is to be taken with a massive grain of salt, and one should be careful to avoid authoritative language about capabilities.
It would be useful to be precise in what you are critiquing, so that the critique actually has merit and applicability. Even saying "LLM" is a misnomer, as modern transformer models are multi-modal and trained on much more than just textual language.
I learned neural networks around 2000, and it was old technology then. The last real jump we saw was going from ChatGPT 3.5 to 4, and that is more than 2 years ago.
It seems you don't recollect how much time passed without any big revolutions in AI. Deep learning was a big jump. But when the next jump comes? Might be tomorrow, but looking at history, might be in 2035.
According to what I see, the curve has already flattened and now only a new revolution could get us to the next big step.
> AI stands for Artificial Intelligence. There are no inherent limits around what AI can and can't do or comprehend.
Artificial, as in Artificial sand or artificial grass. Sure, it appears as sand or grass at first, but upon closer examination, it becomes very apparent that it's not real. Artificial is basically a similar word to magic - as in, it offers enough misdirection in order for people to think there might be intelligence, but upon closer examination, it's found lacking.
It's still impressive that it can do that, going all the way back to gaming AIs, but it's also a veil that is lifted easily.
What a ridiculous response, to scold the GP for criticising today's AI because tomorrow's might be better. Sure, it might! But it ain't here yet buddy.
Lots of us are interested in technology that's actually available, and we can all read date stamps on comments.
I had a fun result the other day from Claude. I opened a script in Zed and asked it to "fix the error on line 71".
Claude happily went and fixed the error on line 91....
1. There was no error on line 91, it did some inconsequential formatting on that line
2. More importantly, it just ignored the very specific line I told it to go to. It's like I was playing telephone with the LLM which felt so strange with text-based communication.
This was me trying to get better at using the LLM while coding and seeing if I could "one-shot" some very simple things. Of course me doing this _very_ tiny fix myself would have been faster. Just felt weird and reinforces this idea that the LLM isn't actually thinking at all.
I suspect if OP highlighted line 71 and added it to chat and said fix the error, they’d get a much better response. I assume Cursor could create a tool to help it interpret line numbers, but that’s not how they expect you to use it really.
That's the thing. We're expecting the tool to have a clear understanding of its own limitations by now and ask for better prompts (or say: I don't know, I can't etc). The fact it just does something wacky is not good at all to the consistency of these tools.
I do not code/program, but I do read thousands of fiction pages annually. LLMs (Perplexity, specifically) have been my lifetime favorite book club member — I can ask anything.
However, I can't just say "on page 123..." I've found it's better to either provide the quote, or describe the context, and then ask how it relates to [another concept]. Or I'll say "at the end of chapter 6, Bob does X, then why Y?" (perhaps this is similar to asking a coding LLM to fix a specific function instead of a specific line?).
My favorite examples of this have been sitting with living human authors and discussing their books — usually to jaw-dropped creators, particularly to Unknowns.
Works for non-fiction, too (of course). But for all those books you didn't read in HS English classes, you can somewhat recreate all that class discussion your teachers always attempted to foster — at your own discretion/direction.
> This was me trying to get better at using the LLM while coding
And now you've learned that LLMs can't count lines. Next time, try asking it to "fix the error in function XYZ" or copy/paste the line in question, and see if you get better results.
> reinforces this idea that the LLM isn't actually thinking at all.
Of course it's not thinking, how could it? It's just a (rather big) equation.
54 def dicts_to_table_string(
55 headings: List[str], dicts: List[Dict[str, str]]
56 ) -> List[str]:
57 max_lengths = [len(h) for h in headings]
58
59 # Compute maximum length for each column
60 for d in dicts:
Personally, I define the job of a software engineer as transform requirements into software.
Software is not only code. Requirements are not only natural language.
At the moment I can't manage to be faster with the AI than manually. Unless its a simple task or software.
In my experience AI's are atm junior or mid-level developers. And in the last two years, they didn't get significant better.
Most of the time, the requirements are not spelled out. Nobody even knows what the business logic is supposed to be. A lot of it has to be decided by the software engineer based on available information. It sometimes involve walking around the office asking people things.
It also requires a fair bit of wisdom to know where the software is expected to grow, and how to architect for that eventuality.
I can't picture an LLM doing a fraction of that work.
I think that's my problem with AI. Let's say I have all the requirements, down to the smallest detail. Then I make my decisions at a micro level. Formulate an architecture. Take all the non-functionals into account. I would write a book as a prompt that is not able to express my thoughts as accurately as if I were writing code right away. Apart from the fact that the prompt is generally a superfluous intermediate step in which I struggle to create an exact programming language with an imprecise natural language with a result that is not reproduce-able.
Back in the day, there used to be professionals known as systems analysts whose job it was to determine the business logic. The information system went through several phases of stepwise refinement. First the requirements were determined: WHO needs WHAT information, WHERE, WHEN, and WHY. (And from WHOM do they get it?) Then, a series of procedures were developed to be implemented either manually or by computer (The "Playscript" format by Les Matthies was particularly useful for the former; standard flowcharts for the latter). Each piece of data, its type, and its purpose was also identified and registered in a standard vocabulary. The programmers only got in on it towards the end: once the major decisions had been made, they wrote the code instructing the computer on HOW to do it. Their focus was on the HOW; the WHAT was not in their job description, and was usually beyond their capability. It takes holistic, big-picture thinking and great social skills to develop an understanding of an entire business division and come up with a plan to implement an information system for that division. The programmers' work entailed about 15% of the total work, and was mainly concerned with the technical details of getting the procedures to be automated running on a computer. These days, with AI, that 15% figure could be reduced even further, perhaps by a factor of five or ten. In the big-iron era, programming was really that simple. The hard decisions had already been made before the first line of code was written. You may think it stodgy and inflexible to develop software this way, but that's how grown-ups did it, and studies showed that it was significantly cheaper than the scattershot, agile-like approach where the programmers where in the lead in terms of systems design.
What's happened since the 1970s is, programming has undergone prestige inflation to the point where programmers are looked to for decisions regarding the actual business procedures, and usually they just don't have the business knowledge or wherewithal to do that. The systems analyst has faded into memory. And as Milt Bryce said, "If this country built bridges the way it builds systems, we'd be a nation run by ferry-boats."
No software job I've done has come with clear, explicit requirements. Its all been a mess under the hood and half my job has been figuring out specifically what they even want
I think it's a common problem to officially have mids who are really just juniors with a modicum of time in the industry and the same goes for seniors who are really just mids.
One of the most useful properties of computers is that they enable reliable, eminently reproducible automation. Formal languages (like programming languages) not only allow to unambiguously specify the desired automation to the upmost level of precision, they also allow humans to reason about the automation with precision and confidence. Natural language is a poor substitute for that. The ground truth of programs will always be the code, and if humans want to precisely control what a program does, they’ll be best served by understanding, manipulating, and reasoning about the code.
“Manual” has a negative connotation. If I understand the article correctly, they mean “human coding remains key”. It’s not clear to me the GitHub CEO actually used the word “manual”, that would surprise me. Is there another source on this that’s either more neutral or better at choosing accurate words? The last thing we need is to put down human coding as “manual”; human coders have a large toolbox of non-AI tools to automate their coding.
Often when you're calling something "manual" you're taking something off the automated happy path to tediously and slowly wind the thing through its complex process by hand. The difference between manual coding and human coding is tedium and laboriousness. It's not laborious to program, but the phrase "manual coding" evokes that image.
Readit News
In No Silver Bullet, Fred Brooks argues that the hard part of software engineering lies in essential complexity - understanding, specifying, and modeling the problem space - while accidental complexity like tool limitations is secondary. His point was that no tool or methodology would "magically" eliminate the difficulty of software development because the core challenge is conceptual, not syntactic. Fast forward to today: there's a lot of talk about AI agents replacing engineers by writing entire codebases from natural language prompts. But that seems to assume the specification problem is somehow solved or simplified. In reality, turning vague ideas into detailed, robust systems still feels like the core job of engineers.
If someone provides detailed specs and iteratively works with an AI to build software, aren’t they just using AI to eliminate accidental complexity—like how we moved from assembly to high-level languages? That doesn’t replace engineers; it boosts our productivity. If anything, it should increase opportunities by lowering the cost of iteration and scaling our impact.
So how do we reconcile this? If an agent writes a product from a prompt, that only works because someone else has already fully specified the system—implicitly or explicitly. And if we’re just using AI to replicate existing products, then we’re not solving technical problems anymore; we’re just competing on distribution or cost. That’s not an engineering disruption—it’s a business one.
What am I missing here?
Stakeholders (client, managers) have been "vibe coding" all along. They send some vague descriptions and someone magically gives back a solution. Does the solution completely work? No one knows. It kinda works, but no one knows for sure.
Most of the time, it's actually the programmers' understanding of the domain that fills out the details (we all know what a correct form submission webpage looks like).
Now the other end has become AI, it remains to be seen whether this can be replicated.
What they have: An undergraduate intern who is a former used car salesperson used to BSing their way through life.
millions of forms around the web would like to have a word… :)
GenAI is such a boon at present because it occasionally delivers acceptable mediocrity to PMs and Stakeholders who will accept said mediocrity because they have no real clue what they (or their customers) actually want. It’s a system of roles and output that delivers based on patterns in incomprehensively large data sets, provided to humans who cannot hope to validate that information is accurate or legitimate (and not just random patterns in a large enough data set), but passed along as gospel from digital machinegods. To a withering empire obsessed with nostalgia and whose institutions are crumbling beneath them, GenAI appears as a savior; to those confident in their position in the world order, it is merely a thin tool in a larger toolbox, to be used toward greater ends rather than middling output.
Those who understand the specification problems are in a position to capitalize off such monumental shifts, while those blindly chasing get-rich-quick schemes, grifts, and fads will be left behind.
Obviously we don‘t as phone numbers can be split in up to 4 fields with conflicting varieties of validation or just be one field.
It's certainly how actually using AI in earnest feels, I have been doing my best to get agents like Claude to work through problems in a complex codebase defined by enormous amounts of outside business logic. This lack of ability to truly intuit the business requirements and deep context requirements means it cannot make business related code changes. But it can help with very small context code changes, ie incidental complexity unrelated to the core role of a good developer, which is translating real world requirements into a system.
But I will add that it shouldn't be underestimated how many of us are actually solving the distribution problem, not technical problems. I still would not feel confident replacing a junior with AI, the core issue being lack of self-correction. But certainly people will try, and businesses built around AI development will be real and undercut established businesses. Whether that's net good or bad will probably not matter to those who lose their jobs in the scuffle.
A terrifyingly large percentage of people employed to write software cannot write software. Not even a little. These are the people that can be easily replaced.
In my prior line of work I wrote JavaScript for a living. There were people doing amazing, just jaw dropping astounding, things. Those people were almost exclusively hobbyists. At work most people struggled to do little more than copy/paste in a struggle just to put text on screen. Sadly, that is not an exaggeration.
Some people did what they considered to be advanced engineering against these colossal frameworks, but the result is just the same: little more than copy/paste and struggle to put text on screen. Yes, they might be solving for advanced complexity, but it is almost always completely unnecessary and frequently related to code vanity.
Virtually none of those people could write original applications, measure anything, write documentation, or do just about anything else practical.
> So how do we reconcile this?
Alienate your workforce by setting high standards, like a bar exam to become a lawyer. Fire those people that fail to rise to the occasion. Moving forward employ people who cannot meet the high standards only as juniors or apprentices, so that the next generation of developers have the opportunity to learn the craft without rewarding failure.
This would work if the world was willing to pay for software. So at the very least you'd have to outlaw the ad-based business model, or do what lawyers do: things that are absolutely critical for software development (think "program needs to be approved or it won't execute", that deep) that normal people aren't allowed ... and unable ... to do.
- Articles about tricky nature of tech debt aren't written by people who call the shots on whether the team can spend the entire next week on something that a customer can't see.
- Articles about systems architecture aren't written by people who decide how much each piece of work was significant for business.
- Books on methodologies are optional read for engineers, not essential for their job, and adoption happens only when they push it upwards.
Most of the buzz about AI replacing coding is coming from people who don't see a difference between generating a working MVP of an app, and evolving an app codebase for a decade and fixing ancient poor design choices in a spaceship which is already in flight.
I've even seen a manager who proposed allocating 33% time every day on 3 projects, and engineers had to push back. Such old and common knowledge that it doesn't work is apparently still not a job requirement in 2025. Despite that organizing and structuring project time allocation is management competency and not an engineering skill, it is de-facto entirely up to engineers to make sure it's done right. The same managers are now proud to demonstrate their "customer focus" by proposing to ask AI to resolve all the tech debt and write all the missing tests so that engineers could focus on business requests, and same engineers have to figure how to explain why it didn't just work when they tried.
To talk about complexity is to repeat the same old mistake. I am sure most engineers already know and I am yet to see an experienced engineer who believes their job will be taken by simple prompts. The problem we should be talking about should be titled something like,
"Software Engineering Has Poor Management Problem, and AI is Amplifying It"
I've seen it written that the main reason managers tend to want to get rid of SWEs is because they don't understand how to interface with them. Using an LLM solves that problem, because you don't need a nerd to operate it.
That’s because software is nebulous enough that you can get away with promising the moon to customers/boss, but in the next meeting, you’re given a reality check by the SWEs. And then you realize the mess you’re thrown everyone in.
Managers knows how to interface with SWEs well (people interface with professionals all the time). Most just hates going back to the engineers to get real answers when they fancy themselves as products owners.
SWEs are also about the most expensive kind of employee imaginable. I imagine that’s incentive enough.
In other words, they are not good managers. A good manager should be able to work with all kinds of people with different backgrounds and personalities.
Until you do. LLMs are great ar building prototypes but at some point if you don’t know what you’ll doing you’ll end up with an unmaintainable mess and you won’t have anyone to fix it.
I mean LLMs perhaps are capable of doing that too but they still need to be guided by people who are capable of understanding their output.
Being able to reduce the number of engineers that you need by e.g. 80% would still be a great deal though.
Oh god please kill me
Most problems businesses face have been seen by other businesses; perhaps some knowledge is in the training set or perhaps some problems are so easy to reason through that a LLM can do the "reasoning" more-or-less from first principles and your problem description.
I am speculating that AI will help with both sides of the No Silver Bullet dichotomy?
It sounds to me like a corporate equivalent of a drug-fueled rager. They want everything good now while deferring all the expenses to tomorrow.
Software engineering for any non-trivial problem means a baseline level of essential complexity that isn't going away, no matter the tool, not even if we someday "code" directly from our minds in some almost-free way via parallel programming thought diffusion. That's because (1) depth and breadth of choice; and (2) coordination/socials, mostly due but not uniquely related to (1) are the real bottlenecks.
Sure, accidental complexity can shrink, if you design in a way that's aligned with the tools, but even then, the gains are often overhyped. These kinds of "developer accelerators" (IDEs, low-code platforms, etc.) are always oversold in depth and scope, LLMs included.
The promise of the "10x engineer" is always there, but the reality is more mundane. For example, IDEs and LSPs are helpful, but not really transformative. Up to a point that people are being payed right now and don't use them at all, and they still deliver in a "economically justifiable" (by someone) way.
Today it's LLMs. Tomorrow it'll be LISP Machines v2.
Downside is there is much more non essential busy work because of which people had their jobs and now loads of those people will lose the job.
People who do work on real essential complexity of systems are far and between. People who say things like "proper professionals will always have work" are utter assholes thinking mostly that "they are those proper professionals".
In reality AI will be like F1 racing team not needing pit workers and have only drivers, how many drivers are there like 20 so it is 10 teams each having 2 drivers. Each team has 300-1000 people that do all the other things.
If you go to corporate level let's say 1 person working on essential complexity requires 10-20 people doing all kinds of non essential stuff that will be taken over by AI, or to be realistic instead of 10-20 people that person will need headcount of 5.
That is still 15 people out of job - are those people able to take over some essential complexity in a different company or different area of the same company, some would but it is also if they would like to do it. So those people will be pushed around or end up jobless, bitter whatever.
That is not great future coming in.
Once you remove all the roadblocks with syntax and language blindspots, the high cost of refactoring, the tedium of adding validation and tests, the challenges of integrating systems... suddenly, the work becomes more pure. Yes, you need to know how to do advanced structural things. But you don't need to learn very much about all the rest of it.
And we very quickly get to a point where someone who can break down problems into tidy Jira tickets is effectively programming. Programming was never really about learning languages but making computers do things, which is a transferrable skill and why so many engineers know so many languages.
Even the simplest tickets sometimes that end up requiring a one line change can require hours of investigation to fully understand/stamp the effects of that change.
And perhaps I havent used the greatest or latest, but in my experience LLMs break down hard st anything sufficiently large. They make changes and introduce new errors, they end up changing the feature, or worst case just ouright break everything.
Id never trust it unless you have an extensive amount of good tests for validation
After using Claude Code to vibe code some stuff, it seems to me that AI doesn't eliminate accidental complexity, it just creates more of it and takes away some of the pain of really bad APIs.
So it seems AI will just let us stretch further and make more accidentally complex systems.
They've started a business selling the exact opposite message to everyone who will buy it
I think there is a promise of that in AI and LLMs (but I remain skeptical, because I it needs a formal and not ad hoc definition). The idea you can build the systems using a fuzzy human language and the things will somehow work out.
This is very much up for debate and the weakest point of the argument I think. If developers are now 2-3x (remains to be seen..) as productive what will happen to the job market?
I suppose it depends on how much potential software is there that's not currently viable due to cost? How much would scope be increased on current products?
on this topic I suggest everybody who works in our industry to read Peter Naur's "Programming as Theory Building"[1] and a nice corollary from Baldur Bjarnson: "Theory-building and why employee churn is lethal to software companies"[2]
[1]: https://pages.cs.wisc.edu/~remzi/Naur.pdf [2]: https://www.baldurbjarnason.com/2022/theory-building/
Small businesses often understand domain less, not more, because they cannot invest as much as big businesses in building expertise. They may achieve something within that limited understanding, but the outcome will limit their growth. Of course, AI can help with discovery, but it may overcomplicate things. Product discovery is an art of figuring out what to do without doing too much or not enough, which AI has not mastered yet.
(and certainly Google's newest AI model is actually a step backwards on this front)
Add to that that nothing changes for difficult work. Writing a driver still requires hardware knowledge ... of the actual hardware. Which AI doesn't have ... and doesn't make any attempt to acquire.
I've seen articles that this part can actually be fixed. If you create a loop where the AI is forced to slowly build up low-level knowledge it can actually find this sort of things. But you need 10 expert AI researchers to do anything like this.
(frankly I don't see how this part could be fixed by better AI models)
What's in danger is the coding job consisting of "write me a form with these 20 fields". The 100% repetitive ones.
I don't think anyone thinks engineers are going away. But companies will hire less employees to do the same amount of work, and they won't pay engineers more.
That doesn't sell stock. Firing high paying employees sells a lot of stock.
I wonder if the independent studies that show Copilot increasing the rate of errors in software have anything to do with this less bold attitude. Most people selling AI are predicting the obsolescence of human authors.
Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual.
> Most people selling AI are predicting the obsolescence of human authors.
It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace.
This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step.
That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.
It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
It is not a reality since it has not happen. In the real world it has not happened.
There is no reason to believe that the current rate of progress will continue. Intelligence is not like the weaving machines. A software engineer is not a human calculator.
They didn’t just see layoffs. There were the constant wars with Napoleon and the War of 1812 causing significant economic instability along with highly variable capital investments in textile production at the time. They we’re looking at huge wealth disparity and losing their jobs for most meant losing everything.
What many Luddite supporters were asking for in many parts of England were: better working conditions, a raise to minimum wage, abolishment of child labour, etc. Sabotage was a means to make such demands from a class that held almost all of the power.
Many of those protestors were shot. Those who survived and were laid off were forced into workhouses.
The capitalists won and got to write the history and the myths. They made it about the technology and not the conditions. They told us that the displaced workers found new, better jobs elsewhere.
Programmers, while part of the labour class, have so far enjoyed a much better bargaining position and have been compensated in kind. Many of us also complain about the quality of output from AI as the textile workers complained about the poor quality of the lace. But fortunately the workhouses were shut down. Although poor quality code tends to result in people losing their life’s savings, having their identities stolen, etc. Higher stakes than cheap lace.
History is not repeating but it sure does rhyme.
See, this is the kind of conception of a programmer I find completely befuddling. Programming isn't like those jobs at all. There's a reason people who are overly attached to code and see their job as "writing code" are pejoratively called "code monkeys." Did CAD kill the engineer? No. It didn't. The idea is ridiculous.
At that point it's less "programmers will be out of work" as "most work may cease to exist".
Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.
The issue with natural language isn’t that it’s impossible to be precise, it’s that most people aren’t, or they are precise about what they want it to do for them, but not what the computer needs to do to make it happen. This leads to a lot of guessing by engineerings as they try to translate the business requirements into code. Now the LLM is doing that guessing, often with less context about the broader business objectives, or an understanding of the people writing those requirements.
No.
Some were concerned that the output of compilers couldn’t match the quality of what could be done by a competent programmer at the time. That was true for a time. Then compilers got better.
Nobody was concerned that compilers were going to be used by capitalists to lay them off and seize the means of producing programs by turning it into property.
Treating LLMs as a scaffolding tool yields better results at least for me personally. I just brain dump what I'm thinking of building and ask for it to give me models, and basic controllers using said models, then I just worry about the views and business logic.
But once you add repo context, domain knowledge etc... programming languages are far too verbose.
Here's a fresh example that I stumbled upon just a few hours ago. I needed to refactor some code that first computes the size of a popup, and then separately, the top left corner.
For brevity, one part used an "if", while the other one had a "switch":
I wanted the LLM to refactor it to store the position rather than applying it immediately. Turns out, it just could not handle different things (if vs. switch) doing a similar thing. I tried several variations of prompts, but it very strongly leaning to either have two ifs, or two switches, despite rather explicit instructions not to do so.It sort of makes sense: once the model has "completed" an if, and then encounters the need for a similar thing, it will pick an "if" again, because, well, it is completing the previous tokens.
Harmless here, but in many slightly less trivial examples, it would just steamroll over nuance and produce code that appears good, but fails in weird ways.
That said, splitting tasks into smaller parts devoid of such ambiguities works really well. Way easier to say "store size in m_StateStorage and apply on render" than manually editing 5 different points in the code. Especially with stuff like Cerebras, that can chew through complex code at several kilobytes per second, expanding simple thoughts faster than you could physically type them.
Dead Comment
AI stands for Artificial Intelligence. There are no inherent limits around what AI can and can't do or comprehend. What you are specifically critiquing is the capability of today's popular models, specifically transformer models, and accompanying tooling. This is a rapidly evolving landscape, and your assertions might no longer be relevant in a month, much less a year or five years. In fact, your criticism might not even be relevant between current models. It's one thing to speak about idiosyncrasies between models, but any broad conclusions drawn outside of a comprehensive multi-model review with strict procedure and controls is to be taken with a massive grain of salt, and one should be careful to avoid authoritative language about capabilities.
It would be useful to be precise in what you are critiquing, so that the critique actually has merit and applicability. Even saying "LLM" is a misnomer, as modern transformer models are multi-modal and trained on much more than just textual language.
It seems you don't recollect how much time passed without any big revolutions in AI. Deep learning was a big jump. But when the next jump comes? Might be tomorrow, but looking at history, might be in 2035.
According to what I see, the curve has already flattened and now only a new revolution could get us to the next big step.
Artificial, as in Artificial sand or artificial grass. Sure, it appears as sand or grass at first, but upon closer examination, it becomes very apparent that it's not real. Artificial is basically a similar word to magic - as in, it offers enough misdirection in order for people to think there might be intelligence, but upon closer examination, it's found lacking.
It's still impressive that it can do that, going all the way back to gaming AIs, but it's also a veil that is lifted easily.
Lots of us are interested in technology that's actually available, and we can all read date stamps on comments.
1. There was no error on line 91, it did some inconsequential formatting on that line 2. More importantly, it just ignored the very specific line I told it to go to. It's like I was playing telephone with the LLM which felt so strange with text-based communication.
This was me trying to get better at using the LLM while coding and seeing if I could "one-shot" some very simple things. Of course me doing this _very_ tiny fix myself would have been faster. Just felt weird and reinforces this idea that the LLM isn't actually thinking at all.
However, I can't just say "on page 123..." I've found it's better to either provide the quote, or describe the context, and then ask how it relates to [another concept]. Or I'll say "at the end of chapter 6, Bob does X, then why Y?" (perhaps this is similar to asking a coding LLM to fix a specific function instead of a specific line?).
My favorite examples of this have been sitting with living human authors and discussing their books — usually to jaw-dropped creators, particularly to Unknowns.
Works for non-fiction, too (of course). But for all those books you didn't read in HS English classes, you can somewhat recreate all that class discussion your teachers always attempted to foster — at your own discretion/direction.
And now you've learned that LLMs can't count lines. Next time, try asking it to "fix the error in function XYZ" or copy/paste the line in question, and see if you get better results.
> reinforces this idea that the LLM isn't actually thinking at all.
Of course it's not thinking, how could it? It's just a (rather big) equation.
So are you.
You need to give LLMs context. Line number isn't good context.
a line number is plenty of context - it's directly translatable into a range of bytes/characters in the file
Am I living in a different timeline than these guys ?
In my timeline, the tech job market is terrible, and AI is the worst for junior developers, because they aren't learning what they should by doing.
Aka, 80% of the work being done by 20% of the people. The problem is you still have to hire the other 80% of the developers to also get those 20%
It also requires a fair bit of wisdom to know where the software is expected to grow, and how to architect for that eventuality.
I can't picture an LLM doing a fraction of that work.
What's happened since the 1970s is, programming has undergone prestige inflation to the point where programmers are looked to for decisions regarding the actual business procedures, and usually they just don't have the business knowledge or wherewithal to do that. The systems analyst has faded into memory. And as Milt Bryce said, "If this country built bridges the way it builds systems, we'd be a nation run by ferry-boats."
(Wow I sound triggered! sigh)
Wait does it? This is a new idea to me, in what way could "manual" have a negative connotation?
Are people really that against doing things manually nowadays?
Deleted Comment
this is okay! it's a sign of your humanity :)
Deleted Comment
A man is a human.
“Manual” comes from Latin manus, meaning “hand”: https://en.wiktionary.org/wiki/manus. It literally means “by hand”.