I have gone from using Claude Code all day long since the day it was launched to only using the separate Claude app. In my mind that is a nice balance of using it, but not too much, not too fast.
there is the temptation to just let these things run in our codebases, which I think for some projects is totally fine. For most websites I think this would usually be fine. This is for two reasons: 1) these models have been trained on more websites than probably anything else and 2) if a div/text is off by a little bit then usually there will be no huge problems.
But if you're building something that is mission critical, unless you go super slowly, which again is hard to do because these agents are tempting to go super fast. That is sort of the allure of them: to be able to write sofware super fast.
But as we all know, in some programs you cannot have a single char wrong or the whole program may not work or have value. At least that is how the one I am working on is.
I found that I lost the mental map of the codebase I am working on. Claude Code had done too much too fast.
I found a function this morning to validate futures/stocks/FUT-OPT/STK-OPT symbols where the validation was super basic and terrible that it had written. We had implemented some very strong actual symbol data validation a week or two ago. But that wasn't fully implemented everywhere. So now I need to go back and do this.
Anyways, I think finding where certain code is written would be helpful for sure and suggesting various ways to solve problems. But the separate GUI apps can do that for us.
So for now I am going to keep just using the separate LLM apps. I will also save lots of money in the meantime (which I would gladly spend for a higher quality Claude Code ish setup).
The reality is that you can't have AI do too much for you or else you completely lose track of what is happening. I find it useful to let it do small stupid things and use it for brainstorming.
I don't like it to do complete PR's that span multiple files.
I don't think the "complete PR spanning multiple files" is an issue actually.
I think the issue is if you don't yourself understand what it's doing. If all you do is to tell it what the outcome should be from a user's perspective, you check that that's what it does and then you just merge. Then you have a problem.
But if you just use it to be faster at getting the code you would've liked to write yourself, or make it write the code you'd have written if you had bothered to do that boring thing you know needs to be done but never bothered to do, then it's actually a great tool.
I think in that case it's like IDE based refactorings enabled by well typed languages. Way back in the day, there were refactorings that were a royal pain in the butt to do in our Perl code base. I did a lot of them but they weren't fun. Very simple renames or function extractions that help code readability just aren't done if you have to do them manually. If you can tell an IDE to do a rename and you're guaranteed that nothing breaks, it's simply a no brainer. Anyone not doing it is simply a bad developer if you ask me.
There's a lot of copy and paste coding going on in "business software". And that's fine. I engage in that too, all the time. You have a blueprint of how to do something in your code base. You just need to do something similar "over there". So you know where to find the thing to copy and paste and then adjust. The AI can do it for you even faster especially if you already know what to tell it to copy. And in some cases all you need to know is that there's something to copy and not from where exactly and it'll be able to copy it very nicely for you.
And the resulting PR that does span multiple files is totally fine. You just came up with it faster than you ever could've. Personally I skipped all the "Copilot being a better autocomplete" days and went straight into agentic workflows - with Claude Code to be specific. Using it from within IntelliJ in a monorepo that I know a lot about already. It's really awesome actually.
The funny thing is that at least in my experience, the people that are slower than you doing any of this manually are not gonna be good at this with AI either. You're still gonna be better and faster at using this new tool than they were at using the previously available tools.
Losing the mental map is the number one issue for me. I wonder if there could be a way to keep track of it, even at a high level. Keeping the ability to dig in is crucial.
Spend time reviewing outputs like a tech lead does when managing multiple developers. That's the upgrade you hust got in your career, you are now bound to how many "team members" you can manage at a single time. I'm grateful to live in such a time.
The code is the mental map. Orchestra conductors read and follow the music sheet as well. They don't let random people comes in and mess with. Neither do film directors with their scripts and their plans.
> I have gone from using Claude Code all day long since the day it was launched to only using the separate Claude app. In my mind that is a nice balance of using it, but not too much, not too fast.
I’m on a similar journey - I never used it all day long but definitely a lot during a brief honeymoon period and now I’m back to using it very sparingly but I put questions to the Claude app all the time
For me the sweet spot for Claude code is when I have a very clear and well documented thing to set up that I really don’t want to do for the umpteenth time - like webhook signature verification - just paste the docs and let it rip - or setting up the most basic crud forms for an admin dashboard - ezpz
But otherwise I’ve gone back to mostly writing everything by hand
I think the whole AI vs non. AI debate is a bit besides the point. Engineers are stuck in the old paradigm of "perfect" algorithms.
I think the image you post at the beginning basically sums it up for me: ChatGPT o3/5 Thinking can one-shot 75% of most reasonably sized tasks I give it without breaking a sweat, but struggles with tweaks to get it to 100%. So I make those tweaks myself and I have cut my code writing task in half or one third of the time.
ChatGPT also knows more idioms and useful libraries than I do so I generally end up with cleaner code this way.
Ferrari's are still hand assembled but Ford's assembly line and machines help save up human labor even if the quality of a mass-produced item is less than a hand-crafted one. But if everything was hand-crafted, we would have no computers at all to program.
Programming and writing will become niche and humans will still be used where a quality higher than what AI can produce is needed. But most code will be done by minotaur human-ai teams, where the human has a minimal but necessary contribution to keep the AI on track... I mean, it already is.
Hard disagree. We'll be able to use more expressive languages with better LLM support for understanding how to express ourselves and to understand compiler results. LLMs are only good at stuff that better languages don't require you to do. After that they fall off the cliff quickly.
LLMs are a communication technology, with a huge trained context of conversation. They have a long way to go before becoming anything intelligent.
LLMs lack intentionality, and they lack the ability to hold a series of precepts "in mind" and stick to those precepts. That is, if I say "I want code that satisfies properties A, B, C, D..." at some point the LLM just can't keep track of all the properties, which ones are satisfied, which ones aren't, what needs to be done or can be done to make them all satisfied.
But LLMs aren't "only good at stuff that better languages don't require you to do." In fact they are very good at taking a bad function definition and turning it into an idiomatic one that does what I wanted to do. That's very intelligent, there is no language that can take a bad spec and make it specific and fit for the specified task. LLMs can. (not perfectly mind you, but faster and often better than I can.) The problem is they just can't always figure out when what they've written is off-spec. But "always" isn't "never" and I've yet to meet an intelligence that is perfect.
Depends on what you do, and what systems you develop for I would reckon. If it's another TODO app or some kind of table + form system that's been done to death - AI can probably have a go at creating a barebones minimal viable product. Targeting code that's outside the sweet spot of the training data ("blurry" area), you'll start to stumble. I've also found agents to be useless in large code bases with distributed logic where parts are in react, web back-end, service system). Slow and unreliable for large systems. Good for small tasks and scaffolding up proof of concepts.
Are you still going to have the skills to review it a year from now? Or 5 years from now when you’ve become accustomed to only writing <20% of the code? I’m already witnessing my coworkers skills degrading because of this, and it’s only going to get worse. Programming is a language, and when you don’t use it, it fades.
> AI can do 80% of the work. I can review it later. And I spend much less time reviewing than I would have typing up everything manually
Are you sure you're actually reviewing the code? Deeply, properly reviewing and understanding it? Because from what I've seen people that say they do, don't. That's why they 'speed up' from using LLM-generated code.
The old adage that it's far harder to review code than to write it still holds true.
This reminds me of the phenomenon of competence being inversely correlated with confidence until some inflection point is reached on the journey to mastery.
I think the argument being put forward here is that writing that boring code is part of the journey to mastery. If you haven't crossed the inflection point, a backwards slide in skills will result in less competence and more confidence, which is a scary thought given how software runs the world.
What you shouldn't forget also is that, while AI may be good at coming up with a "first shot" solution, it may be much worse if you want to change/correct parts of it.
In my experience, AI very often gets into a sort of sunk-cost fallacy (sunk prompt?) and then it is very hard to get it to make significant changes, especially architecturally.
I recently wrote an extension for a popular software product and gave AI the same task. It created a perfectly working version however it was 5x the lines of code of my version because it didn't know the extension API as well, even though I gave it the full documentation. It also hard coded some stuff/solutions to challenges that we totally don't want to be hard coded.
A big reason why I arrived at a much better solution was that I used a debugger to step through the code and noted down just the API interactions I needed.
The AI also was convinced that some things were entirely impossible. By stepping through the code I saw that they would be possible by using parts of the internal API. I suggested a change to make the public API better for my use case in a GitHub issue and now it is totally not impossible.
At the end of the day I have to conclude that, the amount of time invested guiding and massaging the AI was too much and not really worth it. I would've been better off debugging the code right away and then creating my own version. The potential for AI to do the 80% is there. At this time though I personally can't accept its results yet but that may also be due to my personal flavour of perfectionism.
The fear and pain of writing boring code gave us the processes and systems we have today. i.e. microservices, (I know they have their own problems) were born out of pain from maintaining monoliths. Agile (again, own problems) was born out of pain from waterfall.
What happens when all pain and boredom is taken away? Why do we need to invent new frameworks, systems, design patterns in that scenario?
However, from my experience, the quality of code produced by developers in project we are working for the last 3+ years gone south. Amount of bugs now literally tripled year on year. I bet the reason is extensive use of AI tools, as the developers are the same.
Its because most people don't have the discipline and obsession of attention to detail to know when one should use an LLM and when one shouldn't.
I highly doubt the best of the best folks are even touching LLMs (barely) because they can see the flaws and the tradeoffs that are not-so-visible to others until they are 50 levels deep, with sunk investments, unwilling to go back to how they used to do things.
Another hard disagree. The crux here is that if u are not an expert in the given domain you do not know where that missing 25% is wrong. You think you do but you dont.
I have seen people bring in thousands of lines of opencv lut code in ai slop form because they didnt understand how to interpolate between two colors and didnt have the experience to know that is what they needed to do. This is the catch 20/20 of the ai expert narrative.
The other part is that improvement has massively stagnated in the space. It is painfully obvious too.
<< you do not know where that missing 25% is wrong
I think there is something to this line of thinking. I just finished a bigger project and without going into details, one person from team supposedly dedicated to providing viable data about data was producing odd results. Since the data was not making much sense, I asked for info on how the data was produced. I was given SQL script and 'and then we applied some regex' explanation.
Long story short, I dig in and find that applied regex apparently messed with dates in an unexpected way and I knew because I knew the 'shape' that data was expected to have. I corrected it, because we were right around the deadline, but.. I noted it.
Anyway, I still see llm as a tool, but I think there is some reckoning on the horizon as:
1. managers push for more use and speed given that new tool
2. getting there faster wronger, because people go with 1 and do not check the output ( or don't know how to check it or don't know when its wrong )
It won't end well, because the culture does not reward careful consideration.
Engineering isn't stuck on perfect algorithms. Management is. There's lip service for AI code gen, but if my name is on a module, I still have to vouch for its correctness. If it's wrong, that might become my problem. I don't always write perfect code, but I aspire to. If I see evidence that these tools are writing more correct and reliable code than I do, then I will start to take it more seriously. For some code, it matters whether it's robust basically.
> Engineers are stuck in the old paradigm of "perfect" algorithms.
Reminds me of a misinterpretation of Knuth.
> Premature optimization is the root of all evil.
He was definitely knocking engineers for wanting to write "perfect" algorithms, but this quote also got bastardized to mean something different from what he said (happens to many clichés). All he said was "grab a fucking profiler before you optimize."
But now, I'm not sure a lot of programmers even know what a profiler is. When was the last time you saw someone profile their code?
Problem is we've taken the idea of "minimum viable product" too far. People are saying "Doesn't have to be perfect, just has to work." I think most people agree. But with the current state of things? I disagree that things even work. We're so far away from the question of optimization. It's bad enough that there are apps that require several gigs to just edit a 30kb document but FFS we're living in a world where Windows Hello crashes Microsoft Outlook. It's not the programs are ugly babies that could be better, they are monstrosities begging to be put to death.
I WISH we could talk about optimization. I WISH our problem was perfectionism. But right now our problem is that everything is a steaming pile of garbage and most people are just shrugging their arms like "it is the way it is". Just because you don't clean up that steaming pile of garbage doesn't mean someone else doesn't. So stop passing the buck.
> When was the last time you saw someone profile their code?
A year ago. I heavily relied on one to optimize a complex data import that took an hour for a million line Excel file. The algorithm translated it to a graph according to a user-specified definition and would update an existing graph in neo4j, keeping the whole thing consistent.
The only other guy who understood the algorithm (a math PhD) thought it was as optimal as it could get. I used the profiler to find all the bottlenecks, which were all DB checks for the existence of nodes, and implemented custom indices to reduce import time from an hour to 3 minutes.
It did introduce a bunch of bugs that I had to fix, but I also discovered some bugs in the original algorithm.
It was one of my best programming experiences ever. Especially the payoff at the end when it went down from an hour to 3 minutes is a dopamine rush like never before. Now I want to optimize more code.
I don't think users cared, though; originally this work would take days by hand, so an hour was already pretty good. Now I made something fiendishly complex look trivial.
I invite your attention to the StatsD telemetry protocol, where:
1. Every single measurement in a timeseries is encoded as a utf-8 string having (roughly) the following format:
"${name}:${value}|${type}|${tags}"
where name is like "my.long.namespace.and.metric.name", value is a string formatted number, god only knows what type is, and tags is some gigantic comma separated key:value monstrosity.
2. Each and every one of these things is fired off into the ether in the form of a UDP datagram.
3. Whenever the server receives these presumably it gets around sometime to assigning them timestamps and inserts them into a database, not necessarily in that or any other particular order.
Ah, I'm bookmarking this. Thanks for writing this :)
I love how you put it: "grab a fucking profiler before you optimize". I get complaints sometimes about using FP because of performance, and I think a variant of "grab a fucking profiler before you optimize" is much better response than "avoid premature optimization". Introducing them to a magical thing called as "profiler" is a nice bonus too.
> Problem is we've taken the idea of "minimum viable product" too far. People are saying "Doesn't have to be perfect, just has to work." I think most people agree. But with the current state of things? I disagree that things even work. We're so far away from the question of optimization. It's bad enough that there are apps that require several gigs to just edit a 30kb document but FFS we're living in a world where Windows Hello crashes Microsoft Outlook. It's not the programs are ugly babies that could be better, they are monstrosities begging to be put to death.
LOL. OMG that was beautiful. It almost feels like we are de-evolving software to a state where shit is going to stop working bad. I know this is not full of facts, but this take reminds me of Jonathan Blow's video "Preventing the Collapse of Civilization"[0] Where he talks about how code runs worse than it ever has and I think he was arguing that civilization is collapsing before our eyes in slow time.
Yeah, I'm trying to branch out to build things outside my comfort zone. AI already is helpful in my normal day-to-day (backend engineer, mostly Python w/ PyCharm/Datagrip working in monorepo style), but Claude Code helped me graft on a Typescript/Javascript UI ecosystem with an independent Webstorm IDE setup that will integrate well, along with all the presumably latest build tools and package management. The research and self-education I would have needed to do this without Claude would have been extensive, and would have been nowhere as complete. I don't see any point in going back to pre-AI. And I don't generally use AI results blindly, I go back and review and set some breakpoints in an IDE and watch it do its thing.
The analogy seems to fall apart because the quality of an assembly line produced car is higher than the hand crafted one. Fords lose out because they engineer to a price point, a Ferrari doesn't have that "problem" - arguably, the more expensive the better.
> Programming and writing will become niche and humans will still be used where a quality higher than what AI can produce is needed. But most code will be done by minotaur human-ai teams, where the human has a minimal but necessary contribution to keep the AI on track... I mean, it already is.
Or alternatively, we will build bigger and better things with the power of AI. Everyone talks about it replacing us, but we aren't running out of things to build. It's not like we're gonna run out of ways to improve the world, and compared to other things the digital world is unconstrained.
Why are you assuming the cases where humans can code better than AI still exists after three years say -- I think in some industries today artisanal products are also not higher quality than machine made ones.
Because we're not even close to that happening. However I've observed a sort of cognitive decline in a few formerly experienced and extremely knowledgeable developers whom I used to respect. They happen to be the most vocal supporters of LLMs now and sometimes our discussions get so ridiculous I have to practically beg them to put the LLM down and just think about what they're saying for just 5 seconds. I think they're about a year away from believing that the sky is green if the AI says so.
So maybe LLMs will win on a technicality by making us more stupid as a species.
Extraordinary claims require extraordinary evidence and if there's one thing we've learned is that progress in AI is not linear nor predictable. We've been a few years away from fully self-driving cars for a really long time now.
Yeah, it's definitely shifting how we work. Feels like an upgrade from high-level languages, but we're still guiding the process. The performance boost is real, but once everyone adopts it, we'll probably just be building more complex things at a similar pace.
IMO LLMs have demonstrated how uncomplicated the man behind the curtain really is. If we do happen to achieve AGI it will likely have many of the problems associated with the real thing which often fails in spectacular fashion.
You are describing accurately where AI is now. But it wasn't there two years ago and it won't be there in two years. You still have 25% of your job today, but this will most likely gonna halve evey x-number of months.
I mean I'm opposed to all work, but the transition to jobless society might be traumatic. I hope you didn't neglect to buy crypto because that's the only remaining way to spontaneously and easily tap into the wealth of the rich when the work is gone.
> yes, let us enable EVERYONE to build little tools for themselves.
It will enable more people, but "everyone" is never, ever going to happen.
This is both because (1) many people don't want to do this, no matter how easy it is -- probably the primary reason and (2) many people won't have the ability to do this in a way that is net profitable for them, because their "tool ideas" just won't be good.
This sounds great in theory but my my experience is that non tech people make horrible SE's. Even if they don't do the coding only participating in the spec. They simply don't even know what they don't know. Which is why SE exists and why these type of projects always fail to gain traction.
In my life of the thousands of non tech people I've worked with I can count in the low double digits that were capable of working in SE without exp/edu in it. Even then they were all up and coming driven people that still often missed what to me were obvious things, because its not their area of expertise. (They were all brilliant in their own area)
The argument against relying on AI for everything is that the humans who curate and architect systems learned what they did through their experience at lower levels.
Overutilization of AI is pulling the ladder up and preventing the next generation of software architects and engineers from learning through experience.
It would be good if there were a mode where AI trained the human operator as it worked, to reduce future reliance. Instead of just writing a document or editing a document, it would explain in a good amount of detail what it was doing, and tailor the information to the understanding level of the operator. It might even quiz the operator to assure understanding.
This would take more time in the short run, but in the long run it would result in more well-rounded humans.
When there are power/internet/LLM outages, some people are going to be rendered completely helpless, and others will be more modestly impacted — but will still be able to get some work done.
We should aim to have more people in the latter camp.
> some people are going to be rendered completely helpless,
This I don't have any hope about. Tech companies have been trying to make their customers ignorant (and terrified) of the functioning of their own computers as a moat to prevent them from touching anything, and to convince them to allow every violation or imposition. They've convinced many that their phones aren't even computers, and must operate by different, more intrusive and corporate-friendly guidelines. Now, they're being taught that their computers are actually phones, that mere customers aren't responsible enough to control those as well, and that the people who should have control are the noble tech CEOs and the government. The companies can not be shamed out of doing this, they genuinely think people are just crops to be harvested. You being dumber means that you have to pay them as a necessity rather than a convenience.
In 1985, they would teach children who could barely tie their shoes what files and directories were on computers, and we'd get to program in LOGO. These days? The panicked look I often see on normal people's faces when I ask them where they saved the file that their life depends on and is missing or broken makes me very sad. "Better to trick them into saving to OneDrive," the witches cackle. "Then if they miss a monthly payment, we can take their files away!"
Can you describe how to access this feature? I can't find anything about this online, and I asked Claude on command line and nothing comes up for "learning mode"
Ah, cool. Is this meant to be used for learning specifically, or just something that can be toggled whenever you're using Claude to help you with anything?
It used to be reading and writing were skills. People would strive to get better, usually by steeping themselves in the work of better writers so some rubs off.
Now, the llm summarizes the email, so you only have to be so literate to understand bullet points. The llm takes your bullet points and turns them into long form writing, because you can’t write. They say this is necessary because they aren’t good writers, but this creates a self fulfilling prophecy if they just avoid writing entirely.
A sad time when people are allowing their ability to read and write fall by the wayside. These used to be skills people valued and took pride in improving for themselves. Now people shut their brain off.
Tools like Claude Code and OpenAI’s Codex CLI have boosted my productivity massively. They already handle about 90% of the coding work, and I just step in to finish the last 10%. Every month they get better, maybe in a year it’s 95%, in two years 97%, in three years 98%. We can all see where this is going.
You're an early adopter, so you're seeing massive gains. But eventually everyone gets the same productivity boost and that becomes the new baseline expectation.
Any clever prompting techniques that give you an edge today will evaporate quickly. People figure out the tricks, models absorb them, and tools automate them away.
There's no substitute for actually coding to learn software development. For new engineers, I'd strongly recommend limiting AI code generation on real work. Use it to explain concepts, not do the work for you. Otherwise you'll never develop the judgment to know what that 10% actually is.
You are fortunate to have the pre-AI background experience to know how to recognize the 10%. People graduating college right now may never be so fortunate.
This doesn't just apply to software development - it applies across the board in any form of knowledge.
An example would be economics - an LLM can spit out a bunch of words about an economic model. But if you don't take the time to learn, visualise and understand it for yourself - it means nothing to you. And in that case, if you already possess mastery why would you waste your resources playing around with an inferior tool to you?
Absolutely agreed. Thinking anything else is nothing but cope, and these comments are FULL of it. it would be laughable if they weren't so gate keepy and disengenuous about it.
Readit News
there is the temptation to just let these things run in our codebases, which I think for some projects is totally fine. For most websites I think this would usually be fine. This is for two reasons: 1) these models have been trained on more websites than probably anything else and 2) if a div/text is off by a little bit then usually there will be no huge problems.
But if you're building something that is mission critical, unless you go super slowly, which again is hard to do because these agents are tempting to go super fast. That is sort of the allure of them: to be able to write sofware super fast.
But as we all know, in some programs you cannot have a single char wrong or the whole program may not work or have value. At least that is how the one I am working on is.
I found that I lost the mental map of the codebase I am working on. Claude Code had done too much too fast.
I found a function this morning to validate futures/stocks/FUT-OPT/STK-OPT symbols where the validation was super basic and terrible that it had written. We had implemented some very strong actual symbol data validation a week or two ago. But that wasn't fully implemented everywhere. So now I need to go back and do this.
Anyways, I think finding where certain code is written would be helpful for sure and suggesting various ways to solve problems. But the separate GUI apps can do that for us.
So for now I am going to keep just using the separate LLM apps. I will also save lots of money in the meantime (which I would gladly spend for a higher quality Claude Code ish setup).
I don't like it to do complete PR's that span multiple files.
I think the issue is if you don't yourself understand what it's doing. If all you do is to tell it what the outcome should be from a user's perspective, you check that that's what it does and then you just merge. Then you have a problem.
But if you just use it to be faster at getting the code you would've liked to write yourself, or make it write the code you'd have written if you had bothered to do that boring thing you know needs to be done but never bothered to do, then it's actually a great tool.
I think in that case it's like IDE based refactorings enabled by well typed languages. Way back in the day, there were refactorings that were a royal pain in the butt to do in our Perl code base. I did a lot of them but they weren't fun. Very simple renames or function extractions that help code readability just aren't done if you have to do them manually. If you can tell an IDE to do a rename and you're guaranteed that nothing breaks, it's simply a no brainer. Anyone not doing it is simply a bad developer if you ask me.
There's a lot of copy and paste coding going on in "business software". And that's fine. I engage in that too, all the time. You have a blueprint of how to do something in your code base. You just need to do something similar "over there". So you know where to find the thing to copy and paste and then adjust. The AI can do it for you even faster especially if you already know what to tell it to copy. And in some cases all you need to know is that there's something to copy and not from where exactly and it'll be able to copy it very nicely for you.
And the resulting PR that does span multiple files is totally fine. You just came up with it faster than you ever could've. Personally I skipped all the "Copilot being a better autocomplete" days and went straight into agentic workflows - with Claude Code to be specific. Using it from within IntelliJ in a monorepo that I know a lot about already. It's really awesome actually.
The funny thing is that at least in my experience, the people that are slower than you doing any of this manually are not gonna be good at this with AI either. You're still gonna be better and faster at using this new tool than they were at using the previously available tools.
I’m on a similar journey - I never used it all day long but definitely a lot during a brief honeymoon period and now I’m back to using it very sparingly but I put questions to the Claude app all the time
For me the sweet spot for Claude code is when I have a very clear and well documented thing to set up that I really don’t want to do for the umpteenth time - like webhook signature verification - just paste the docs and let it rip - or setting up the most basic crud forms for an admin dashboard - ezpz
But otherwise I’ve gone back to mostly writing everything by hand
I think the image you post at the beginning basically sums it up for me: ChatGPT o3/5 Thinking can one-shot 75% of most reasonably sized tasks I give it without breaking a sweat, but struggles with tweaks to get it to 100%. So I make those tweaks myself and I have cut my code writing task in half or one third of the time.
ChatGPT also knows more idioms and useful libraries than I do so I generally end up with cleaner code this way.
Ferrari's are still hand assembled but Ford's assembly line and machines help save up human labor even if the quality of a mass-produced item is less than a hand-crafted one. But if everything was hand-crafted, we would have no computers at all to program.
Programming and writing will become niche and humans will still be used where a quality higher than what AI can produce is needed. But most code will be done by minotaur human-ai teams, where the human has a minimal but necessary contribution to keep the AI on track... I mean, it already is.
LLMs are a communication technology, with a huge trained context of conversation. They have a long way to go before becoming anything intelligent.
But LLMs aren't "only good at stuff that better languages don't require you to do." In fact they are very good at taking a bad function definition and turning it into an idiomatic one that does what I wanted to do. That's very intelligent, there is no language that can take a bad spec and make it specific and fit for the specified task. LLMs can. (not perfectly mind you, but faster and often better than I can.) The problem is they just can't always figure out when what they've written is off-spec. But "always" isn't "never" and I've yet to meet an intelligence that is perfect.
AI can do 80% of the work. I can review it later. And I spend much less time reviewing than I would have typing up everything manually.
I recently used it to add some logging and exception handling. It had to be done in multiple places.
A simple 2 line prompt one shotted it. Why do I need to waste time writing boring code?
Some people actually enjoy that, believe it or not.
Are you sure you're actually reviewing the code? Deeply, properly reviewing and understanding it? Because from what I've seen people that say they do, don't. That's why they 'speed up' from using LLM-generated code.
The old adage that it's far harder to review code than to write it still holds true.
The better question is: should that boring code be written? Code should only be non-boring.
The boredom of writing the code is not the only problem. The subsequent continued indefinite existence of that code is also a problem.
I think the argument being put forward here is that writing that boring code is part of the journey to mastery. If you haven't crossed the inflection point, a backwards slide in skills will result in less competence and more confidence, which is a scary thought given how software runs the world.
In my experience, AI very often gets into a sort of sunk-cost fallacy (sunk prompt?) and then it is very hard to get it to make significant changes, especially architecturally.
I recently wrote an extension for a popular software product and gave AI the same task. It created a perfectly working version however it was 5x the lines of code of my version because it didn't know the extension API as well, even though I gave it the full documentation. It also hard coded some stuff/solutions to challenges that we totally don't want to be hard coded. A big reason why I arrived at a much better solution was that I used a debugger to step through the code and noted down just the API interactions I needed.
The AI also was convinced that some things were entirely impossible. By stepping through the code I saw that they would be possible by using parts of the internal API. I suggested a change to make the public API better for my use case in a GitHub issue and now it is totally not impossible.
At the end of the day I have to conclude that, the amount of time invested guiding and massaging the AI was too much and not really worth it. I would've been better off debugging the code right away and then creating my own version. The potential for AI to do the 80% is there. At this time though I personally can't accept its results yet but that may also be due to my personal flavour of perfectionism.
Deleted Comment
What happens when all pain and boredom is taken away? Why do we need to invent new frameworks, systems, design patterns in that scenario?
I highly doubt the best of the best folks are even touching LLMs (barely) because they can see the flaws and the tradeoffs that are not-so-visible to others until they are 50 levels deep, with sunk investments, unwilling to go back to how they used to do things.
I have seen people bring in thousands of lines of opencv lut code in ai slop form because they didnt understand how to interpolate between two colors and didnt have the experience to know that is what they needed to do. This is the catch 20/20 of the ai expert narrative.
The other part is that improvement has massively stagnated in the space. It is painfully obvious too.
I think there is something to this line of thinking. I just finished a bigger project and without going into details, one person from team supposedly dedicated to providing viable data about data was producing odd results. Since the data was not making much sense, I asked for info on how the data was produced. I was given SQL script and 'and then we applied some regex' explanation.
Long story short, I dig in and find that applied regex apparently messed with dates in an unexpected way and I knew because I knew the 'shape' that data was expected to have. I corrected it, because we were right around the deadline, but.. I noted it.
Anyway, I still see llm as a tool, but I think there is some reckoning on the horizon as:
1. managers push for more use and speed given that new tool 2. getting there faster wronger, because people go with 1 and do not check the output ( or don't know how to check it or don't know when its wrong )
It won't end well, because the culture does not reward careful consideration.
But now, I'm not sure a lot of programmers even know what a profiler is. When was the last time you saw someone profile their code?
Problem is we've taken the idea of "minimum viable product" too far. People are saying "Doesn't have to be perfect, just has to work." I think most people agree. But with the current state of things? I disagree that things even work. We're so far away from the question of optimization. It's bad enough that there are apps that require several gigs to just edit a 30kb document but FFS we're living in a world where Windows Hello crashes Microsoft Outlook. It's not the programs are ugly babies that could be better, they are monstrosities begging to be put to death.
I WISH we could talk about optimization. I WISH our problem was perfectionism. But right now our problem is that everything is a steaming pile of garbage and most people are just shrugging their arms like "it is the way it is". Just because you don't clean up that steaming pile of garbage doesn't mean someone else doesn't. So stop passing the buck.
A year ago. I heavily relied on one to optimize a complex data import that took an hour for a million line Excel file. The algorithm translated it to a graph according to a user-specified definition and would update an existing graph in neo4j, keeping the whole thing consistent.
The only other guy who understood the algorithm (a math PhD) thought it was as optimal as it could get. I used the profiler to find all the bottlenecks, which were all DB checks for the existence of nodes, and implemented custom indices to reduce import time from an hour to 3 minutes.
It did introduce a bunch of bugs that I had to fix, but I also discovered some bugs in the original algorithm.
It was one of my best programming experiences ever. Especially the payoff at the end when it went down from an hour to 3 minutes is a dopamine rush like never before. Now I want to optimize more code.
I don't think users cared, though; originally this work would take days by hand, so an hour was already pretty good. Now I made something fiendishly complex look trivial.
1. Every single measurement in a timeseries is encoded as a utf-8 string having (roughly) the following format:
where name is like "my.long.namespace.and.metric.name", value is a string formatted number, god only knows what type is, and tags is some gigantic comma separated key:value monstrosity.2. Each and every one of these things is fired off into the ether in the form of a UDP datagram.
3. Whenever the server receives these presumably it gets around sometime to assigning them timestamps and inserts them into a database, not necessarily in that or any other particular order.
"it is the way it is[1]."
[1] https://github.com/statsd/statsd?tab=readme-ov-file#usage
I love how you put it: "grab a fucking profiler before you optimize". I get complaints sometimes about using FP because of performance, and I think a variant of "grab a fucking profiler before you optimize" is much better response than "avoid premature optimization". Introducing them to a magical thing called as "profiler" is a nice bonus too.
LOL. OMG that was beautiful. It almost feels like we are de-evolving software to a state where shit is going to stop working bad. I know this is not full of facts, but this take reminds me of Jonathan Blow's video "Preventing the Collapse of Civilization"[0] Where he talks about how code runs worse than it ever has and I think he was arguing that civilization is collapsing before our eyes in slow time.
[0]: https://youtu.be/pW-SOdj4Kkk?si=LToItJb1Cv-GgB4q&t=1089
What? So Your Jeep Compass is higher quality than a 458?
Or alternatively, we will build bigger and better things with the power of AI. Everyone talks about it replacing us, but we aren't running out of things to build. It's not like we're gonna run out of ways to improve the world, and compared to other things the digital world is unconstrained.
So maybe LLMs will win on a technicality by making us more stupid as a species.
Deleted Comment
... and can now measure my productivity in dependencies added per day! :)
I mean I'm opposed to all work, but the transition to jobless society might be traumatic. I hope you didn't neglect to buy crypto because that's the only remaining way to spontaneously and easily tap into the wealth of the rich when the work is gone.
It's like asking -- "will robots be good for building things?"
Sure, some things. What things?
Personally, I'm hoping for the revival of the idea that Hypercard was intended for; yes, let us enable EVERYONE to build little tools for themselves.
It will enable more people, but "everyone" is never, ever going to happen.
This is both because (1) many people don't want to do this, no matter how easy it is -- probably the primary reason and (2) many people won't have the ability to do this in a way that is net profitable for them, because their "tool ideas" just won't be good.
50 years ago, you don't have to be a car guy, but if you know one, that's all you need to save a LOT of money and headache.
Today, that kind of works -- unless you own e.g. a Tesla.
Lol lets get real. Most people want to live life and switch off their brain.
Absolutely nothing wrong with that - you can't fight what the body wants instinctively.
In my life of the thousands of non tech people I've worked with I can count in the low double digits that were capable of working in SE without exp/edu in it. Even then they were all up and coming driven people that still often missed what to me were obvious things, because its not their area of expertise. (They were all brilliant in their own area)
Overutilization of AI is pulling the ladder up and preventing the next generation of software architects and engineers from learning through experience.
This would take more time in the short run, but in the long run it would result in more well-rounded humans.
When there are power/internet/LLM outages, some people are going to be rendered completely helpless, and others will be more modestly impacted — but will still be able to get some work done.
We should aim to have more people in the latter camp.
This I don't have any hope about. Tech companies have been trying to make their customers ignorant (and terrified) of the functioning of their own computers as a moat to prevent them from touching anything, and to convince them to allow every violation or imposition. They've convinced many that their phones aren't even computers, and must operate by different, more intrusive and corporate-friendly guidelines. Now, they're being taught that their computers are actually phones, that mere customers aren't responsible enough to control those as well, and that the people who should have control are the noble tech CEOs and the government. The companies can not be shamed out of doing this, they genuinely think people are just crops to be harvested. You being dumber means that you have to pay them as a necessity rather than a convenience.
In 1985, they would teach children who could barely tie their shoes what files and directories were on computers, and we'd get to program in LOGO. These days? The panicked look I often see on normal people's faces when I ask them where they saved the file that their life depends on and is missing or broken makes me very sad. "Better to trick them into saving to OneDrive," the witches cackle. "Then if they miss a monthly payment, we can take their files away!"
Seems very cool.
Now, the llm summarizes the email, so you only have to be so literate to understand bullet points. The llm takes your bullet points and turns them into long form writing, because you can’t write. They say this is necessary because they aren’t good writers, but this creates a self fulfilling prophecy if they just avoid writing entirely.
A sad time when people are allowing their ability to read and write fall by the wayside. These used to be skills people valued and took pride in improving for themselves. Now people shut their brain off.
Any clever prompting techniques that give you an edge today will evaporate quickly. People figure out the tricks, models absorb them, and tools automate them away.
There's no substitute for actually coding to learn software development. For new engineers, I'd strongly recommend limiting AI code generation on real work. Use it to explain concepts, not do the work for you. Otherwise you'll never develop the judgment to know what that 10% actually is.
An example would be economics - an LLM can spit out a bunch of words about an economic model. But if you don't take the time to learn, visualise and understand it for yourself - it means nothing to you. And in that case, if you already possess mastery why would you waste your resources playing around with an inferior tool to you?
You wouldn't.
So how many hours off work that 90% equates to?
I wonder if some of the proponents know where the line is in the art. I suspect not.