Disclosure, I do work for Josh, and I can tell you that he's thought quite deeply about the negative implications of the agents that are coming. Among enumerating the ways in which AI agents will transform knowledge work, this points out the ways which we might come to regret.
> Even if this plays out over 20 or 30 years instead of 10 years, what kind of world are we leaving for our descendants?
> What should we be doing today to prepare for (or prevent) this future?
No, "motivation" is what puts one into motion, hence the name. AIs have constraints and even agendas, which can be triggered by a prompt. But it's not action, it's reaction.
DeepSeek may produce a perfectly good web site explaining why Taiwanese independence is not a thing, and how Taiwan wants back to the mainland. But it's won't produce such a web site by its own motivation, only in response to an external stimulus.
The article doesn't talk about any agents outside of coding work. Coding is not the work the world is running on. Agent concept requires much more selling than chat bots, which means they are solutions searching for problems.
the author also starts with "The fundamental difference with AI agents is that they take the human completely out of the loop, and this changes everything.", then focuses on coding. Is anyone actually having success with completely autonomous agents coding; no human oversight or validation?
He then presents a very naive vision of how agents are superior, where it basically all comes down to "generate code more efficiently" - has that ever been the crux challenge to solving problems with software?
a substack that's less than a month old with some rando pumping AI; I guess you can always look at the bandwagon and ask "room for one more?"
Some of the world is running on emails and excel sheets. That’s doable for an agent already, if you’re willing to let it loose. Problem is, how do you get all your values and unknown knowns into its context?
Willing to let them loose is the more salient point. If you let your agents loose on your entire body of output and tools at work, then you'll build that knowledge up pretty quickly.
Tall ask right now, with privacy and agency (no pun intended) concerns
I can say with absolute certainty that I have never used an LLM to tweak an email, and will never, ever use an LLM “agent” on my email, work or personal.
“Hey, how’s that hardware/software integration effort coming? What are your thoughts on the hardware so far?”
Interesting thought experiment, replace "AI agent" with "computer" in this article. Seems our parents/grandparents may have been having some of the same conversations 50 years ago.
---
The advantages of computers over human employees:
1. The best computer can be copied infinitely.
2. Computers can run 24/7
3. Computers could theoretically think faster than humans
4. Computers have minimal management overhead
5. Computers can be instantly scaled up and down
6. Computers don’t mind running in a nightmare surveillance prison
There are a lot of things you can do from a shell prompt, and now we have AI ghosts that can do them too, sometimes better than us. Yes, within some industries, this is going to be huge!
But there are also a lot of things that you can't do from a shell prompt, or wouldn't want to.
The largest resource use of AI over the next 50 years will be generating entertainment structures for humans. Productivity focused AI will be the most economically useful, however it'll be far less resource intensive than the entertainment generation (generally speaking, AI tasked with driving human pleasure).
World building alone will be at least a magnitude greater in resource use than all productivity-focused AI combined (including robotics + AI). Then throw in traditional media generation (audio, images, video, textual).
AI will be the ultimate sedative for humanity. We're going into the box and never coming back out and absolutely nothing can stop that from happening. For at least 95% of humanity the future value that AI offers in terms of bolstering pleasure-of-existence is far beyond the alternatives it's not really worth considering any other potential outcome, there will be no other outcome. Most of humanity will lose interest in the mundane garbage of dredging through day to day mediocrity (oh I know what you're thinking: but but but life isn't really that mediocre - yes, it definitely is, for the majority of the eight billion it absolutely is).
Out there is nothing, more nothing, some more nothing, a rock, some more nothing, some more of what we already know, nothing, more nothing, and a lot more nothing. In there will be anything you want. It's obvious what the masses will overwhelmingly choose.
This is basically the modern version of an Influencer...just on Substack instead of YouTube. Big claims, slick framing, zero rigor. It sells a narrative about “agents” as a brand, not an analysis of what actually works.
This article is basically just saying if we have AGI then there might be big consequences for humans. Well yes, obviously. People have been discussing that for decades...
Readit News
I am glad I don't work for this person.
> Even if this plays out over 20 or 30 years instead of 10 years, what kind of world are we leaving for our descendants?
> What should we be doing today to prepare for (or prevent) this future?
DeepSeek may produce a perfectly good web site explaining why Taiwanese independence is not a thing, and how Taiwan wants back to the mainland. But it's won't produce such a web site by its own motivation, only in response to an external stimulus.
He then presents a very naive vision of how agents are superior, where it basically all comes down to "generate code more efficiently" - has that ever been the crux challenge to solving problems with software?
a substack that's less than a month old with some rando pumping AI; I guess you can always look at the bandwagon and ask "room for one more?"
Tall ask right now, with privacy and agency (no pun intended) concerns
“Hey, how’s that hardware/software integration effort coming? What are your thoughts on the hardware so far?”
Fuck me if I let an LLM answer that.
---
The advantages of computers over human employees:
1. The best computer can be copied infinitely.
2. Computers can run 24/7
3. Computers could theoretically think faster than humans
4. Computers have minimal management overhead
5. Computers can be instantly scaled up and down
6. Computers don’t mind running in a nightmare surveillance prison
7. Computers are more tax efficient
But there are also a lot of things that you can't do from a shell prompt, or wouldn't want to.
World building alone will be at least a magnitude greater in resource use than all productivity-focused AI combined (including robotics + AI). Then throw in traditional media generation (audio, images, video, textual).
AI will be the ultimate sedative for humanity. We're going into the box and never coming back out and absolutely nothing can stop that from happening. For at least 95% of humanity the future value that AI offers in terms of bolstering pleasure-of-existence is far beyond the alternatives it's not really worth considering any other potential outcome, there will be no other outcome. Most of humanity will lose interest in the mundane garbage of dredging through day to day mediocrity (oh I know what you're thinking: but but but life isn't really that mediocre - yes, it definitely is, for the majority of the eight billion it absolutely is).
Out there is nothing, more nothing, some more nothing, a rock, some more nothing, some more of what we already know, nothing, more nothing, and a lot more nothing. In there will be anything you want. It's obvious what the masses will overwhelmingly choose.
Which means they would have no empathy when tasked with running a nightmare surveillance prison for humans.