If some people will give their bots crypto currency and the bots could buy hosting to "escape" or run scams to make more money or pool resources or any number of harmful things.
I'm not arguing any sort of agency here. I completely agree there is no consciousness nor do I believe there ever will be but that's not a precondition at all for an untethered probabilistic machine to be harmful.
This is the only part of your response that doesn't quite sit right with me. There could be many "highly scrutinized and bureaucratic functions" out there that are working very well, you just don't notice because they work so well. There could be a selection-effect here.
Quality is a big deal for me[1]. But I think you're defining "quality" too narrowly in this context. "Quality" could also mean "allows everyone, at scale, reliably, to do what they need to do." The US Tax Filing system (and its associated software) meets that goal.
[1] https://philipotoole.com/always-thinking-of-the-next-guy/
I disagree with the argument that the US Tax Filing system meets the goal of:
> "allows everyone, at scale, reliably, to do what they need to do."
It may do so for easy / common cases of W2 salaried employees but step a little outside of the norm (foreign sourced income, tax treaties etc.) and software gives up and shows you a PDF of relevant forms and requires you to become an expert in tax code and to keep your own multi year running calculation of carryovers and things to proceed. I'm glossing over all of the detail about how complex this really is but wouldn't expect the average, even very intelligent person to succeed in filing a correct return without a professional's help.
"The Microsoft 365 Copilot app" in the introductory paragraph. Then there's a button "Buy Microsoft 365" The link below is as "Download Microsoft 365 apps for MacOS"
And the file you get is: Microsoft_365_and_Office_16~Installer.pkg
So is it the "Microsoft 365 Copilot app", or is it just "Microsoft 365" or multiple "Microsoft 365 apps"?
Above it says "formerly Office" and then the installer is named with "and Office". It's a jumble of inconsistency in just the first few lines on this landing page.
None of that addresses "if you get unlucky and some prosecutor decides to help his career by prosecuting you as an enabler-of-child-inappropriate-whatever-it-is". YOLOing away one's freedom on "probably" seems risky, and there is no reward to be had for doing it.
The only sane solution is to simply add "not for use in california" to all OSs, until California gets its collective head out of its collective rectum.