> The usage of LLMs at work, in government, policing, coding, etc is so concerning because of that. They will validate whatever poor reasoning people throw at them.
These machines are too useful not to exist, so we had to invent them.
https://en.wikipedia.org/wiki/The_Unaccountability_Machine
> The Unaccountability Machine (2024) is a business book by Dan Davies, an investment bank analyst and author, who also writes for The New Yorker. It argues that responsibility for decision making has become diffused after World War II and represents a flaw in society.
> The book explores industrial scale decision making in markets, institutions and governments, a situation where the system serves itself by following process instead of logic. He argues that unexpected consequences, unwanted outcomes or failures emerge from "responsibility voids" that are built into underlying systems. These voids are especially visible in big complex organizations.
> Davies introduces the term “accountability sinks”, which remove the ownership or responsibility for decisions made. The sink obscures or deflects responsibility, and contributes towards a set of outcomes that appear to have been generated by a black box. Whether a rule book, best practices, or computer system, these accountability sinks "scramble feedback" and make it difficult to identify the source of mistakes and rectify them. An accountability sink breaks the links between decision makers and individuals, thus preventing feedback from being shared as a result of the system malfunction. The end result, he argues, is protocol politics, where there is no head, or accountability. Decision makers can avoid the blame for their institutional actions, while the ordinary customer, citizen or employee face the consequences of these managers poor decision making.
Is it really so difficult to imagine how people will use (or misuse) tools you build? Are HNers or tech people in general just very idealistic and naive?
Maybe I'm the problem though. Maybe I'm a bad person that is always imagining how many bad ways I would abuse any kind of system or power that I can, even though I don't have any actual intention to actually abuse systems