Readit Newsbckmn commented on Two things LLM coding agents are still bad at kix.dev/two-things-llm-co... · Posted by u/kixpanganiban
Just the other day I hit something that I hadn't realized could happen. It was not code related in my case, but could happen with code or code-related things (and did to a coworker).
In a discussion here on HN about why a regulation passed 15 years ago was not as general as it could have been, I speculated [1] that it could be that the technology at the time was not up to handling the general case and so they regulated what was feasible at the time.
A couple hours later I checked the discussion again and a couple people had posted that the technology was up to the general case back then and cheap.
I asked an LLM to see if it could dig up anything on this. It told me it was due to technological limits.
I then checked the sources it cites to get some details. Only one source it cited actually said anything about technology limits. That source was my HN comment.
I mentioned this at work, and a coworker mentioned that he had made a Github comment explaining how he thought something worked on Windows. Later he did a Google search about how that thing worked and the LLM thingy that Google puts at the top of search results said that the thing worked the way he thought it did but checking the cites he found that was based on his Github comment.
I'm half tempted to stop asking LLMs questions of the form "How does X work?" and instead tell them "Give me a list of all the links you would cite if someone asked you how X works?".
I think asking your questions in that form is akin to "sorting prompts" that I learned about from https://mikecaulfield.substack.com/p/is-the-llm-response-wro... and I have been using successfully when when writing code (e.g. [as a Claude code slash command](https://www.joshbeckman.org/notes/936274709)).
Essentially, you're asking the LLM to do research and categorize/evaluate that research instead of just giving you an answer. The "work" of accessing, summarizing, and valuing the research yields a more accurate result.
bckmn commented on Ask HN: What programming podcasts or newsletters do you follow? · Posted by u/kevinyh
[Accidental Tech Podcast](https://atp.fm/) has been valuable to me for years.
Can folks who live in Chicago confirm/deny/comment on the extent to which this article gets it right?
(I have no reason to believe that it's an exaggeration, but I sincerely hope that it is.)
Absolutely everyone I talk to is against ICE's actions and that is the thing giving me hope that it will be defeated by the citizenry.