OP here. It's kind of ironic that making the docs AI-friendly essentially just ends up being what good documentation is in the first place (explicit context and hierarchy, self-contained sections, precise error messages).
It's the same for SEO also. Good structure, correct use of HTML elements, quick loading, good accessibility, etc. Sure, there are "tricks" to improve your SEO, but the general principles are also good if you were not doing SEO.
And yet in practice SEO slop garbage is SEO slop garbage. Devoid of any real meaning or purpose other than to increase rankings and metrics. Nobody cares if it’s good or useful, but it must appease the algorithm!
Yeah, I've started to think AI smoke tests for cognitive complexity should be a fundamental part of API/schema design now. Even if you think the LLMs are dumb, Stupidity as a Service is genuinely useful.
It's a good tool to use for code reviewing, especially if you don't have peers with Strong Opinions on it.
Which is another issue, indifference. It's hard to find people that actually care about things like API design, let alone multiple that check each other's work. In my experience, a lot of the time people just get lazy and short-circuit the reviews to "oh he knows what he's doing, I'm sure he thought long and hard about this".
It has changed how I structure my code. Out of laziness, if I can write the code in such a way that each step follows naturally from what came before, "the code just writes itself!" Except now it's literally true :D
Reminds me of that Asimov story where the main character was convinced that some public figure was a robot, and kept trying to prove it. Eventually they concluded that it was impossible to tell whether they were actually a robot "or merely a very good man."
From a docs-writing perspective, I've noticed that LLMs in their current state mostly solve the struggle of finding users who both want to participate in studies, are mostly literate, and are also fundamentally incompetent
Thank you for sharing this, it's really helpful to have this as top-down learning resource.
I'm in the process of learning how to work with AI, and I've been homebrewing something similar with local semantic search for technical content (embedding models via Ollama, ChromaDB for indexing). I'm currently stuck at the step of making unstructured knowledge queryable, so these docs will come in handy for sure. Thanks again!
We see a surprising number of folks who discover our product from GenAI solutions (self-reported). I'm not aware of any great tools that help you dissect this, but I'm sure someone is working on them.
A really effective prompt is created by developing an accurate “mental model” of the model, understanding what tools it does and doesn’t have access to, what gives it effective direction and what leads it astray
It's a bit different though; the soft skills you mention are usually realtime or a chore that people don't like doing (writing down specifications / requirements), whereas "prompt engineering" puts people in their problem solving mental mode not dissimilar to writing code.
1. Stuff that W3C already researched and defined 20 years ago to make the web better. Acessibility, semantic simple HTML that works with no JS, standard formats. All the stuff most companies just plain ignored or sidelined.
2. Suggestions to workaround obvious limits on current LLM tech (context size, ambiguity, etc).
There's really nothing to talk about category 1, except that a lot of people already said this and they were practically mocked.
Regarding category 2, it's the first stage of AI failure acceptance. "Ok, it can't reliably reason on human content. But what if we make humans write more dumb instead?"
As soon as some widget in the corner of a site wiggles to get my attention, I leave. If you/they want people to actually read their articles they shouldn't try to distract readers as soon as they start.
As soon as some widget in the corner of a site wiggles to get my attention, I leave.
Here's a bookmarklet I found on HN years and years ago. I have it bound to a hot key so whenever a web site does something stupid like that, I can dismiss it with a keystroke.
Works about 90% of the time, and doesn't require any installation of anything.
Thanks! I do have ublock origin and can typically get rid of these if I need to. It's just the frustration of going to websites that ostensibly want me to read something that see fit to destroy my focus as soon as I try to start.
I have a similar one that removes any fixed or sticky element, works for popovers like newsletter subscription prompts as well as long as they don't block scrolling:
If your software business relies on people coming to your site to read docs then you were cooked from the start. It's about enabling your users whether they're on your site, ChatGPT, or anywhere else.
I wish companies would invest more in docs. It's too hard to keep the quality high if it's just another thing for engineers to do. I've seen too many cases where a small group invests lots of time and effort bringing the docs up to standard and then another person or group comes along and starts dragging down the quality because they can't be bothered taking to time to see how and where their information fits and ensuring the formatting and styles are maintained.
Eventually the quality drops to such a level that some poor bastard spends their time bringing it all back up to standard - and the cycle repeats.
The most important characteristic of any internal documentation is trust. People need to trust it. If they trust it, they'll both read it and contribute to it. If they don't trust it they'll ignore it and leave it to rot.
Gaining that trust is really hard. The documentation needs to be safe to read, in that it won't mislead you and feed you stale information - the moment that happens, people lose trust in it.
Because the standard of internal docs at most companies is so low, employees will default to not trusting it. They have to be won over! That takes a lot of dedicated work, both in getting the documentation to a useful state and promoting it so people give it a chance.
An effect I've found of building agents that interacts with an API is that the bot serves as a smoke test for error handling. If your API doesn't return a clear and actionable error, or doesn't validate json fields clearly or correctly, the bot will end up finding it out and getting confused.
I can imagine a near future where crud endpoints are just entirely tested by an AI service that tries to read the docs and navigate the endpoints and picks up any inconsistencies and faults.
Documentation communicates intent, design decisions, and usage patterns that are often implicit or scattered throughout the source code, making it valuable even when the code is available.
Code is the final implementation, you / an AI can read the code and explain what it does, but it can't explain why it was written or when it should be used without more context.
Docs function both as a summary of code (functionality) and use cases. The day ai can infer both from code perfectly is the day ai can write the dependency on the fly.
Readit News
https://stytch.com/blog/if-an-ai-agent-cant-figure-out-how-y...
Which is another issue, indifference. It's hard to find people that actually care about things like API design, let alone multiple that check each other's work. In my experience, a lot of the time people just get lazy and short-circuit the reviews to "oh he knows what he's doing, I'm sure he thought long and hard about this".
I'm in the process of learning how to work with AI, and I've been homebrewing something similar with local semantic search for technical content (embedding models via Ollama, ChromaDB for indexing). I'm currently stuck at the step of making unstructured knowledge queryable, so these docs will come in handy for sure. Thanks again!
We see a surprising number of folks who discover our product from GenAI solutions (self-reported). I'm not aware of any great tools that help you dissect this, but I'm sure someone is working on them.
0: Generative Engine Optimization
It's just effective linguistics and speech; what people have called "soft skills" forever is now, obviously, trying to be a science for some reason.
Otherwise known as empathy
(assumption / personal theory)
1. Stuff that W3C already researched and defined 20 years ago to make the web better. Acessibility, semantic simple HTML that works with no JS, standard formats. All the stuff most companies just plain ignored or sidelined.
2. Suggestions to workaround obvious limits on current LLM tech (context size, ambiguity, etc).
There's really nothing to talk about category 1, except that a lot of people already said this and they were practically mocked.
Regarding category 2, it's the first stage of AI failure acceptance. "Ok, it can't reliably reason on human content. But what if we make humans write more dumb instead?"
Here's a bookmarklet I found on HN years and years ago. I have it bound to a hot key so whenever a web site does something stupid like that, I can dismiss it with a keystroke.
Works about 90% of the time, and doesn't require any installation of anything.
javascript:(function()%7B(function%20()%20%7Bvar%20i%2C%20elements%20%3D%20document.querySelectorAll('body%20*')%3Bfor%20(i%20%3D%200%3B%20i%20%3C%20elements.length%3B%20i%2B%2B)%20%7Bif%20(getComputedStyle(elements%5Bi%5D).position%20%3D%3D%3D%20'fixed')%20%7Belements%5Bi%5D.parentNode.removeChild(elements%5Bi%5D)%3B%7D%7D%7D)()%7D)()
> Step one, write the documentation yourself.
> Step two, bots hit your website hundreds of times per minute.
> Step three, users never come to your site, they use OpenAI's site.
> Step four, ??? openAI profits
Eventually the quality drops to such a level that some poor bastard spends their time bringing it all back up to standard - and the cycle repeats.
Gaining that trust is really hard. The documentation needs to be safe to read, in that it won't mislead you and feed you stale information - the moment that happens, people lose trust in it.
Because the standard of internal docs at most companies is so low, employees will default to not trusting it. They have to be won over! That takes a lot of dedicated work, both in getting the documentation to a useful state and promoting it so people give it a chance.
I can imagine a near future where crud endpoints are just entirely tested by an AI service that tries to read the docs and navigate the endpoints and picks up any inconsistencies and faults.