> If you don’t review, check, and merge docs the same way your org reviews, checks, and merges code, you’re not doing docs-as-code — you’re doing docs-as-bore.
While some WYSIWYG cloud-based docs platforms make it easier to make changes, that's not necessarily what you want. Docs are a critical component of how your users perceive your product - you want to have checks that it meets certain quality and accuracy standards. Just like your code.
And if you're an engineering lead company, you probably want your docs updates to be coordinated with your product releases. Git is just the logical place to put your docs in that case.
I've even created a company specifically to help with this workflow: https://www.doctave.com
Also, lots of comments here seem to be thinking of docstrings and other in-code documentation. I think that's really a different category that has a different set of goals and issues. This post is specifically about customer-facing documentation.
> you probably want your docs updates to be coordinated with your product releases
People keep saying that but I can't help to wonder if they're every on the receiving end of documentation.
Documentation isn't code. It may all be text, and share other similarities, but it's something fundamentally different. You can't substitute code for docs, or docs for code. The above reasoning is what ends up publishing /docs/5.3/chapter1/installing.html. Reading documentation for a specific version of the product is not desirable.
To understand the product, it's very relevant what happens in version 5.4, and I should not have to diff different releases of the docs to find out. The documentation should say clearly that this function will be deprecated, or look different in the next version. In short, documentation should neither be branched with code, nor released with it. It should live in parallel and describe not only how the product works, but more importantly why it works like it does.
Your product looks really interesting. I'm a big advocate for docs-as-code at my company. We use Confluence generally, but our team uses GitLab and Gatsby for our documentation/blogs as we really value the Merge Request workflow.
Our biggest challenge is local development. A WYSIWYG is just so useful in that regard. Is that what Doctave Studio is trying to solve?
You're correct - Doctave Studio is for making local development easier. It packages the whole "authoring environment", so it's all you need to start writing.
It's technically not WYSIWYG, but you do get a side-by-side real-time preview of your rendered Markdown content and OpenAPI specs that update as you type.
You get autocomplete, broken links checking, etc. Everything you'd expect from an editor.
Having worked both as a developer and as a technical communicator (for software), I'm thinking that low friction for developers is paramount, and that therefore the way to get developers to put some effort into it is to have documents both (a) written in Markdown (or adoc or rST or typst) and (b) co-located with code under version control. Change the code, change the docs, no screwing around, BUT a quick & simple brain dump can suffice because of [see next paragraph].
To whit: I have yet to hear of a documentation system that provides fully bidirectional updates between such documents-as-code and edits made further downstream along the documentation production pipeline. That is to say, when a TC person or a reviewer makes edits to content, these changes should propagate back to the docs-as-code material.
Then everyone benefits, including senior devs whose scarce time is optimised by having TC people expand and polish their hasty scribblings, and junior devs who have well-maintained documentation at-hand in-place.
Documentation near code is a good idea, but unfortunately it covers only part of the problem.
There is a big portion of documentation that should be available to "other persons", such as architects or project managers, who may not want to visit the codebase.
Another challenge is that for these people, diagrams are typically more useful than text. This still requires some manual effort which is difficult to achieve with Markdown.
I recently started keeping docs in Markdown but with a CD job to render them and publish to Confluence. We only have internal users but many of them are non-technical. This seems to be a sweet spot -- easy for us to update, but also easy for users to find.
It also opens up the possibility of generating the docs themselves from any data that are available at build time. For example, we have a page with a pretty complex Graphviz visualization that gets built from some of the data files we ship with the product. When the data changes, so do these diagrams. They literally can't get out of sync without us noticing (the build breaks). I see more opportunities for this kind of thing all the time now that I'm looking.
Agreed with your point of accessibility to people who don't live in the code.
But that doesn't preculude documentation near code nor Markdown. It just means that you need a CI job to publish the doc stored in the git repo as (for example) HTML or PDF.
For diagrams stuff like PlantUML are great, edit as text, publish as images.
Personally, I think they should get over it. Git is not a "software dev" program. Version control is crucial to pretty much everyone who touches the product. I think therefore everyone should know how to navigate their codebase and use Git, at least a little.
Also, you can use git submodules for your documentation so it's kind of separate from code. The only problem is submodules kind of suck ass, so I don't know if devs would be keen for that.
Regarding para 2, I am assuming that there is something like a CMS in place that can pluck docfiles from version control and massage them and insert them into an outline/ToC. (And then propagate changes back to version control.)
Regarding para 3, there's now many GUI tools for working with Mermaid et al. But are any of them properly integrated into documentation systems ?
Architects should definitely be visiting the codebase. Why would project managers be updating docs? I'd have thought stuff like docs translations done by translators would be a better candidate for non-code editing.
the second you put docs into version control next to code it's no longer low friction enough. Suddenly you have peer review for docs change is far too much work.
In all my 26 years of working as a software dev, in the ~20 companies that I've worked or consulted for in various countries, never once did I see code documentation work out well.
It wasn't that the will was not there: in almost every single of the companies the devs and management agreed that good documentation was important. And every now and then some heroic effort would be made to finally clean up Confluence or whatever system was used at the time. And for a little while that would work, then slowly documentation would become neglected here and there, and in the end it would become so bad that nobody would use it anymore. Except perhaps to showcase your commitment to best dev practices to your manager when it was time for your performance review.
It doesn't matter if documentation has to be added in the code, or to some external system or both. Sooner or later it will be neglected, become out of sync and eventually become worse than no documentation at all.
With the exception of external libraries or API's, software simply changes too fast. You'd need an army of technical writers to keep all documentation in sync.
And unlike what the author writes, I have actually been working for a couple of companies that gave up and simply used the code as documentation, and surprisingly this worked out pretty well.
Sad to read your n=20 was like that. There are also cases where documentation led companies to growth—one example is Stripe.
Poking the finger at documentation is easy because it's visible and readily available. Would you say the bit rot phenomenon you described never happened to code, or processes, or UIs? Docs reflect organizations.
If people aren't reading your documentation it means they don't trust your documentation.
The solution is to build that trust, by building a culture of active documentation maintenance.
My favorite trick for that is to keep the docs in the same repo as the code and actively enforce relevant documentation updates as part of the code review process.
Once developers learn that new code cannot land on main without accompany documentation updates they learn to trust that documentation pretty fast.
> My favorite trick is to [...] actively enforce [...]
If active enforcement were an option, and it led to success, I don't imagine there would be a problem.
My favorite trick is to tell people really nicely they should write comments (preferrably doc comments on internally exported interfaces).
Yet, they don't. And so we're back to the dilemma:
If a significant percentage of competent programmers choose to not comment their code, or update the comments that are in the code, the comments will lie over time. And so it is probably better to not include them.
We haven't even reached the point in the conversation where we ask "But why is it you, dear competent programmers with decades of experience, think we should continue to minimize the amount of comments with good conscience?" or ask "Why is it, dear project managers, that we tolerate a total lack of communication through any other means than executable code?" And the answer is probably: Because a lot of autists produce a lot of valuable code, and they don't like to talk if they can avoid it, and we value their work and relay what they made to the wordy-word people.
> If people aren't reading your documentation it means they don't trust your documentation.
I don't trust your documentation unless there's alignment among developers. Alignment is a luxury you don't get in legacy shops.
Executable tests in Rustdoc are amazing. For those not involved, they are run when you write `cargo test` and they are included as markdown code blocks in your crate’s documentation.
They’re not an excellent place for extensive testing. But they are super useful for making sure your documentation examples are updated and functional.
#![deny(missing_docs)] is also a great way to ensure you don’t forget to document things.
I read the article as referring to documentation for end users -- not internal documentation / comments written by developers who thought they knew what the code they wrote was doing.
I personally read and write a lot. I track things in git messages by cross-referencing issues, and I comment my code.
But my point is: when you have a cultural divide among competent programmers on whether to comment, not commenting game-theoretically wins, because the outcome where you have comments that get updated some percent of the time is worse than not having those comments.
Instead, embed what you want to say in a comment in the code itself, or in a test.
Documents your libraries and APIs if they are used by people outside your team.
I tried to do it seriously a number of times. Perhaps I am doing it wrong, but my productivity drops like a stone.
Writing down your thoughts as you go and maintaining them takes serious time. The romantic notion of writing (and reading) code like a book is appealing, but writing books is hard and arduous, it cannot be underestimated as a craft on its own right, and there is coding to be done.
There is also the question of structuring the literate code. Telling a story of how you are building it or explaining how it works has a very different flow and order to how code is usually structured for good maintainability.
Please correct me if I’m wrong, because I would love to dive into it, but I don’t think there has ever been any major piece of software developed following literate programming (at least as Knuth envisioned it). I also don’t think there is any significant book that contains a sizeable working program embedded in it throughout, that can be compiled and executed as-is.
In practice Knuth was most concerned with embedding short code snippets in his papers and books. Having the whole thing be an actual compilable program was secondary, and it was mostly short academic proof-of-concept prototypes and algorithms.
Don’t get me wrong, I love the concept, that’s why I have given it multiple serious tries over the years, likely I will again, and why I think I have some insight of what happens when you use it for “real-world” work.
Knuth is not the average programmer by far. And I am not talking about coding skills. Knuth is a writer at heart. He was also from a time where writing code on paper was the norm. Literate programming is good for Knuth, but maybe not for most coders today, who grew up on fast computers and IDEs.
my favourite documentation is minimal running code examples. give me example inputs to get the job done - i.e. not just "inputs: x is a y", but actually create a minimal version of y and show it going into f(x) and coming back out as a genuine object (as opposed to a mock) that I can inspect/prod until I understand what's happening.
Depends how you write code. When I use Semantic Kernel, my KernelFunctions include a well-defined documentation for the inputs and outputs, then using the System Prompt you can provide the concepts and glue between the various plugins. It is the function specification as a whole. Precision is important, although GPT is not yet perfect -- perhaps in another year or two it will be.
Not at any level? My imagination is reeling—I’m thinking of Cold War era Spy Novels describing siloed groups who don’t know what any other group is doing. Each is laser focused on their own tasks and everything is a tactical choice. It’s the great reveal in the Movie adaptation when the separate threads come together and the grand national strategy is revealed. And now I’m also imagining the comedy versions of this genre, where the reveal has no purpose.
At some level there is _structure_, and it can be communicated. If for no other reason than to validate the evident structure in code.
> “…sitting at the same table as the almighty coding knights? […] Remove documentation…and your products cease to exist, their inner workings left to the imagination of…”
The author finished the sentence with “users”, but who are they talking about? Those coding knights and their imagination which sort of parallels my Spy genre example.
That said, the article seems a bit naive about the depth and breadth—reads like Ra-Ra.
I prefer when developers and project managers create massive google docs for specs and descriptions. Double points if you share the document with only a handful of favorite employees. Also, ignore all requests to get permission to this document. Eye roll in all meetings if someone hasn't read this document. You can get to god mode if you hide comments around the doc.
Doing docs-as-code by putting documentation in the codebase is somewhat of an oversimplification.
Usually, you want to use feature flags to decouple changes in Product from Engineering-driven deployments, so that new feature work can be rolled out slowly and its impact on KPIs measured, and so that engineering can de-risk the delivery of large changes by putting those changes behind feature flags with 0% rollout.
The problem is, documentation is much closer to Product than Engineering, meaning, doing docs-as-code correctly means putting your documentation behind the same feature flags. So you should have both a public static site that shows the documentation for features whose flags have 100% rollout, plus your application itself should also show the documentation, so that you can show the documentation to users for feature flags enabled for those users.
There aren't really any frameworks that have this docs-coupled-to-feature-flags pattern available off-the-shelf and most companies don't consider it high-enough priority to build in-house.
No, the closest is staging documentation changes until the features reach 100% rollout, then manually pressing a button to try to launch the docs at the same time. But when users don't find docs for new features (that are rolled out only to them), they naturally revert back to trying to talk directly to Engineering, which doesn't scale.
An example of this mismatch is when QA isn't testing documentation changes: best-practice QA discovery-testing is for QA to have a production tenant, where the feature flag is enabled only for the QA tenant, but if QA doesn't have documentation for how the feature is supposed to work, then they end up having completely separate infrastructure that nobody else pays attention to, or else QA is talking directly to Product/Engineering, which somewhat defeats the purpose since that's something that customers can't do.
I was thinking this might have been a treatise on prompt engineering, or something like Rational Rose.
I was actually pleasantly disappointed. I am glad to see documentation treated as a "first class" engineering Discipline. I would say the same goes to creative authors, such as graphic designers, or 3D modelers.
I'm fairly big on good documentation[0], myself, and feel its absence, whenever I look at most codebases, these days.
> Documentation is vital software infrastructure
Definitely agree. I am feeling the decline in Apple's documentation (I'm an old-time Apple app developer, and remember Inside Macintosh). It's gotten absolutely awful, these days. I'm just starting on a SwiftUI app (I think I may have finally found a good application for it), and, boy howdy, is that documentation ... less than ideal.
Fortunately, someone else thinks similarly, and did something about it[1]. I hate to rely on these types of things, because they often lose sync with the subject, but it is totally necessary.
This article is awesome and resonates with my experience! I used to work for one of the big cloud vendors. We frequently had doc writers in our sync calls. Quite frankly, they never had a chance! Much of our documentation, our errors, status codes etc. was burried in the code and we (developers) were simply to busy to work effectively with technical writers.
Don't get me wrong here. Where I say "busy" I don't mean their work wasn't important (it was!) but we had our own projects and jumping on a call to improve errors messages is not too exciting for most of us - and it's not the stuff that drives promotions. As a develper it's simply to easy to gatekeep code.
Now follows a shameless plug: I'd be so excited to hear your opinion about a tool I'm building. It's called https://api-fiddle.com. Conceptually, it should help developers work API-first instead of code-first. It's great for devs because it makes the work faster and safer. BUT, an added benefit is that API docs are no longer burried in code and technical writers have a nice interface to contribute to them (and hold devs accountable!).
Readit News
> If you don’t review, check, and merge docs the same way your org reviews, checks, and merges code, you’re not doing docs-as-code — you’re doing docs-as-bore.
While some WYSIWYG cloud-based docs platforms make it easier to make changes, that's not necessarily what you want. Docs are a critical component of how your users perceive your product - you want to have checks that it meets certain quality and accuracy standards. Just like your code.
And if you're an engineering lead company, you probably want your docs updates to be coordinated with your product releases. Git is just the logical place to put your docs in that case.
I've even created a company specifically to help with this workflow: https://www.doctave.com
Also, lots of comments here seem to be thinking of docstrings and other in-code documentation. I think that's really a different category that has a different set of goals and issues. This post is specifically about customer-facing documentation.
Nit: engineering-led
People keep saying that but I can't help to wonder if they're every on the receiving end of documentation.
Documentation isn't code. It may all be text, and share other similarities, but it's something fundamentally different. You can't substitute code for docs, or docs for code. The above reasoning is what ends up publishing /docs/5.3/chapter1/installing.html. Reading documentation for a specific version of the product is not desirable.
To understand the product, it's very relevant what happens in version 5.4, and I should not have to diff different releases of the docs to find out. The documentation should say clearly that this function will be deprecated, or look different in the next version. In short, documentation should neither be branched with code, nor released with it. It should live in parallel and describe not only how the product works, but more importantly why it works like it does.
Our biggest challenge is local development. A WYSIWYG is just so useful in that regard. Is that what Doctave Studio is trying to solve?
It's technically not WYSIWYG, but you do get a side-by-side real-time preview of your rendered Markdown content and OpenAPI specs that update as you type.
You get autocomplete, broken links checking, etc. Everything you'd expect from an editor.
To whit: I have yet to hear of a documentation system that provides fully bidirectional updates between such documents-as-code and edits made further downstream along the documentation production pipeline. That is to say, when a TC person or a reviewer makes edits to content, these changes should propagate back to the docs-as-code material.
Then everyone benefits, including senior devs whose scarce time is optimised by having TC people expand and polish their hasty scribblings, and junior devs who have well-maintained documentation at-hand in-place.
My 0.02€, YMMV. Maybe too niche.
There is a big portion of documentation that should be available to "other persons", such as architects or project managers, who may not want to visit the codebase.
Another challenge is that for these people, diagrams are typically more useful than text. This still requires some manual effort which is difficult to achieve with Markdown.
It also opens up the possibility of generating the docs themselves from any data that are available at build time. For example, we have a page with a pretty complex Graphviz visualization that gets built from some of the data files we ship with the product. When the data changes, so do these diagrams. They literally can't get out of sync without us noticing (the build breaks). I see more opportunities for this kind of thing all the time now that I'm looking.
But that doesn't preculude documentation near code nor Markdown. It just means that you need a CI job to publish the doc stored in the git repo as (for example) HTML or PDF.
For diagrams stuff like PlantUML are great, edit as text, publish as images.
Personally, I think they should get over it. Git is not a "software dev" program. Version control is crucial to pretty much everyone who touches the product. I think therefore everyone should know how to navigate their codebase and use Git, at least a little.
Also, you can use git submodules for your documentation so it's kind of separate from code. The only problem is submodules kind of suck ass, so I don't know if devs would be keen for that.
Regarding para 2, I am assuming that there is something like a CMS in place that can pluck docfiles from version control and massage them and insert them into an outline/ToC. (And then propagate changes back to version control.)
Regarding para 3, there's now many GUI tools for working with Mermaid et al. But are any of them properly integrated into documentation systems ?
It wasn't that the will was not there: in almost every single of the companies the devs and management agreed that good documentation was important. And every now and then some heroic effort would be made to finally clean up Confluence or whatever system was used at the time. And for a little while that would work, then slowly documentation would become neglected here and there, and in the end it would become so bad that nobody would use it anymore. Except perhaps to showcase your commitment to best dev practices to your manager when it was time for your performance review.
It doesn't matter if documentation has to be added in the code, or to some external system or both. Sooner or later it will be neglected, become out of sync and eventually become worse than no documentation at all.
With the exception of external libraries or API's, software simply changes too fast. You'd need an army of technical writers to keep all documentation in sync. And unlike what the author writes, I have actually been working for a couple of companies that gave up and simply used the code as documentation, and surprisingly this worked out pretty well.
Poking the finger at documentation is easy because it's visible and readily available. Would you say the bit rot phenomenon you described never happened to code, or processes, or UIs? Docs reflect organizations.
how do you know this ?
Any "documentation" that isn't embedded in code will never be opened again.
Any "documentation" in the form of doc-strings and comments will, over time, lie.
This is because competent programmers cannot agree on whether to even comment.
When any large percentage of competent programmers do not comment, it is better to not rely on comments.
Here's another pitch:
The solution is to build that trust, by building a culture of active documentation maintenance.
My favorite trick for that is to keep the docs in the same repo as the code and actively enforce relevant documentation updates as part of the code review process.
Once developers learn that new code cannot land on main without accompany documentation updates they learn to trust that documentation pretty fast.
If active enforcement were an option, and it led to success, I don't imagine there would be a problem.
My favorite trick is to tell people really nicely they should write comments (preferrably doc comments on internally exported interfaces).
Yet, they don't. And so we're back to the dilemma:
If a significant percentage of competent programmers choose to not comment their code, or update the comments that are in the code, the comments will lie over time. And so it is probably better to not include them.
We haven't even reached the point in the conversation where we ask "But why is it you, dear competent programmers with decades of experience, think we should continue to minimize the amount of comments with good conscience?" or ask "Why is it, dear project managers, that we tolerate a total lack of communication through any other means than executable code?" And the answer is probably: Because a lot of autists produce a lot of valuable code, and they don't like to talk if they can avoid it, and we value their work and relay what they made to the wordy-word people.
> If people aren't reading your documentation it means they don't trust your documentation.
I don't trust your documentation unless there's alignment among developers. Alignment is a luxury you don't get in legacy shops.
They’re not an excellent place for extensive testing. But they are super useful for making sure your documentation examples are updated and functional.
#![deny(missing_docs)] is also a great way to ensure you don’t forget to document things.
But my point is: when you have a cultural divide among competent programmers on whether to comment, not commenting game-theoretically wins, because the outcome where you have comments that get updated some percent of the time is worse than not having those comments.
Instead, embed what you want to say in a comment in the code itself, or in a test.
Documents your libraries and APIs if they are used by people outside your team.
If it's good enough for Knuth it's good enough for me.
Writing down your thoughts as you go and maintaining them takes serious time. The romantic notion of writing (and reading) code like a book is appealing, but writing books is hard and arduous, it cannot be underestimated as a craft on its own right, and there is coding to be done.
There is also the question of structuring the literate code. Telling a story of how you are building it or explaining how it works has a very different flow and order to how code is usually structured for good maintainability.
Please correct me if I’m wrong, because I would love to dive into it, but I don’t think there has ever been any major piece of software developed following literate programming (at least as Knuth envisioned it). I also don’t think there is any significant book that contains a sizeable working program embedded in it throughout, that can be compiled and executed as-is.
In practice Knuth was most concerned with embedding short code snippets in his papers and books. Having the whole thing be an actual compilable program was secondary, and it was mostly short academic proof-of-concept prototypes and algorithms.
Don’t get me wrong, I love the concept, that’s why I have given it multiple serious tries over the years, likely I will again, and why I think I have some insight of what happens when you use it for “real-world” work.
At some level there is _structure_, and it can be communicated. If for no other reason than to validate the evident structure in code.
> “…sitting at the same table as the almighty coding knights? […] Remove documentation…and your products cease to exist, their inner workings left to the imagination of…”
The author finished the sentence with “users”, but who are they talking about? Those coding knights and their imagination which sort of parallels my Spy genre example.
That said, the article seems a bit naive about the depth and breadth—reads like Ra-Ra.
I'm in the camp of a little verbosity and repetition for the sake of clarity is worth it.
Usually, you want to use feature flags to decouple changes in Product from Engineering-driven deployments, so that new feature work can be rolled out slowly and its impact on KPIs measured, and so that engineering can de-risk the delivery of large changes by putting those changes behind feature flags with 0% rollout.
The problem is, documentation is much closer to Product than Engineering, meaning, doing docs-as-code correctly means putting your documentation behind the same feature flags. So you should have both a public static site that shows the documentation for features whose flags have 100% rollout, plus your application itself should also show the documentation, so that you can show the documentation to users for feature flags enabled for those users.
There aren't really any frameworks that have this docs-coupled-to-feature-flags pattern available off-the-shelf and most companies don't consider it high-enough priority to build in-house.
An example of this mismatch is when QA isn't testing documentation changes: best-practice QA discovery-testing is for QA to have a production tenant, where the feature flag is enabled only for the QA tenant, but if QA doesn't have documentation for how the feature is supposed to work, then they end up having completely separate infrastructure that nobody else pays attention to, or else QA is talking directly to Product/Engineering, which somewhat defeats the purpose since that's something that customers can't do.
I was actually pleasantly disappointed. I am glad to see documentation treated as a "first class" engineering Discipline. I would say the same goes to creative authors, such as graphic designers, or 3D modelers.
I'm fairly big on good documentation[0], myself, and feel its absence, whenever I look at most codebases, these days.
> Documentation is vital software infrastructure
Definitely agree. I am feeling the decline in Apple's documentation (I'm an old-time Apple app developer, and remember Inside Macintosh). It's gotten absolutely awful, these days. I'm just starting on a SwiftUI app (I think I may have finally found a good application for it), and, boy howdy, is that documentation ... less than ideal.
Fortunately, someone else thinks similarly, and did something about it[1]. I hate to rely on these types of things, because they often lose sync with the subject, but it is totally necessary.
[0] https://littlegreenviper.com/leaving-a-legacy/
[1] https://swiftui-lab.com
Don't get me wrong here. Where I say "busy" I don't mean their work wasn't important (it was!) but we had our own projects and jumping on a call to improve errors messages is not too exciting for most of us - and it's not the stuff that drives promotions. As a develper it's simply to easy to gatekeep code.
Now follows a shameless plug: I'd be so excited to hear your opinion about a tool I'm building. It's called https://api-fiddle.com. Conceptually, it should help developers work API-first instead of code-first. It's great for devs because it makes the work faster and safer. BUT, an added benefit is that API docs are no longer burried in code and technical writers have a nice interface to contribute to them (and hold devs accountable!).