My problem is, I know that we are missing information, what I don't know is how to tease out that information from myself and other coworkers. Do you have any advice on how to get the knowledge out of my coworkers and into the knowledge base? Or, do you have a book you recommend? I have found a few books on knowledge bases but the reviews seem to be pretty hit and miss.
Edit: After talking with some friends about it I think I was able to articulate my main issue I have. In making this documentation it feels like I am winging it. However, all my training has been in coding and coding always has standards, guidelines and frameworks. It's hard for me to just work on the docs as I feel like I should be following some sort of standard as I do it.
Edit: Currently I am stuck with Confluence as the actual technology to use.
As an expert I often think things are obvious that are not. So it takes a few rounds to make useful documentation.
It takes discipline, as otherwise everyone skips the "write documentation" step, and it will annoy some people with high priority issues.
But IMHO, it's the only thing that actually scales if you truly want comprehensive, continuously-updated documentation.
Side note: An excellent way to generate drafts for missing content is have your new hires write them as they get up to speed. By definition they're looking everything up, so they'll notice gaps. And they have time, before they're fully up to speed. And as long as someone with knowledge proofs their draft, their lack of experience shouldn't matter.
We are currently in the process of creating a data dictionary for our company.
I put down a rule that we are not ever going to create a SEPARATE WordDoc/Wiki/Evernote/GenZ-tool.
If it is code, document its meaning (english explanation for laymen) in the doc string. If it is data, document its meaning in the column's metadata (all database systems provide a property/comment/description capability at table & column level). If it requires complex diagrams, put these in whatever files (doc/image/pdf/mp3), store it as a blob in a database and create a link it in original code comments/column metadata.
All this data is then constantly pulled into some data-mart on top of which a query-able UI is created.
Every documentation must be literally "Tied" to actual systems making money for the company.
If a new code/data is created or updated, and it's missing documentation, PR is not approved. Once the basic technical setup of tying code/data with comments/metadata is done, enforcing this rule of updating documentation is the job of management/CTO culture.
How do you deal with outdated info that's still useful to keep around?
I have a few internal wiki-pages which are 80% 'stuff I tried that doesn't work but I'd like to keep around because the tries are valuable in case I re-visit these angles', 20% 'stuff that worked in the end'. I find the 80% confuse newcomers, and I still have to explain which parts are important.
You separate out them out into 2 related parts of documentation. My guess is that you want them in the same wiki page because that's how you wrote it and that's how the information lives in your head since it's all for the same project, but from an information management point of view, they're two different pieces of information with mutually exclusive metadata properties.
More broadly, ideally you talk to other people and find out where THEIR 80% 'stuff I tried that doesn't work but I'd like to keep around because the tries are valuable in case I re-visit these angles' information lives and collect it all together and presented in a way where people know going in that they're looking at historical information for the sake of current decision making.
On the other hand if you have good version control over the wiki, and a way to search the version history as well, kill the zombies.
Just like killing zombies in your code, killing zombies in documentation sounds good. https://www.bitnative.com/2012/10/22/kill-the-zombies-in-you...
Ultimately, most people are digital maximalists when they'd be better off being digital minimalists. I (in my personal life and in the small teams I work with) purposefully delete things that I do not wish to accept the burden of owning. Either something is important enough to justify the burden of ensuring it's accurate, or it isn't and it is deleted.
I have found that thinking about documentation in these terms, it becomes clear what to document and what not to document, and it forces better practices elsewhere in the business -- like writing code that clearly communicates the why, or designing business processes that leverage a core set of business information.
Respective team meetings, it's a agenda to talk about what needs to be updated, after updating a draft will be sent and later uploaded in the portal.
- Often, teams "own" stuff but the real knowledge is stuck inside individuals. This type of thing would need to survive the relentless reorg cycle that seems ubiquitous in tech companies.
- The health sector seems to have longer overall tenures than the tech sector. At the very least this kind of system should somehow link into the HR administration, so that if a document owner leaves there can be an alert "This document is now unowned!". Of course, that alert also needs someone to monitor that.
- What would you use as a metric for the asset-ness of knowledge? Some types of knowledge seem more important than others, but it is not clear to me how to quantify that.
Your point about digital minimalism is very good though. A lot of things that should be deleted are instead kept around in a form of digital hoarding behavior.
Every document is either "live" or a "snapshot".
Live documents have an expiry.
Documents are snapshots by default.
Knowledge is something you consume, digest and then use to inform thinking. New knowledge is built on old, and business outputs can be traced back through the knowledge that shaped them. Documentation, on the other hand, is a statement of fact(s) that are applicable only in a specific context (often temporal) that do not contribute to the evolution of a business.
Liabilities aren't inherently negative, they're a valuable tool in the right circumstance: a piece of documentation written for a customer support agent describing "how to issue a refund" has value but it can't be leveraged to grow the business, it can't be built upon, and it must be maintained. The business then must make a tradeoff between investing in knowledge or taking on liabilities.
If we were to represent this visually, knowledge would be the branches of a tree that other branches grow from, and documentation is a dead end.
Most of the documentation I've encountered in my professional life has been written to meet an arbitrary requirement for there to be "documentation" about an output, and so it's a rushed recital of the bare-minimum facts about a thing that exists. Nothing can grow from it. Whereas, when I've encountered it, the knowledge used along the way has been very valuable. A business can create an asset by capturing everything learned, and leverage that asset to inform the next decision, and so on and so forth. After all, that's what individual employees are doing already, it's just happening siloed in their heads.
Returning to the customer support example: years of knowledge collected about our business might teach us that our product's sizing is unique in the marketplace and many customers end up refunding their first purchase to order a different size. If issuing a refund takes 5 minutes per customer, and if all we have is documentation describing the 10 step process, a request from stakeholders to improve customer support efficiency might manifest itself as software development work to reduce issuing a refund to a 5 step process... but knowledge about the business would inform us that what customers really want is to ability to find the right size that fits. Rather than make a documented process easier, we might implement a purchase option where we charge for 1 item, send out 2 sizes, and the customer returns the one they don't want to keep. If all our business knowledge is captured, and not siloed, then a new employee should be able to theorise that, not only an employee who has been around from the start.
I'm not sure how confident I am in the way I've framed this, and using the word "documentation" seems like a risk because "documentation" means "everything written down" to most people, but I appreciate your question because it was interesting to think about: maybe someday I'll have more clarity in my thinking on this.
"Only Bob Jones knows the rest of this story. Call him at x1234 to ask him about it."
Speaking as a technical writer who has worked deeply with engineering teams. You might be "missing the forest for the trees" here a bit by focusing on documentation technology rather than engineering culture. Your team doesn't feel compelled or rewarded to create documentation. Go to your head of engineering, CEO, etc., share the story of losing critical company knowledge when these people left, and suggest to them that the team needs to be properly incentivized to create docs. Make documentation a factor in promotion, bonuses, social recognition, etc.
The technology is more likely to take care of itself after that. Rather, you'll have more engagement from stakeholders and will have more confidence around what will work.
But beyond that, in terms of wrapping your head around information architecture, Divio's documentation system is a great start: https://documentation.divio.com/
You can hire or find technical writers to give brown bag lunches on documentation basics. I would do it for you. Poke around on my HN profile and you'll find how to contact me.
Last, I would set the expectation that it's OK to feel like you're winging the documentation. I suspect a lot of engineers don't write docs because they feel they are bad writers and don't know what they're doing. Just focus on creating a safe space to creating docs and keeping them up-to-date first. Over time as you all use each other's docs you will figure out why they suck.
The Write The Docs conference videos on YouTube are also an excellent resource.
Each page has an owner, a contact, and an expiry date. Owner is responsible for keeping it up to date, and if you see a page that is expired you can ping the owner (unfortunately we don't have "alert on expired page" yet). If the owner is unavailable, the backup should be able to answer any queries/update the page, and if they can't then the failure of that lies on the page owner. There are a few pages that have empty backups, by virtue of our company size, but all the critical stuff has an owner, backup and is checked once every month or two.
We tend to keep more ephemeral project management stuff in our tracker or google docs and more long-lived content in our Notion tool. I'd be curious how your team balances that.
But it's definitely better than no system at all!
Documentation management and information architecture planning is work, and properly planned your maintenance will be minimal (NOT non-existent, but minimal) because the system will be designed to be as frictionless as possible.
But nobody wants to either pay for that or accept the time it takes.
This is true of any system of knowledge sharing, even if it's in-person-watercooler chat. The hard part is getting people to do it, no matter what the system is.
> This becomes a problem if multiple people leave at once or someone is very busy when expiration comes up and rubber stamps the renewal with outdated information...
The only solve for multiple people leaving at once is to ensure that you have all of their knowledge documented ahead of time, and if you have that, you don't need anything like this. A tool or a process can't solve people problems. For outdated information, this is no worse than the confluence abyss that I've experienced in previous jobs.
We have a document controller who's job (not her entire job, but say 20% of her job) is just to ensure that documents are organised correctly, adhere to guidelines and are actually reviewed and updated when they are supposed to be (I.e. she will come and nag you if a document is overdue and you have been ignoring your notifications).
As part of the ordinary rhythm of business, metrics on how well we are doing on the document control front are produced and reviewed roughly every two months by a senior manager in a quality team meeting. If you don't keep up to date with your document control tasks it will be brought up in your appraisal (in a reasonable fashion, you might just have too many docs assigned to you).
There's other stuff, but let's just say document management is quite important to that business because procedures matter when getting them wrong kills people and / or leaves smoking holes in the ground.
Importantly the process of writing a doc is dead easy and anyone (literally, some of the most useful are written by the admin assistants and lab techs) can create one at any time. Theres a standard template and style guide, you write your doc, you choose an approval pathway, classify it and then submit it for approval. While it's waiting to be approved it sits in the drafts register with a warning on it saying not yet approved, but anyone can still access it.
In practice what happens is rather than writing a long email explaining how to do something you'll just write a doc, stick it in the QMS and give someone a link. Then when someone else needs to know that they can either find it themselves or you can just point them at it.
The admin overhead of the whole system is surprisingly small and means that we actually do have up to date documentation on most things.
If only other industries took matters as seriously, then an in-house writing culture would (should!) improve dramatically. Sounds like a necessary, yet still insufficient, requirement.
Your wiki is always destined to fail if you are never going to use it as part of your daily ops.
This is why our internal wiki with "Clickup" is working OK thus far. We use clickup because it allows us to create tasks from within our wiki and reference docs in our clickup wiki, and link the latter to tasks or even email chains. As long as someone organized is handling the doc repo and the wikis, its quite good.
In summary. Its great for meeting agendas and opening epics or tasks during the meeting so there's accountability (no GDocs). Its great for doc repo that is "alive" to replace Gdrive. Its great to start customer comms (email outbound that threads inside Clickup) within tasks that themselves are referencing docs in clickup. You could also have a true HR / company policy wiki if you wanted, but we are too small for that. Instead we effectively have an OPS wiki. I think Clickup has found a great product market fit within the B2B space. It may even be great for B2C too.
Full disclosure. We are a customer. We make no money from them in any way.
I also bet that Clickup will eat Notion unless Notion does something dramatic.
I think we have been structuring documentation ineffectively for a while. GainKnowHow structures knowledge by prerequisites in a Knowledge Web, which I think is the best way to figure out how something works.
My goal is to let you "program" your organization through skills in a Knowledge Web that represents how your organization runs. You can write tests to see if employees actually understand their skills and there are change management features that tell the relevant employees when a skill has changed through skill diffs.
Here's an example of how to train your software engineers https://gainknowhow.com/software-companies.html
We've been using a mix of tools until now, Guru, Google Docs... but have lacked discoverability, powerful search, and in Google the ability to create sites for teams, etc.
Personally I use Obsidian and the files are sync'd via Syncthing.
What I wish would exist: A markdown driven wiki with Git (or something like it) in the backend so that you can clone the info locally, or refer to the master version, and even stage big changes to an area, etc. - basically something halfway between Obsidian and Confluence I guess.
To serve your md files as a traditional wiki in browsers, there's a git backed wiki named Gollum that also uses md files in a basic folder structure. https://github.com/gollum/gollum You can see where I'm going with this.
Gollum doesn't have user authentication or anything fancy, it just renders and edits md files. I tried it. There didn't seem to be a difference between Obsidian's and Gollum's markdown. When I committed my entire Obsidian vault to a git repo, I could still choose to have Gollum serve the entire vault, or just a subdirectory in the repo. I could also disable all editing in Gollum.
While Obsidian is working directly with the md files, Gollum doesn't update until I actually commit the changes. Obsidian is basically an IDE for my wiki now.
I was mostly satisfied with Joplin syncing to OneDrive prior to today's experiment. But now I think Obsidian + Git + Gollum deserves a closer look. It might be a bit overkill for my personal wiki, but it could work in a team setting if everyone works on the wiki like they would a normal git project.
As for you ask, the best solution that I've found, is to let people document things on Github and index those documents in another tool. That way there is no duplication, you let people use the tool they know, can use Markdown, but still benefit from searching, assigning owner, up to date date, rights ... That's how we do it at Dokkument
[1] - https://www.athensresearch.org/
Obviously it's not user-friendly at all. Only people who know how to use git can use it, which isn't great for collaboration with non-techie folks. Ideally, I'd build a little editor widget that could be embedded in the page...
Your users can use any Markdown editor that they like. But one person should be responsible for creating new documents, so that there will be consistency in naming and placing in the hierarchy. Naming things is hard, leave that part up to the Knowledge Base owner.
Well, that's against the philosophy of a wiki. Also, in a small team, nobody is the "knowledgebase owner" - he's just another developer. Putting a human in the workflow creates friction. But I'm totally with you on the "naming things" problem.
A full-blown corporate knowledge management system is a repository you can throw anything into - invoices, emails, memos, reports, etc.; the system will automatically generate a knowledge taxonomy and article summaries. But such systems require much more maintenance effort than something simple like a wiki, and are overkill for a small team of developers.