As others have said the C4 model is a great way to address a number of these issues.
I can’t find the right video at the moment but Simon Brown (creator of C4) gives a great talk about creating his DSL, Structurizr, for C4, which he developed during COVID lockdown (if memory serves). There are many videos on YouTube of Simon talking about “C4 Models as Code” so I’m sure any one of those will suffice.
The focus is on creating the model of your system architecture, from which the diagrams you extract are a specific projection of that model. Rather than a diagram be the main artifact. It’s a simple but very powerful concept that I’m always surprised isn’t more widely used.
Structurizr models can also be exported to display as ilograph diagrams, mermaid diagrams and more.
Also very much worth a mention is icepanel, a lovely tool for architectural model that implements the C4 model heavily.
I saw Simon talk at a conference in Sydney about 10-15 years ago and heard about C4 for the first time in that talk, it’s been one of the most influential talks I’ve been to in my career as it made a lot of fuzzy things in my head all start to come together in a way that just made sense.
I’ve never seen modeling tools widely used. When it’s attempted it’s always a top-down initiative that always addresses management concerns, but always creates more risk for developers without generating value for them, and it never takes off outside of being a box on a process list to check off. Developers promptly stop using these tools as soon as their use is no longer enforced.
The long term destiny of models is that they will always diverge from the code, sometimes in ways that matter. In that situation, the best model of the code is the code it’s self because it’s a model that you can rely on. You can certainly update the diagrams, but that’s a priority call vs. what else you’re doing, and it doesn’t always win out over other tasks.
Some enterprising tool developers will make modeling tools that will generate code, therefore linking the diagrams and the model to the development of the code. When this is employed, it’s complex and it’s never a perfect implementation. Generated code can be buggy, or it may not do what you want. It may be a significant effort to make the models generate the code that is needed, and you may find your self further developing the code generation tools to generate the code you want. I’ve never seen this have a good benefit to cost ratio.
The problem with any of these tools is that they solve only one part of the puzzle. Take Structurizr for example, it doesn't automatically create the diagrams for you or notify you when it detects architectural drift (and automatically update the diagram).
Others miss other pieces of the puzzle, such as having a list all your APIs, all your system docs in a single place (ADRs, reqs, etc.), connecting to your repos, etc.
> Take Structurizr for example, it doesn't automatically create the diagrams for you
The Structurizr DSL is designed for manual authoring (which is what most people tend to do), but there's nothing preventing you from writing some code (using one of the many open source Structurizr compatible libraries) to reverse-engineer parts of the software architecture model from source code, binaries, your deployment environment, logs, etc.
> or notify you when it detects architectural drift
If you do the above, there's then nothing preventing you from writing some tests to notify of architectural drift, etc.
* Devs: We need a better language than Javascript as the lingua franca of the web!
* Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web.
Keep in mind that the consumers of technical diagrams are often non-technical folks. And they don't care about how they get their diagrams. They just want to be able to understand, at a high level, what's going on in the black box.
You can either convince every single one of them that devs need to focus on better system design tools ... or you can continue to give them the diagrams they want, just using a smarter process to generate them.
Or you can treat them as entirely separate problems, because fundamentally system design tools are building tools, and system diagrams are communication tools. In most cases you can improve them independently.
Exactly. Users don't care about the underlying tech as long as it works and works on the platforms they're using and with other systems they're using. That's it. When users start caring about the underlying tech, you need to actively dissuade them unless there is a sound technical argument. Otherwise you end up with people insisting on using DOS for enterprise applications in 2024 because that's what they've used since 1984 (wish I hadn't experienced this).
They are, however, implicitly saying it with the position that "I don't care as long as it works."
Javascript is available on every modern browser. It's not going away any time soon. Migrating to some other frontend language, no matter how demonstrably better it is than Javascript, is a Herculean effort. Actually, it's more of an Atlassian effort, since it would require practically an Earth-wide shift.
I'm not sure I've met a consumer of diagrams who ever really wanted to know what's going on, so much as they wanted to have their pre-conceptions validated for the brief time they're interested in the problem.
I have had too many conversations with managers who want to talk about "pipelines" but then you ask how the system is handling dropped or failed messages and get a deafening silence or "just don't worry about it it's rare".
People love arrows showing the "direction" data is going, but do things like assume that direction means "security" (it doesn't - any channel other then a full on data-diode is 2-way) or implies other operations (i.e. about storage).
Compiling to Javascript is a really low bar: all Turing-complete languages are equivalent, one can be translated to another. You could have a Lisp or Haskell compile to Javascript. The problem is: everyone has their favorite language, so such ecosystem would be fragmented, and Javascript is the lowest common denominator.
The article does indeed argue that we need smarter ways to create diagrams - so that devs don't have to manually create / update them and other teams get the info that they need.
Saving time and effort for devs, while making dynamic visualizations with different levels of detail (i.e. more and less technical for different audiences) is possible.
It's not so hard to do static and tracing-based analysis to capture all the calls that various systems make between themselves. Similarly, it's not so hard to graph all of the metrics of a system.
That's not really all that useful. Diagrams, like other forms of documentation, are a format for communicating something to the reader. It means that they should spend more space on the important flows rather than the exceptional ones. They'll explain the meaningful parts of a sequence rather than just giving a series of function calls. The various tools we have for doing this (boxes showing what systems talk to what, sequence diagrams, database schema diagrams) provide a rich enough language for that communication.
Death star diagrams are bad because they spend a bunch of time and effort to convey one piece of information: "there are a lot of systems here" rather than anything actually useful for someone attempting to navigate a specific part of the system.
I think at least part of the problem is that todays "agile" methodologies lure people into almost completely neglecting analysis and design, as if these steps are forbidden in agile. So for many dev teams I have seen, agile basically means "we have no plan".
I’d characterize it as the degradation of the product manager role. Too many project managers get promoted to product but don’t really know what that job entails so they just project manage harder.
And to a project manager “I’m gonna take a week and think about it” is not an acceptable answer.
I agree that many teams, in an effort to move away from waterfall development and Big Design Up Front, have gone the opposite way and completely skip system design. Which is a mistake, because you need some upfront design.
As Dave Thomas said: “big upfront design is dumb. No upfront design is dumber”.
I believe the answer is to have a single model of everything.
You can build the model from whatever: Draw it with a GUI, query your server infrastructure, analyze binaries, collect it from databases, parse it from documentation, process your issues, etc. The important points is to merge all the information into a single model.
You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions.
Does it exist today? Many tools and processes have been developed to achieve it. For example, "Model-based systems engineering" is a very formal approach. I have yet to see it realized in practice though.
I'm pretty sure it isn't a single tool or process though. By definition it integrates with nearly everything and that means a lot of customization. You won't get it get it off the shelf.
This is something I that’s been living in my head for a few years now—and no, it does not exist yet. I’m also convinced the only way to get it right is to have a single graph, modelling the entire system, and apply filters on the graph to get only the nodes you’re currently interested in, then have different diagrams produced from that as the output. How else would you describe the flow of packages from the internet through the firewall, what the logical network looks like, and which physical location things are located at? These questions all interfere with each other on a conceptual level, yet are all conceivable using different attributes of the same connected graph nodes.
It’s very complex, very interesting, and lots of work.
The problem with "a single model of everything" is that it is an inferior tool.
It's a drawing or diagramming tool, but the stand-alone diagramming tools are better.
It's a text composition tool, but word processors are better.
It's a code writing tool, but IDEs are better.
In every single thing it tries to do, a specialized tool is better. So this single tool needs to be close enough to the best in every category in order to not be a boat anchor holding you back. So far, nothing has come close.
Not sure, but IMO a dataflow diagram from good ol' SSADM got a lot of the way there. Unlike a lot of the modern techniques which model (groups of) deployed components, DFDs were strictly logical and functional and included data stores and users as well as functional components. So it was possible to model data in flow and at rest, and the characteristics of each, including where it crossed system boundaries.
IMO this was the best diagram format to get an overall view of the system, or reveal which areas needed further analysis.
It sounds to me like what you're describing is a single *tool*, not a single model. Is that possible?
I agree that we need multiple views, but it isn't just a matter of filtering part of a single view - it's also different fundamental primitives that are needed to communicate different kinds of ideas or aspects of a given system.
To me, this seems parallel to the evolution of LLMs into multi-modality by default. Different queries will need different combinations of text, heat maps, highlights, arrows, code snippets, flow charts, architecture diagrams, etc.
It's certainly not easy, but it's exciting to consider :)
> You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions.
There is Capella which is a tool (software) and method to design systems. It used the Arcadia method (mbse). It is quite extensible.
I don't understand how you can write an article about system design without mention to enterprise architect or Capella. I've read this as "we need to come up with a solution that already exists".
Almost all of his complaints stem from using generic drag-and-drop diagramming tools. A modern take on system diagramming, like Ilograph[0], solves most of these issues (IMBO).
There are quite a few tools cropping up trying to solve this problem. Multiplayer.app is one example - they use OTel to gather distributed traces from your system and ensure you automatically get notified when there's drift.
>Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams,
One issue is that diagrams in general are pretty universal. As long as your tool can makes shapes and connect them, you can use it for any kind of architecture. For example it will work flow charts for 50 year old Fortran to UML from the 90s to microservices diagrams from now and I bet to whatever is common 50 years from now.
>> >Today, we have the technology and knowledge to create tools that prevent developers from wasting valuable development time deciphering static, outdated diagrams,
"Outdated", imo, is subjective to the content and is bias'd by the time since creation.
As long as wireframed thought is the bare skeleton, the diagrams are merely a read-only user interface to communicate to a particular end-user audience.
I don't think the issue is with diagrams per se, but with how we create them. They are super helpful in conveying meaning but why do we need to create and update them manually?
Thank you everyone for reading my article! I’m the author, Thomas Johnson.
This article stems from my frustration with the typical approach of asking, “What diagramming tool should we use?” instead of addressing the root problem: the need for up-to-date, easily accessible system architecture information.
That’s why I co-founded Multiplayer. We focus on automating the creation and maintenance of system architecture diagrams and creating a single source of truth for system information. This includes your individual components, APIs, dependencies, and repositories.
We’re language and environment agnostic and you can start with a napkin sketch or a photo of your whiteboard. And this is just the start, we have many plans for how to evolve system design tooling including supporting popular integrations and models like C4.
If every acknowledgement of the(or any imo) issue at hand, was accompanied with a minimally viable solution (even if it doesn't work) -- as the defacto standard for providing critical feedback..
As long as some effort is put in towards a countering possible solution in response-- I truly feel that we would be closer to an end solution.
Check out https://www.multiplayer.app/
The author is one of the co-founders, but I understand not wanting to just promote his tool and just have a conversation about the problem.
Readit News
I can’t find the right video at the moment but Simon Brown (creator of C4) gives a great talk about creating his DSL, Structurizr, for C4, which he developed during COVID lockdown (if memory serves). There are many videos on YouTube of Simon talking about “C4 Models as Code” so I’m sure any one of those will suffice.
The focus is on creating the model of your system architecture, from which the diagrams you extract are a specific projection of that model. Rather than a diagram be the main artifact. It’s a simple but very powerful concept that I’m always surprised isn’t more widely used.
Structurizr models can also be exported to display as ilograph diagrams, mermaid diagrams and more. Also very much worth a mention is icepanel, a lovely tool for architectural model that implements the C4 model heavily.
I saw Simon talk at a conference in Sydney about 10-15 years ago and heard about C4 for the first time in that talk, it’s been one of the most influential talks I’ve been to in my career as it made a lot of fuzzy things in my head all start to come together in a way that just made sense.
https://c4model.com/
https://structurizr.com/
https://www.ilograph.com/
https://mermaid.js.org/
https://icepanel.io/
I’ve never seen modeling tools widely used. When it’s attempted it’s always a top-down initiative that always addresses management concerns, but always creates more risk for developers without generating value for them, and it never takes off outside of being a box on a process list to check off. Developers promptly stop using these tools as soon as their use is no longer enforced.
The long term destiny of models is that they will always diverge from the code, sometimes in ways that matter. In that situation, the best model of the code is the code it’s self because it’s a model that you can rely on. You can certainly update the diagrams, but that’s a priority call vs. what else you’re doing, and it doesn’t always win out over other tasks.
Some enterprising tool developers will make modeling tools that will generate code, therefore linking the diagrams and the model to the development of the code. When this is employed, it’s complex and it’s never a perfect implementation. Generated code can be buggy, or it may not do what you want. It may be a significant effort to make the models generate the code that is needed, and you may find your self further developing the code generation tools to generate the code you want. I’ve never seen this have a good benefit to cost ratio.
It's pretty plug&play, using static analysis & LLMs we generate documentation + an interactive system diagram you can recursively explore.
https://vision.useadrenaline.com
All the auto-generated diagrams are exportable as mermaid
Is there no public demo?
Others miss other pieces of the puzzle, such as having a list all your APIs, all your system docs in a single place (ADRs, reqs, etc.), connecting to your repos, etc.
I recommend checking https://www.multiplayer.app/ as an alternative.
The Structurizr DSL is designed for manual authoring (which is what most people tend to do), but there's nothing preventing you from writing some code (using one of the many open source Structurizr compatible libraries) to reverse-engineer parts of the software architecture model from source code, binaries, your deployment environment, logs, etc.
> or notify you when it detects architectural drift
If you do the above, there's then nothing preventing you from writing some tests to notify of architectural drift, etc.
Dead Comment
* Devs: We need a better language than Javascript as the lingua franca of the web!
* Users: Sure, use whatever language you want -- just make sure it compiles to Javascript, which is already the lingua franca of the web.
Keep in mind that the consumers of technical diagrams are often non-technical folks. And they don't care about how they get their diagrams. They just want to be able to understand, at a high level, what's going on in the black box.
You can either convince every single one of them that devs need to focus on better system design tools ... or you can continue to give them the diagrams they want, just using a smarter process to generate them.
Or you can treat them as entirely separate problems, because fundamentally system design tools are building tools, and system diagrams are communication tools. In most cases you can improve them independently.
I don't think users are saying this and I don't think they care as long as it works.
They are, however, implicitly saying it with the position that "I don't care as long as it works."
Javascript is available on every modern browser. It's not going away any time soon. Migrating to some other frontend language, no matter how demonstrably better it is than Javascript, is a Herculean effort. Actually, it's more of an Atlassian effort, since it would require practically an Earth-wide shift.
I have had too many conversations with managers who want to talk about "pipelines" but then you ask how the system is handling dropped or failed messages and get a deafening silence or "just don't worry about it it's rare".
People love arrows showing the "direction" data is going, but do things like assume that direction means "security" (it doesn't - any channel other then a full on data-diode is 2-way) or implies other operations (i.e. about storage).
Users: Sure, use whatever presentation you want - just make sure it can be shown on a 2-dimensional screen, which the lingua franca of presentations.
Saving time and effort for devs, while making dynamic visualizations with different levels of detail (i.e. more and less technical for different audiences) is possible.
That's not really all that useful. Diagrams, like other forms of documentation, are a format for communicating something to the reader. It means that they should spend more space on the important flows rather than the exceptional ones. They'll explain the meaningful parts of a sequence rather than just giving a series of function calls. The various tools we have for doing this (boxes showing what systems talk to what, sequence diagrams, database schema diagrams) provide a rich enough language for that communication.
Death star diagrams are bad because they spend a bunch of time and effort to convey one piece of information: "there are a lot of systems here" rather than anything actually useful for someone attempting to navigate a specific part of the system.
And to a project manager “I’m gonna take a week and think about it” is not an acceptable answer.
As Dave Thomas said: “big upfront design is dumb. No upfront design is dumber”.
You can build the model from whatever: Draw it with a GUI, query your server infrastructure, analyze binaries, collect it from databases, parse it from documentation, process your issues, etc. The important points is to merge all the information into a single model.
You cannot represent this model on a screen in any meaningful way. Instead you can only get certain "views" into it to understand an aspect or answer specific questions.
Does it exist today? Many tools and processes have been developed to achieve it. For example, "Model-based systems engineering" is a very formal approach. I have yet to see it realized in practice though.
I'm pretty sure it isn't a single tool or process though. By definition it integrates with nearly everything and that means a lot of customization. You won't get it get it off the shelf.
It’s very complex, very interesting, and lots of work.
It's a drawing or diagramming tool, but the stand-alone diagramming tools are better.
It's a text composition tool, but word processors are better.
It's a code writing tool, but IDEs are better.
In every single thing it tries to do, a specialized tool is better. So this single tool needs to be close enough to the best in every category in order to not be a boat anchor holding you back. So far, nothing has come close.
Not sure, but IMO a dataflow diagram from good ol' SSADM got a lot of the way there. Unlike a lot of the modern techniques which model (groups of) deployed components, DFDs were strictly logical and functional and included data stores and users as well as functional components. So it was possible to model data in flow and at rest, and the characteristics of each, including where it crossed system boundaries.
IMO this was the best diagram format to get an overall view of the system, or reveal which areas needed further analysis.
I agree that we need multiple views, but it isn't just a matter of filtering part of a single view - it's also different fundamental primitives that are needed to communicate different kinds of ideas or aspects of a given system.
To me, this seems parallel to the evolution of LLMs into multi-modality by default. Different queries will need different combinations of text, heat maps, highlights, arrows, code snippets, flow charts, architecture diagrams, etc.
It's certainly not easy, but it's exciting to consider :)
Also the design tools for it were hilariously expensive and notoriously cumbersome.
It sounds like you're describing: https://www.ilograph.com/blog/posts/concrete-diagramming-mod...
I don't understand how you can write an article about system design without mention to enterprise architect or Capella. I've read this as "we need to come up with a solution that already exists".
[0] https://www.ilograph.com
https://c4model.com
One issue is that diagrams in general are pretty universal. As long as your tool can makes shapes and connect them, you can use it for any kind of architecture. For example it will work flow charts for 50 year old Fortran to UML from the 90s to microservices diagrams from now and I bet to whatever is common 50 years from now.
"Outdated", imo, is subjective to the content and is bias'd by the time since creation.
As long as wireframed thought is the bare skeleton, the diagrams are merely a read-only user interface to communicate to a particular end-user audience.
We’re language and environment agnostic and you can start with a napkin sketch or a photo of your whiteboard. And this is just the start, we have many plans for how to evolve system design tooling including supporting popular integrations and models like C4.
It’s early days for us. I’d love to hear what you think: https://www.multiplayer.app/ (It’s free!)
https://github.com/bokner/solverview
https://github.com/bokner/solverl
https://www.minizinc.org/
YMMV, as I've yet to personally try a toy project with this approach. =3
If every acknowledgement of the(or any imo) issue at hand, was accompanied with a minimally viable solution (even if it doesn't work) -- as the defacto standard for providing critical feedback..
As long as some effort is put in towards a countering possible solution in response-- I truly feel that we would be closer to an end solution.
I guess should this "problem" truly exist.
Dead Comment