Readit News logoReadit News
ryan-duve · 2 years ago
> Ideally this diagram would be automatically generated using tooling...

The biggest problem I've seen with architecture diagrams is they fall out of sync with the code base. In my opinion, automatic generation of these diagrams is necessary. Otherwise, teams have no way to know whether the picture in front of them accurately represents the latest state of the system.

layer8 · 2 years ago
Diagrams are similar to textual documentation: You generally can’t auto-generate useful ones from code, unless the code has extra markup that specifies what to generate. Diagrams often present a specific perspective that emphasizes certain features while omitting others. You might have several diagrams for the same entity, each illustrating a different aspect or scenario.

The upshot is that diagrams have to be maintained in conjunction with the code and application architecture, just like textual documentation. There’s just no alternative to simply putting the work in, and making it a regular part of change management.

simpaticoder · 2 years ago
The correct solution is a synthesis: generate the boxes and arrows from code. Then let a human hide, move, and style those objects. When the code changes, the boxes and arrows will change, and perhaps the style will want to change, but at least the diagram will remain correct. This is precisely the distinction between semantic markup and css, btw. Implying that the html would be auto generated, the css would be hand crafted.
mumblemumble · 2 years ago
Agreed. I experimented with autogenerating C4 diagrams from source a while back, but quickly abandoned the project when I realized that the output was inevitably the flowchart equivalent of

  bool mystery_func(int i) { // define a new function
    int x = (int)(i / 2);    // x is i divided by 2 and truncated
    int y = x * 2;           // y is twice x
    return y == i;           // return true if y == i
  }                          // end of function

mpweiher · 2 years ago
> You generally can’t auto-generate useful [architecture diagrams] from code,

The question is: why is this the case?

IMNSHO, the reason is that we don't have a way to express architecture in or as code. Instead, we have to compile the actual architecture of the system into one that is expressible using the call/return architectural style that our programming languages support.

That's a lossy process.

When we can program with actual architectural connectors, auto-generating useful architectural diagrams becomes trivial.

https://objective.st

rafaelmn · 2 years ago
I use diagrams for two distinct use cases :

- planning/outlining a solution

- documenting/insight into the system

I disagree that second can't be autogenerated - I've used class diagrams, database schema diagrams etc. to visualize projects, a lot of the time over the documentation - precisely because I can trust the generated diagrams to reflect current state.

Documentation is nice for context but I'd take good visualisation tools over most documentation I've seen on projects I've worked on.

Tooling to connect/validate documentation against code is non-existent, in my world at least. Maybe LLMs can change that down the line - have PR review against docs run as a part of CI/CD pipeline.

barrysteve · 2 years ago
You can generate it from code.

Diagrams are a flat map of someone's subjective and structured, interpretation of the code architecture.

Words displayed on a computer screen must be developed into supporting subjective knowledge structures.. free from the usual objective mindset of engineering.

You have to go up to first order concepts to get it. The vast majority of (popular) programming culture generates more of the same. Everybody tries to paper over this truth with metadata and it metaphysically does not work.

Computers are capable of much more, they are politically limited down to a small subset of what's possible.

bdg · 2 years ago
The code-model gap is why we don't have this.

You don't organize code the way you mentally model it in many projects, and nearly all languages lack a way to solve this. Annotating code is prone to the same issue as keeping a diagram up to date, and the same issue as keeping comments or documentation up to date.

codeflo · 2 years ago
> nearly all languages lack a way to solve this

The only thing I’ve seen that goes in this direction is Knuth’s literate programming. I’ve tried it. In its current form it’s still clumsy, lacks tool support and IMO doesn’t fully solve the problem of how to deal with documenting a changing piece of software yet. Knuth got his requirements correct on the first try; the rest of us aren’t so lucky.

hcks · 2 years ago
In my experience diagrams that are pedestrian enough to be automatically generated from the codebase don’t add much value

Dead Comment

rewmie · 2 years ago
> The biggest problem I've seen with architecture diagrams is they fall out of sync with the code base. In my opinion, automatic generation of these diagrams is necessary.

Architecture diagrams document how the software is expected to be organized. They represent the goal, not the current state. The code needs to comply with the diagram, and not the other way around.

The only scenario where it makes sense to generate diagrams from code is when we have people trying to onboard to a project that's not documented, and even then these diagrams are only generated once, polished to remove noise, and from that point onward serve as the reference.

hobofan · 2 years ago
So how would you expect to insight on whether the current code differs from the planned design documents? By always applying a lot of manual human labor?
taeric · 2 years ago
On the contrary, I think auto docs are robbing the team of the ability to think in terms of the higher level of abstraction.

High level diagrams should be disposable and rapid to generate. They are as important for what they omit as for what they show.

Waterluvian · 2 years ago
Some problems are people or process problems and can’t always be waved away with tools. Sometimes the answer is to enforce growth of the professional discipline to update documentation alongside code changes.

A way I addressed this was to add a checklist item automatically to PRs, “did you review and update the docs?” And put the docs in the same repo so that a code change will have documentation updates in the same PR. It’s mostly worked but still relies on discipline.

It’s kind of interesting how hard this is for some. The code change is 5 mins. Testing is 20. Documentation is another 10. I’ve seen lots of people not want to do the testing and really not want to do the documentation.

photonbeam · 2 years ago
Ive seen testing reluctantly tied to “they’ll make me change everything in code review, so why waste the effort yet”
BerislavLopac · 2 years ago
The idea of C4 is to document the higher-level elements of the system - applications, components and the like. The first three Cs stand for "context", "container" and "component", while the fourth level, "code" is deemed optional.

IcePanel [0], a great tool for building C4 documentation, renames "containers" to "applications" for clarity, and instead of code diagrams simply links to the corresponding repos.

[0] https://icepanel.io/

gonzo41 · 2 years ago
You can't really make complexity go away. It just get's moved about. Auto creating diagrams will either mean specifying a new code artifact that will need to be kept up to date, and or create dependencies that will themselves fall out of sink with the code base. Or they'll be really simplistic and useless.

I think the best way to document a system is to write doco and just specifyc the intent of the system. What was this thing meant to do. That context is really useful for contrasting with the use of the system in a prod environment.

abhishekjha · 2 years ago
Why is it that compilers don't do this? They have a parse tree for how the sysmbols connect.

Would it not be appropriate to extend the compiler for visualising relationships between software components with zoom-in and zoom-out facilities. Zoom-in takes you to Assembly and zoom-out to the CTO.

nonameiguess · 2 years ago
I think comments like these are too parochial in scope. Note the first actual example here, which is the system context. In this example, it describes the relationships between a banking customer, an Internet banking system, a backend backend banking mainframe, and an e-mail server.

Yes, your software may explicitly model all of these system components and potentially you can generate a system model from the code, but that would an entirely wrong approach. As a sibling comment says, this system context view describes the real world, not your software. The code is supposed to conform to the model, not the other way around. If the implementation has drifted to not be in-sync with the model, there are a few reasons this may happen:

- The legal or regulatory landscape actually changed. In this case, yes, the code may be more up-to-date and you need to change the model.

- External components your organization doesn't directly control changed. In this case, also, it may be the model that is wrong.

- The model is right and your code needs to change. Maybe you are not correctly handling an external third-party API. Maybe you're not correctly meeting your customer's needs. In the worst case, maybe you're breaking a law.

I would also think the reality at something as expansive as a bank, there is no such thing as the codebase. You don't have a single product. You have the backend data store and transaction processing system. You have kiosk software for your ATMs. You have workstation software for your tellers. You have a public-facing website for your customers. You have a mobile app. You may have an entirely separate set of insurance products, investment products, and so on. You have internal management and accounting system for generating reports. Most likely all of these need to be separate system, at least because one temporally predates others. In part because a bank is formed by mergers, acquisitions, and divestments, so some products may have originally been part of a totally separate organization and some may be destined to be their own totally separate organizations. Strategically as a company, you can't afford to give up that level of financial agility by creating hard software-level couplings between your entire product suite.

So sure, at the level of any single component, you may be able to autogenerate a high-level architecture diagram. But at the level of the entire system, you can't. This is probably most clear and obvious with something like the DODAF: https://dodcio.defense.gov/Library/DoD-Architecture-Framewor...

These are much-maligned and for good reason. They're often incomprehensible. But to the extent you're trying to model something like the operation of a war campaign, you're now involving:

- C2 systems for multiple branches of the military

- ISR systems for those same branches

- Communications systems

- Operational capabilities of all of the various intelligence agencies and foreign allies you interoperate with

- Weapons systems

- Tablet terminal, man-pack, and in-vehicle devices for your forward tactical elements

All of these are software systems, but they're developed on different cadences, by separate contractors on separate contracts, with separate fiscal appropriations bills and lines of accounting. Nonetheless, there is still a need at the strategic level to model the entire system. In order for this system to have any hope of working, it needs to be based on specifications with the expectation being that implementations will conform to the spec, not the other way around. It's more like developing the Internet than developing a web app. You can't autogenerate a diagram of the Internet, at least not one with any authority, by pointing it at the code for a server, a browser, an endpoint networking stack, and the networking stacks for various appliances like core routers, and figuring out some way to link those together, especially given you'd have to cut across arbitrarily many programming languages and code styles.

donutshop · 2 years ago
I think what Adam Jacob is doing with system initiative addresses this, right off the get go.
mikehollinger · 2 years ago
I like C4. I'd previously discounted the "context" diagrams that inevitably show "User -{do their job}-> System-of-Record" as useless. Now I see the point.

I'm working through a new piece of system design for a complex system using C4 with our extended teams.

In drawing the top level this time around though I realized that b/c we have a semi-disconnected mobile app (aside from a web UI) we need to degrade certain behaviors gracefully if we don't have certain results yet, either b/c we're offline, or the backend is just slow for some reason.

That rippled down into the next layer of the system as a job queue, which then rippled into the UX, because we can now say definitively which parts of the system may take an unbounded amount of time to complete a request.

I'm sure we'd have gotten to this conclusion eventually but the analysis put the issue front and center.

All because I had to draw a picture with "Remote-User --{Mobile Stuff}--> Backend."

Also whichever tool you use is up to you. I just use draw.io for example with different tabs for each layer. As we elaborate on this at some point we'll break the lower layers out to their own files.

discodachshund · 2 years ago
I get the points about not needing a formalised system for diagrams, even if C4 is quite light touch in my opinion (I never go down to the code level).

However, I do have to credit structurizr, which outputs C4 diagrams, as a great productivity boost. One model producing multiple views at all specificities is fantastic compared to mermaid, plantuml, etc, where every diagram needs to be self contained and define and redefine the same components over and over. Branching off, editing the model, presenting ideas, has been a very successful workflow for me.

williamdclt · 2 years ago
I’ve only tried structirizr briefly, but I found it too inflexible. No way to draw concepts that aren’t strictly part of C4. For example I wanted to encapsulate 2 components in a box to represent that they’re currently deployed as one service (not necessarily a good thing to do in absolute but would have helped my team to grok it better): impossible.

I do like the idea of describing architecture formally, but the lack of customizability of diagram output was too painful

simon_brown · 2 years ago
I don't have the full context of what you're trying to model, but "two components deployed as one service" might be better represented as a container. If that's not the case, you can use the "group" concept to group components together -> https://docs.structurizr.com/dsl/cookbook/groups/

Alternatively, https://likec4.dev provides a way to create an arbitrary number of abstraction levels (although I wouldn't recommend such an approach).

jhund · 2 years ago
Plus one for structurizr. Its model driven approach makes it a lot easier to keep the diagrams consistent with the code: You describe the architecture, and structurizr renders the diagrams.
phlakaton · 2 years ago
Mermaid's definitely not well-suited for the amount of text and text formatting that C4 requires. You can do it, but you will be jumping through hoops, and the autolayout breaks down pretty quickly when the arrows between boxes get chatty.
gengstrand · 2 years ago
True but the big selling point for mermaid is the github integration and you can audit the changes to the diagrams via git log -p
wokwokwok · 2 years ago
I dunno, I kind of subscribe to software architecture being like, a set of design decisions that guide the implementation. That's what (in my experience) most software architects do; lay down guidance and structure for the software engineers.

> The C4 model was created as a way to help software development teams describe and communicate software architecture, both during up-front design sessions and when retrospectively documenting an existing codebase.

> It's a way to create maps of your code, at various levels of detail, in the same way you would use something like Google Maps to zoom in and out of an area you are interested in.

This seems different.

I have no idea why you would refer to the a code diagram as your software architecture. That's literally the code level. How is that architectural?

It's like saying the circuit diagram in the plug should go on the house blueprint. "You should use automated tools for this" ... so, it's for documenting existing code bases at the per-function level?

How is that useful for architecting / designing / planning software?

That sounds like software structure to me, not software architecture.

Sure, a map of existing software that explains how it's structured sounds cool... but I dunno. Like, if you're talking about design patterns, you're not gonna give someone a function-by-function map of how to implement a singleton. They're not stupid. You'd tell them you think it should have a singleton (or repository, or whatever).

Software design is totally a thing, and this seems entirely reasonable for designing software.

...but software design and implementation and software architecture are not the same thing and they're not done by the same people, in my experience.

This feels more like... systems design, which a software architect would contribute advice to in the way that the systems were designed so they aligned with good software architecture principals.

Maybe I'm just being pedantic. /shrug

onetimeuse92304 · 2 years ago
> I dunno, I kind of subscribe to software architecture being like, a set of design decisions that guide the implementation.

I think this is the one mandatory part of software architecture which is giving developers information to help them make their own design decisions without constantly deferring to the authority.

But the exact place where architecture ends and development starts varies a lot.

I like to think that architecture itself decides what is important from the architecture standpoint. If architecture decides a certain low level application detail is important, then it becomes an architecture detail.

In fact, in most organisation architecture controls at the very least some top level design like components, communication patterns, APIs and technology in use.

In some organisation architecture goes as far as individual classes. Not all classes, but maybe classes modelling the domain of the problem especially, if that model is used as a language for multiple project or even implemented as a shared artifact.

I think C4 is suitable for those organisations where architecture is concerned with more low level structure than just listing applications and their integration interfaces.

datadrivenangel · 2 years ago
C4 for housing would be:

* Have a high level blueprint of the house for typical humans.

* Have a detailed blueprint of the house for contractors.

* Have a circuit diagram for the utility systems like electricity

* (Optional) Have a detailed specification for every plug and fixture etc.

Basically, have a diagram for every level that will be notably unique or useful to have for the system. If you need a code level diagram, either you're doing some complicated custom work or most likely over specifying the design that can be left as an implementation detail.

dexterbt1 · 2 years ago
Interesting, as I kinda share also that there are lines to be drawn between software architecture, design and implementation (all three).

Much like in the real world building of real-estate for example, I see there are different roles between the architect vs the engineers vs. the foremen.

What do you draw the line? What concerns/areas are to be covered in each of these [arch vs design vs implem]?

maximinus_thrax · 2 years ago
> I kind of subscribe to software architecture being like, a set of design decisions that guide the implementation

Do you mean software architecture is such as a set of design decisions? Or that software architecture is a set of design decisions?

chrisweekly · 2 years ago
Excellent points.

(also, not too pedantic)

(also, principals -> principles)

IshKebab · 2 years ago
Interesting but I don't think this especially adds anything. The high level diagrams are great but then you can already make those without having to read any fancy visual model websites. The site even kind of admits this:

> As an industry, we do have the Unified Modeling Language (UML), ArchiMate and SysML, but asking whether these provide an effective way to communicate software architecture is often irrelevant because many teams have already thrown them out in favour of much simpler "boxes and lines" diagrams.

This is very true, but then they seem to have ignored the reasons why UML class diagrams were a total failure. They somehow acknowledge that nobody likes them, but then the 4th C of their model (code) is... UML class diagrams!

You don't need this. Just do ad-hoc diagrams and make them clear, with labelled arrows. Their first three diagrams are actually good examples of this.

sgt · 2 years ago
That's what I do. Boxes and lines between them. It helps a lot when thinking up new concepts and brainstorming too, so that the tool doesn't slow you down.

I use Freeform now, which is super simple but works. Mark two objects and cmd-click/right click, then click "add connecting line".

almostnormal · 2 years ago
One step further: Powerpoint. Sooner or later the diagrams will usually be needed in a presentation. Having them in a presentable format saves another step, and is a constant reminder to produce something worth presenting.

Those diagrams that don't end up in presentations will be write-only anyway, and could be in any random format. No-one is going to read them.

wejick · 2 years ago
Is it Freeform as the Apple Freeform? I tried it last week to create high Level design, still not as smooth as I expected. Mainly the text editing flow feels not seamless yet, the deal breaker was I couldn't get the file successfully shared thru any available options.

To be fair it's still in its early of its time, I'll wait bit longer.

pjmlp · 2 years ago
UML is still highly used at the architecture command deck, even if the dream to generate code from diagrams failed.

Just a couple of months ago I finalized yet another archictecture document, with enough class diagrams, sequence diagrams and use cases on it.

nitwit005 · 2 years ago
Try testing if people can comprehend the diagrams by asking a question after showing them a somewhat complicated one. You'll likely be disappointed. We don't seem to be using them because they work.
captaincaveman · 2 years ago
In practice this doesn't work very well in an enterprise, the boxes end up a mix bag of different levels of things, some abstract some concrete, some just random.

So if you need to piece together larger chunks of architecture, a supply chain for example, you basically have to redo it all. There is a lot of waste in this, and no one has a sufficiently clear view of the overall architecture, which leads to other waste and quality issues.

For some reasons most engineers get very defensive about a more formal approach, I suspect mostly because it doesn't immediately benefit them or they haven't had too deal with large scale architecture and don't see the problem.

bdg · 2 years ago
Having structured architecture conventions becomes more important when you have many architecture conversations across multiple people and would like to gain insights from the diagrams too.
IshKebab · 2 years ago
But you don't need and official standard that everyone learns. Just use words. If your box is a class, write "class Foo". If your arrow means "contains 1 or more" just write that next to the arrow.
nickdothutton · 2 years ago
This is how you know software engineering is still an immature discipline. It’s 25+ years (in my case) as still we are having largely the same discussions about “basic” things. I’m not saying this is bad, but progress is slow, and we can’t blame any particular company or consortia or commercial interest here.
bdg · 2 years ago
Honestly we have the same problem outside of engineering. C4 is really just a way to focus conversations along one level of abstraction.

How many times have you talked to someone who bounces up and down from high detail high complexity to zoomed-out simplicity in the same thought, and not understood what they wanted to tell you? We have the same issue when devs think "I will just yolo some boxes and arrows on this board, it makes sense in my head so it is a good diagram!"

wholesomepotato · 2 years ago
Even as a non civil engineer I can understand floor plans and alikes and find then immediately useful. Even as a professional SWE these example diagrams don't seem useful or informative enough to justify bothering with them.

Code is too multidimensional to be approximated with 2d drawings. It just doesn't work.

What could work are more conventional architectural code patterns. Things like structured concurrency, but also for higher level. Maybe.

ozim · 2 years ago
Mostly because software architecture diagrams need a lot of context or have to be in specific context and quite tight scope.

Floor plan has really fixed context and very limited scope. Like I don't have to explain windows ... but in software most of the time you actually have to explain "window" because every "window" or element will be different, you cannot just say this is window and copy it 20x.

Veuxdo · 2 years ago
If you find the C4 model to be too arbitrary and over-specified, I wrote on an alternative practice earlier this year: https://www.ilograph.com/blog/posts/concrete-diagramming-mod...