I love this! It's the "jazz music" of software development. Something which breaks all the "rules" but does so purposefully and explicitly so that it can become better than the "rules" allow.
A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements, has a lot of "pointless comments" that just describe what the line or few lines around it is doing, and has a lot of "logic" in the comments which could quickly become outdated or wrong compared to the actual code.
Yet at the same time, it's probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files, it contains a lot of the inherently complex work it's doing to this file, and it is so well and heavily commented that it should be pretty easy to ensure that any changes also keep the comments up to date (after all, any change without changing the resulting comments should show up like a sore thumb, and will most likely prompt the reviewer to look into it at the very least).
> I love this! It's the "jazz music" of software development. Something which breaks all the "rules" but does so purposefully and explicitly so that it can become better than the "rules" allow.
I kind of see it as the opposite: “space shuttle style” is code that adheres to heavyweight rules that most software development has abandoned in favor of a more improvisational style.
But in either case it illustrates that code style rules are desirable, or not, based on the context; you need to understand what purpose rules serve and what trade-offs they involve to understand which rules to use; there's no one-size-fits-all solution.
I've had this latent thought for a while that I'm finally putting to words:
The complexity goes somewhere.
It's either into lots tests, or it's into something like shuttle style with lots of comments, or it's into a huge QA department, or it's into the type system / DB schema. It could even be going into the org structure!
But something, somewhere is handling the complexity and it is doing so as a partial function to the economic importance of the software to the stakeholders of the software.
> I kind of see it as the opposite: “space shuttle style” is code that adheres to heavyweight rules that most software development has abandoned in favor of a more improvisational style.
Most software other than that written for space shuttles and other seriously critical applications.
A colleague of mine works for a telco, and changes to software that runs on satellites takes months to approve and goes through verification stages that include running on simulators and duplicate hardware that is on the ground.
> probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files
I'm only halfway through John Ousterhout's book Philosophy of Software Design but I think it agrees with you on this -- that smallness-of-file or smallness-of-function is not a target to shoot for because it prevents the things you build from being deep. That you should strive to build modules which have deep functionality and small interfaces and should contain their complexity within them so the users don't have to know that complexity.
I just finished his book yesterday; he has a lot to say about size and comments. For size, your summary is spot-on. I'd only add that he notes overeager splitting of methods and classes makes code involved in a particular abstraction to be no longer in one place, leading developers to constantly jump around files, which makes it more difficult to understand the code and increases the chances of making bugs.
As for comments, this file is essentially Ousterhout taken to the extreme. Still, I think he would have like it, given how critical this file is. In the book, he encourages writing more comments than the current trends would suggest, pointing out that you can't fully express abstractions in code, so all the things the code doesn't contain - the high-level concepts, the rationale, the caveats - should be documented in comments in appropriate places.
Overall, I'm extremely impressed by the book, and its focus on reducing and mitigating complexity.
Now i'm squarely in frontend web-app development right now which definitely changes things (mainly the complexity is centered around enabling fast changes/additions to the codebase, and not the actual business logic for the most part), but while "deep functionality and small interfaces" sounds good on paper, most of the time giant files with a few functions exported aren't a good way to manage that.
Sure, it solves the problem when viewed from the outside. "users" of the software (users being other devs in this case) get a nice small interface and docs that explain how to use it, but internally it's much harder to work with. Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it.
In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. And it means that any devs that truly do understand it top-to-bottom would just waste a lot of time switching around files if it were split up.
Putting it all in the same file here nudges you to really make sure you understand it top to bottom before making any big changes. It's intimidating and scary for a reason, because at its core it is a complex piece of code, and dressing it up in "simple code's clothing" won't help.
> That you should strive to build modules which have deep functionality and small interfaces...
this. interfaces are not the most important thing in a software but definitely one of the most. you can live with a crappy implementation but your interfaces must cater to the usecase and should only hcange when core premises change.
Haven't read the book. Goes to my Wish List. I completely agree about encapsulation of complexity inside the module.
Similar to UX design principles. Just like google or other successful sites present a very simple interface (in case of google the home page is quite clean) but behind it lies very complex code.
> that smallness-of-file or smallness-of-function is not a target to shoot for
I disagree. Unless you are methodical and know what you are doing (like NASA or the authors of Kubernetes), it is hard to create large functions and files by keeping levels of detail consistent and not repeating code.
How do you test a function that has 100 unique outcomes? How do you safely maintain it to ensure it won't break? How do you even know it's working?
I agree. My day job is working on code that isn't this level of critical, but also has the characteristic of being low level, both closer to the metal than typical backend code and also called by so much frontend and backend code that if there was such a thing as "even backend-ier code" this would be a good example. If you miss a nuance, a horde of angry developers will show up at your desk the moment the build deploys, and if you don't fix it fast, that turns into a horde of angry customers.
With code that far down in the inner loops of mission critical code, the dominant time cost isn't really writing or even understanding the code, it's ramping up on the nuances of the scenarios you serve, checking the direct effects of your changes, and then checking the second order and third order effects of your changes.
When your code is so far down that every change you make has third order effects, it's a lot easier to maintain code that is meticulously commented and written for maximum thoroughness explicitly baked into the file in front of you, because you're much more likely to miss the nuances if you just finished piecing together ten different files to figure out how the happy path works.
Well said. I feel that 80% of this thread is people talking past each other, with different assumptions about what the code in question is going to do.
I completely agree. For code that is unavoidably complex, I love this style too.
I am all for code that is concise and whose syntax/naming is expressive, but sometimes comments are necessary to clearly spell out the logic or business use case. Expressive code can only go so far. Well-crafted comments significantly reduce the amount of time required for other developers to dive in and become productive with an unfamiliar code base.
The key is keeping the comments up to date. There's nothing worse than an inaccurate comment. One's code review process must include a review of the comments accompanying the modified lines.
Outdated comments that explain the business use case or purpose are still better than no comments. It gives you background information how the code evolved or what it was supposed to do.
It's probably because reading comments only is worse than reading code without comments, that some devs developed an aversion towards outdated comments and thus comments in general.
Comments are additional information and no source of truth, always take them as that and read code and comments.
About five years ago I worked on a codebase with a similar bit of code. It wasn't nearly this big, but it was branchy, procedural, and verbosely commented. I didn't write the initial version but worked on it quite a bit and learned to appreciate the advantages of the style for the nasty bit of logic it implemented. I ended up having to vigorously defend it against another developer's half-cocked attempt at "refactoring", which got deployed more than once (behind my back) and thankfully broke in pretty obvious ways each time.
I warned him at the beginning that he needed to spend a couple of hours wrapping his head around the code before trying to modify it.
I warned him that the code arrived at its current state after long and painful experience.
I warned him that other developers had worked on the code, had exactly his initial reaction, and eventually admitted they couldn't find a way to improve it.
I warned him, after he confidently declared that it just needed to be "more object-oriented," that I had written a lot of object-oriented code, was open to writing it in an object-oriented style if it would improve things, and could not think of any way to do so that would not make things worse.
But he could not even bring himself to read the existing code. He never did. He worked on it for three weeks, wrote thousands of lines of code, and even tried to deploy his own version without ever spending a single contiguous, focused, hour-long block of time reading the existing code. He was "refactoring" the whole time.
The code he produced was exactly how you imagine it. The code went from several hundred lines in a single file with no classes (just a containing class acting basically as a namespace) to thousands of lines scattered over half a dozen files with at least that many classes. The guy kept adding classes and kept adding test after test after test. He couldn't get his code to pass the test cases I had written, so several times he declared that my test cases were wrong and changed or removed assertions. He also claimed that the existing code couldn't "pass" his tests because he had hundreds of lines of tests for components that only existed in his code. According to him, this proved that there was a ton of "hidden untested functionality" in the existing code, which was therefore "dangerous."
I was pretty busy with other things, and this was his project now, so if he had been a little bit more clever he probably could have made his abomination stable enough to replace the existing version over my objections. Thankfully it was never good enough to survive in production.
Why do all anti oop posts sound like completely unlikely, exaggerated lies? A few hundred lines to thousands with dozens of files and classes (gasp). Lol
> A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements. A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements, has a lot of "pointless comments" that just describe what the line or few lines around it is doing, and has a lot of "logic" in the comments which could quickly become outdated or wrong compared to the actual code.
I actually understood it immediately when I read it. There's something to be said when you have a decent roadmap for debugging critical code segments. It's not just software maintenance here, it's for the "holy crap something went wrong and we can't understand how we got here!" moments.
This is why I prefer older JavaScript to the more recent stuff, with transpiration pipelines and AI-like optimizing compilers that gnaw on giant state trees.
That stuff is lovely and terse when it works. And when it doesn’t you are at a total loss.
The beauty of imperative code is there’s a beginning and an end, and at every step you can easily see what follows and without too much work what came before.
I don't understand the objection to having more, smaller files, at least in Go where they can all be in the same package.
Once two functions are too far apart to be on screen at the same time, jumping back and forth between two functions in the same file doesn't seem any easier than switching between different files. If anything, switching between two different files is easier since they each get an editor tab.
On the other hand, being wary about extracting functions (which moves things logically together further apart) makes more sense.
For me, it's less about number of files than it is "hoeany files do I have to open to figure out how something works? How many levels of indirection do I have to keep in my head?"
I started out writing low-ish level code. Motion control, image processing, digital imaging, and the application level code that coordinated it all. I've steadily moved up the abstraction tree over the last 13 years and there's one thing I hate about it; too many fundtions and classes which do far too little. Abstraction for abstraction's sake.
It's a bit subjective of course and not everyone has their own style, but there is a Starbucks difference in philosophy between the two types of groups.
This is what a lot of Go code looks like. This "space shuttle" code honestly isn't much more verbose than most Go code I interact with. The main difference is they have more comments here.
Likewise, I've seen a lot of Go that is far away from the idomatic Go heaven that is the standard library. Although that happens in every language as soon as there's reasonable complexity.
As for the "space shuttle" term, it's more of a euphemism for "do not even attempt to refactor this mess, it's so complex that you'll surely fuck up if you do, and you'll make it harder for the one guy who can actually understand this"
It's freaking kubernetes, so obviously it will be a well understood file by a LOT of people. However this is the kind of file that also exists in unknown, non-open source products, and while may have been understood by lots of people (at the respective companies) sometimes a period of 5 years of inactivity and layoffs and hiring goes by... By gosh darn it- this is it. It's the reason we have shitty code but great products.
Straightforward concepts should be coded consisely and complex concepts should be coded verbosely. The commenting in this file slows down the thinking of the reader to an appropriate level.
To be honest, I remember myself back during my first dev seps struggling a lot reading code since a lot of the senior programmers tend to code very effectively, so it was extremely hard to follow an end-to-end solution without terribles headaches.
This is just brilliant, not only because as educational exercises explains perfectly what's doing but the business/thinking process/context.
Never read 1k lines so quickly before, never enjoyed so much.
As a novice programmer, I was absolutely stunned that this was not standard practice. A typical source file provides zero context, background on the subject, pointers to reference material/blog posts/books explaining the concepts, information on how it fits into the program's 'bigger picture', or (most importantly) the thought process that resulted in the file (i.e., why the choice was made to do _this_ rather than _that_, challenges faced, trade-offs, etc.).
It still baffles me. Every programmer has to start from scratch dealing with a new codebase, and it makes improving any non-trivial program impossible unless one is willing to spend hours of archaeological examination. To open-source developers: if you want to get people contributing to a project (and make everyone's effort much more enjoyable!), these sorts of comments are essential. Not to mention they'll save everyone boatloads of time; it's a shame that every programmer has to piece together knowledge from scratch, rather than being 'tutored' by their peers' comments.
There's plenty of good reasons to not write 95% of code with big walls of explanation. The first is a matter of cost: Writing a good explanation around everything is very expensive to do at first. A whole lot of the custom code you find in random companies, from the shiny SV startup to the old enterprise, is unimportant, cobbled together pieces. We have no idea of whether we are writing code that will be thrown away in a week, a month, a year or whether it will last two decades. Context can change fast enough that the comments become worse than useless, as the terminology might have changed, or had been misguided in the first place, turning the long comments into outright unintended deception. This gets even worse as we do not evaluate all the comments in all the files whenever there's a code change: It's crazy how a large comment block in one place can become harmful after it's forgotten, and someone else makes a correct, business critical change in another file. No matter where I am working, it's rare for me to not find multiple examples every year where the code and the comments provide very different impressions of what is going on, and it's the code that is accurate.
This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. By then, it's clear which part of the system are dangerous, suspicious and unclear. Where there's more need for high quality error handling, and where thousands of lines of error handling never get hit, because the failing case that was originally considered doesn't really happen in this dimension anymore.
Writing code so that someone, even a future you, can pick it back up and improve it when it's needed, while still delivering the code at a good pace is a kind of skill that many just don't learn, either because they are always in greenfield teams that never pay for their mistakes, or have an approach to maintenance that involves not becoming intimate familiar with a system, and instead either hack or rewrite.
But nobody looks great in a resume by saying that they are a specialist in software maintenance.
> As a novice programmer, I was absolutely stunned that this was not standard practice.
I agree with your point, and I will be benefit from this style if it were the standard, too. But don't you think a good community culture can make people maintain a good git history for this purpose?
My daily job is a Linux kernel developer. I found that source codes are only the "What" part, git comments can and should state the "How/Why" part, and if all those still make no sense to me, I look for the original mailing list for the deeper "Why". Most of the time the information is sufficient.
> unless one is willing to spend hours of archaeological examination
Currently the only person in-office over Christmas on my first software dev job. Debugging a 50k LOC COBOL beast that digs into three other beast programs and ends up in a final 20k LOC uncommented piece where things are supposed to happen and be returned back.
Nothing is commented, the programs are huge and one can only debug one program at a time, requiring me to submit untested changes in one of them to the shared dev environment since I'm making changes in two, before debugging the other program.
If it just said somewhere what half of the stuff is I would save an insane amount of time getting to know the system.
I think your desire is right, but think about this every time you create a file, and how much slower your work would be. The question then becomes: "how much commenting exactly is needed before this becomes more time than the technical debt it creates?
I think this type of summary should not be per source file but per package/folder/module/project. A high-level developer overview with sufficient depth will also help stop repetition of philosophy in subsequent files.
In this case, I do agree with its existence though because of both the length and complexity of the file. For many files, neither are the case.
The information you are talking about is called documentation and it should exist outside of actual code files.
If you need to read comments to understand code, you don’t know how to read code. Comments can lie, only code is the source of truth, once you gain experience you won’t bother with comments.
Sure, and then the next day the whole thing is just a long boring text which has no correlation to reality because the business rule changed and the developer next to you refactored the code.
If the developer next to you refactored the code without refactoring the comments, they did a shit job. Period. They need to be told to go back and fix it.
One thing that I wish Linux kernel code had. Maybe it does but the few times I have found myself reading Linux code I go to the top and there is zero context in the comments, just a bunch of licensing information.
There are few linux drivers for particularly buggy hardware that are written in this style. Although OTOH what I think of is hme.ko, where the comments are more on the hillariously funny side than descriptive.
This. Documentation (comment blocks in code, as well as all other forms of documentation) tends to become outdated because it is not maintained in sync with every code change.
Solution: write clear, simple, modular code that is self-documenting and does not need extensive commenting.
How soon? I've rarely seen it happen in a meaningful way (i.e. other than someone using automated refactoring on some name, and breaking some comment reference that wasn't written in a way the autorefactor tool could read). When it happens and you see it, you should fix it like any other bug.
Key trick is writing comments as close as possible to the code they affect. Then it's hard to miss affected comments if one's not doing a shoddy job.
Overall liked the original post. But also liked what you said.
But here are a couple of Martin Fowler quotes (from his Refactoring book) I tend to follow:
“A heuristic we follow is that whenever we feel the need to comment something, we write a method instead.”
“Whenever I have to think to understand what the code is doing, I ask myself if I can refactor the code to make that understanding more immediately apparent.”
Having spent 25+ years writing, viewing, commenting on and reviewing code in a multitude of languages, this is good stuff to see - regardless of the 'style' of programming (or the language broadly-speaking).
Stepping back and whilst we can all overlook it, good code comments can make an enormous difference in productivity - both for an individual, a team and indeed a business. It aids repository knowledge (something that is easily lost between current and prior teams/individuals), which shouldn't be mistaken for the intelligence of someone looking at code...
I've spent far too much time personally and otherwise attempting to reverse-engineer code written by someone with no comments or explanation. At times, super experienced programmers/developers will take performance-enhancing shortcuts that the less-experienced don't understand; They'll compress routines and functions that are a result of their explicit knowledge of a language and/or domain but without explanation...
On a basic level, comments should: inform, educate, outline and help others understand the sometimes complex routines and functions that we all create and often under an enormous amount of pressure.
There are those that believe good code shouldn't need explanation and to some degree that's true, but you can't apply that brush to every codebase. Code can become complex, awkward, spaghetti-like and almost unfathomable at times.
I've always strived to teach less experienced developers to comment well, efficiently and with a little humor/humour (where possible); Something that allows us to understand code quickly, appreciate the efforts of those before us and smile/grin at the complexity of a challenge.
Personally, I don't really care about the code/comment ratio - It's a complete red herring. At times, code comments can be worth more than the code itself. At other times, they just help you get your job done; quickly, efficiently, no fuss, just great code.
> There are those that believe good code shouldn't need explanation and to some degree that's true, but you can't apply that brush to every codebase. Code can become complex, awkward, spaghetti-like and almost unfathomable at times.
I have yet to find a codebase that couldn't be made clear as soon as a programmer actually put some effort into doing so. Far too often adding a comment is used as an excuse to give up on making the code readable before you've even started.
3. Is the comment adding more information, making the code more clear?
If Yes: Put that information into the code. Rename variables. Pull out code into subroutines.
If No: Delete the comment.
You’ll be amazed at how often this practice works. Doing it all the time will make your code more readable.
We have a second, enforced practice at work, thanks to code review. The question in your head is simply: “Am I gonna get a comment about this at review time?” If yes, you gotta make the code clearer/simpler/better. Because you’ll have to answer and address the comment, and that’s just gonna slow you down more than if you just fix the problem now.
Well, you seem to be providing data against your own theory.
Usually codebases are a mess, so were people commenting more on what's the goal then refactoring it would be easier.
Software is very susceptible to the "perfect is the enemy of done" mantra. Software works as soon as it works for the first time. And then it gets a shipped. It's 1.0, even if it looks like a mess internally. Because IT development is expensive, there is rarely budget to go that extra mile (which is again usually would cost a lot more than the first few miles).
> On a basic level, comments should: inform, educate, outline and help others understand the sometimes complex routines and functions that we all create and often under an enormous amount of pressure.
Take the time to simplify the complex routines, clean them up and make them readable and don't waste your precious time in writing "good comments".
> Code can become complex, awkward, spaghetti-like and almost unfathomable at times.
The solution for this is not "comment more". It's "clean up the mess".
> At times, code comments can be worth more than the code itself.
That doesn't make any sense. If the code has no value, you can remove it.
> I've always strived to teach less experienced developers to comment well, efficiently and with a little humor/humour (where possible); Something that allows us to understand code quickly, appreciate the efforts of those before us and smile/grin at the complexity of a challenge.
Please don't do that. Of course it's your code base, but usually it's best to find better venues to express your humor than code comments. Also: teach the junior developers to write clean code first.
Comments too often are used as a bad device to fix code. A developer first writes a horrible mess of code and stops. He may then realize that perhaps no-one will understand it. But if we now teach them not to clean up the code but rather just "write some funny comments", do you think it's any better?
I've seen too many code bases written with this attitude. The comments usually don't help at all, they distract the reader and they distract the author. It's too often useless noise. Many developers hide the comments from these code bases so that they can concentrate on what ACTUALLY is relevant: the code.
The worst part about comments is that they just add a cognitive load of reading and parsing extra lines of text.
What if the underlying logic changes? Do I need to update the original witty comment as well?
How do I know if the comment is still relevant? No compiler nor tests will tell me that and now I'm left with a task of parsing a language with much more degrees of freedom (English), without any support from the IDE. It becomes even harder in international teams.
Well said, I was cringing when I was reading the bad advice and you addressed the points very well.
It is so frustrating to hear when someone thinks commenting more and adding humour is somehow cleaning up the code. But to then hear that they are teaching this to juniors just hurts so much.
Simpler advice to juniors would be to read Clean Code by Robert C Martin, apply some of those techniques and then politely ignore “more experienced” developers who think writing humourous essays as code comments is a good thing.
I’ve even heard phrases banded around like “the more comments the better” - I mean seriously WTF.
I like the analogy of...
When in an unfamiliar city, having no map at all is much better than a map that you have no idea whether you can trust or not. Code comments are absolutely that untrustworthy map, doesn’t matter how many code reviews or convoluted PR process takes place, you still need to read the code to know the truth so more effort making the code communicate is the key to maintainable code. Simple things like good variable names and grouping behaviour into functions at the same level of abstraction can completely negate the need for comments.
Comments have their place, but should be the exception not the norm.
One of my pet peeves is JS projects. I am not sure why but the front-end developers refuse to add any comments at all. This trend, oddly enough, started with ES6. Pre-ES6, JSDoc style comments at least, were quite common.
Just because some popular Javascript project doesn't have comments doesn't mean yours shouldn't. Straight-forward code may not need comments but most of these projects definitely need to explain why or how are things supposed to work or why things are done a certain way.
> Straight-forward code may not need comments but most of these projects definitely need to explain why or how are things supposed to work or why things are done a certain way.
Code comments usually are not the best place to elaborate on "why things are done a certain way". One can also write documentation about the architecture, design etc.
"it became clear that we needed to ensure that every single condition was handled and accounted for in the code"
This is a feature of several (mostly functional) programming languages, e.g. Haskell. Fun to see that often people figure out that these types of concepts are a smart way to write your code. Too bad it usually means many people reinvent the wheel instead of learning about computer science history and other languages.
I know a business coach who regularly asks his audience "Who here makes better burgers than McDonalds?". When half the audience raises their hand, he asks them why they don't outsell this giant company.
Functional programming advocats, especially for the "pure" ones like Haskell, always strike me as odd. It seems that all the beauty of those languages make people obsess over that beauty and purity while keeping them from being productive.
Meanwhile, people with simpler languages like Go just get stuff done that is useful and makes people happy. Now if I _ever_ came across a useful Haskell product, I'd be happy to test drive it, shouldn't be a problem by now with all the container technology. But the closest I ever came to using a functionally developed product was RabbitMQ (written in Erlang). That one was _such_ a pain to use and operate — must have been the developers still dreaming in the purity of its code instead of writing some installation docs. I moved on to Kafka later and didn't regret it a minute.
Selling a lot of burgers encompasses much more than making good burgers. By the same token, good products entails much more than making a programming language choice.
Functional programming, at its heart, is about using self-imposed constraints to avoid certain classes of programming mistakes.
If your application domain doesn't have big consequences for these classes of programming mistakes, then it can seem like functional purity can be a luxury or frivolous. However, if your application domain suffers greatly from those classes of programming mistakes, such as distributed systems, then it may be worth it to consider what functional programming might buy you.
So yes, just because you use a functional programming language won't help you sell your widgets or make a great product. And you can spend lots of time fucking around with it for its own sake and still not sell widgets or make a great product. However, if you understand what its constraints buys you, then you can make your job of building these things in certain domains much easier.
> I know a business coach who regularly asks his audience "Who here makes better burgers than McDonalds?". When half the audience raises their hand, he asks them why they don't outsell this giant company.
There is indeed a lesson here, but which one do you think it is? It's certainly not that McDonald's makes better (or "simpler") hamburgers: once you taste good hamburgers you can never go back to McDonald's (and yes, I make better hamburgers too, even though mine won't win any prizes!). To me, the lesson is that there's more to success than product quality. The established brand matters, the scale at which you can sell a (possibly inferior) product matters, how low you can get away with paying your employees matters, how well you can survive PR disasters matters, etc.
If you had to make burgers, would you rather make cheap and mediocre ones, or would you rather enjoy making premium burgers? :)
> Meanwhile, people with simpler languages like Go just get stuff done that is useful and makes people happy.
Kubernetes' reputation is just the opposite: that far from being a simple and useful thing, it's an overengineered, overcomplicated solution to a self-inflicted problem (deploying a distributed monolith).
> But the closest I ever came to using a functionally developed product was RabbitMQ (written in Erlang). That one was _such_ a pain to use and operate — must have been the developers still dreaming in the purity of its code instead of writing some installation docs. I moved on to Kafka later and didn't regret it a minute.
Erm, Kakfa was developed in Scala, whose advocates far more of a reputation for purist pontification than Erlang developers do. Maybe all that beauty and purity is actually good for something?
The paucity of useful tools/libraries written in Haskell as compared to languages like Go are more due to the fact that there are far more people in the Go ecosystem than Haskell, rather than because Haskellers are too busy navel gazing.
This disparity in numbers is in turn is primarily because golang/Python/C is inherently much more approachable than Haskell because the average programmer has cut his/her teeth writing imperative code for a significant portion of their early programming career. The jump to functional way of thinking requires a certain leap of the mind, that most people don't want to take the trouble doing because they seem to be happy "getting things done" in golang or Python.
However, that is not to say that we shouldn't be striving for correctness in code that Haskell fosters, or the inherent simplicity that it forces on you because you are forced to separate your IO and effect-full code from the pure bits. These are things worth striving for. These are broad principles worth emulating even when you are coding in an imperative language.
To turn your McDonalds analogy around: sure, a McD will let you just get your food requirements out of the way quickly and cheaply (i.e. just "get stuff done"). But in the long run, it's bad for you.
Haskell is a healthy salad to an [insert favourite imperative language] burger.
I agree with the general sentiment of your post, but the idea that Kafka is somehow less painful to operate than Rabbit is ridiculous. I love Kafka, but I can't think of a more painful software to maintain in production.
>But the closest I ever came to using a functionally developed product was RabbitMQ (written in Erlang). That one was _such_ a pain to use and operate — must have been the developers still dreaming in the purity of its code instead of writing some installation docs. I moved on to Kafka later and didn't regret it a minute.
Err, RabbitMQ is one of the easiest to setup and stabler queues/messaging systems. And I'm no user/fan of Erlang...
I agree with the spirit of what you're saying. Too often I've seen folks on HN dismiss languages because they didn't have features X or Y, usually with a condescending "maybe this language's creators should read some PL theory". I remember circa 2013 or 2014, it became impossible to read threads related to Go because the entire thread would always be "a Real Language(TM) needs Generics". Look at the success Go has seen in the last decade while doing the opposite of what HN thought was correct.
I wonder if it’s not the language that causes the products, but more that programmers that are drawn to functional languages are less likely to care about product.
That said, lots of things are functional: chunks of Facebook, Twitter, and Microsoft (and I assume Google) are written in OCaml, Haskell, Reason, and F#. Jet is built in F#. Jane Street famously uses OCaml, and Scala is becoming the standard language for hedge funds. Spark apps are usually written in Scala. Etc.
Erm, excuse me? Idiomatic Erlang is definitely not the purely functional, obsessed with correctness and types oasis of Haskell. And a the community that birthed Kafka (Scala) has much more in common with the Haskell community than the Erlang community.
Furthermore, why are you using the user interface as a yard-stick to judge language paradigms with?
> I know a business coach who regularly asks his audience "Who here makes better burgers than McDonalds?". When half the audience raises their hand, he asks them why they don't outsell this giant company.
Because I'm not in the business of making hamburgers nor am I interested in doing so.
Struggling to see this business coach's point. Is he saying that McDonalds makes better burgers than me because I'm not selling a million burgers a day? Because that's a load of crap.
OCaml is not Haskell, but it does provide almost all of the guarantees Haskell does. Here's a list of popular software you might have used written in OCaml:
- the Flow and Hack compilers
- Facebook's Messenger app
- Jane Street's entire trading infrastructure (and everything else they do)
- XenServer
- some parts of Docker, including the TCP/IP networking stack on OS X and Windows
Seems like you have bad experiences with functional programming, but it's a little strange to rat on the advocates that are trying to figure out how to take potentially useful functional programming concepts and make them mainstream and/or explore alternative ways to quickly build robust systems.
Good examples of this translating to huge gains for the overall community are React + Redux.
I'm quick to admit that functional language ecosystems are not as mature, which might lead to lesser organizational productivity but no need to rat on functional programming in general.
If you a proffesional chef, I would hope you are fammilar with other methods of producing burgerz than McDonalds, even if they don't sell as well.
Haskell is a highly opinionated reasearch language. This makes it a great languages to talk about. People could make many of the same points with, say Scala or Ocaml, but because those languages arent as opinionated, it is harder to use them as a basis for discussion than a language which is heavily opinionated about the subject in question.
FP advocates strike me the odd way that they trying to argue FP can reduce the intrinsic complexity of the problem itself, but they have no proof of it, or their 'proovies' are essentially faith.
Why do you say Haskell is an example of this? You can return undefined for anything, and have incomplete pattern matching, where if you don't mention a particular case, there is a runtime crash.
An example would be head, which takes the first element of a list and throws an error on an empty list.
I really love Haskell but it seems as if they are going for something different here than what Haskell provides.
In all seriousness though, Rust does get a little closer - it requires exhaustive matching (or explicit opt out) and deliberate error handling (or explicit opt out). There are still ways around it, but the happy path in Rust is handling errors.... well, maybe not happy. It is a little verbose.
For "Haskell-in-practice", incomplete pattern matching isn't an issue - there's a warning for that, you should have it on, and you should have it error. `undefined` is more of an issue, although easy to exclude with code review - relying on habits isn't great but especially when the habits are this simple it's not a problem in practice.
That said, Haskell still isn't actually an example, chiefly because exceptions can be thrown by any code, and are reasonably often used in practice.
That's not really a limitation of the "language" but more of an implementation detail of GHC and Prelude imo. There are very few escape hatches from the type system and you can run the compiler so that these cases are treated as a compile time error.
My point was not really to mention a particular programming language, but rather to make the point that people seem to continuously reinvent the wheel rather than reading up on history and investigating other programming languages.
If you only know C/Java/Python/Go and you learn about <radically-different-language>, it will probably make you a better programmer even if you never write a single line of <radically-different-language>.
I really like the idea of "sound" programming languages. Elm is a great example of a reasonably simple, very sound programming language that force you to handle all cases (short of a compiler bug, hardware failure, or an explicit fail the world statement, it basically cannot throw exceptions).
Unfortunately the odds of this ending up in a mainstream language this decade is pretty low: the extreme focus on terse code and DRY means the average dev balks at the verbosity (thus the coment in the linked piece of code being necessary). It's a shame, as it's objectively superior by many metrics.
Code terseness and case analysis (handling all cases) are totally orthogonal: OCaml is pretty terse and yet the compiler is great at letting you know if you forgot to handle some case.
> This is a feature of several (mostly functional) programming languages
This is just one of a common practice of programming, not a feature of functional languages. These practices are not even "reinventing the wheel".
I mean, this is obvious in many areas: from implementing complex logics like Kubernetes to making hardware drivers in C. Programming languages themselves can't automagically ensure every single conditions because these often happen outside of the program (e.g. targeting hardware state). We need to cover and test all cases by hand anyway.
This is one of the painfully obvious facts that tend to be totally ignored by most. Even if you formally verify all of you code it does not mean that you have covered all the possible states of the outside world. And this can mean both unexpected states on the outside and unexpected hardware failures.
On one project I work with guy with significant railway signalling experience and this is one of the issues that I'm somewhat unable to explain to him. Probably because our product's design target is not to fail-fast-and-safe but fail-secure.
> This is a feature of several (mostly functional) programming languages, e.g. Haskell
I didn't see that.
In fact this speaks to me as mission critical software like this needs to be as tediously documented as possible to eliminate surprises. Those branches and conditions are collected through a huge pool of trial-and-errors, implying Haskell can provide those valuable use cases out-of-box is misleading, no it can't.
Model checker like TLA+ or Pluscad might do the trick.
I think what the parent comment referred to is that in Haskell if/then/else is an expression (like everything else) so you must by definition have to have an else “branch”. Basically it frees you from a subtle type of error.
In general, I think that if you find yourself "needing" to write in an intentionally complex and verbose style, this is symptomatic of bad design choices elsewhere, maybe even at the language level.
My take away from reading this code is that it is a huge mess that may be impossible to clean up. At some point they failed to introduce abstractions that would remove the need for all this complexity. They are probably right that now that it works that it will be hard to refactor it without leaving out some critical case. However, I pity anyone that works on this code base.
Given where they are, it seems that now the major abstraction they are missing is type safety. Strict types that allowed a monadic structure would simplify this whole mess into a series of maps and folds.
Not only does this add the stricture of a compiler enforcing correct return types for all conditionals, it is also idiomatic of any language that supports this.
Having said all that, changing languages is rarely an option. Maybe this is the best solution given the tool available. If so, it demonstrates that maybe this tool isn't right for this job.
Yeah, I just don't buy this. It says nothing. This is the bottom-barrel suggestion most people bring up, and there's more to simplifying codebases than "strict types."
What in this <2k LOC file suggests strict types help anything whatsoever? It's not massive. It's large, but fold this and it immediately becomes more readable. Any attempts to simplify this would simply be reasoning about it differently.
Would you agree that sometimes, abstractions can just distribute and hide complexity whereby actually all of that context is needed to comprehend the algorithm or process at hand (some things just ARE complex)?
In this case right here, what's your counter-example, or what would you use instead of their specific approach?
> Would you agree that sometimes, abstractions can just distribute and hide complexity whereby actually all of that context is needed to comprehend the algorithm or process at hand (some things just ARE complex)?
No, or at least not often enough to be worth thinking about. It is of course possible to use abstractions badly, but the problems that business software has to solve are always fairly simple because they're human problems; human business processes were never that complex. So if a program looks really complicated, the overwhelmingly likely cause is failing to use appropriate abstractions.
> In this case right here, what's your counter-example, or what would you use instead of their specific approach?
As others have said, a result type would greatly simplify this code without sacrificing any safety. No doubt after such a simplification made the code shorter and easier to comprehend, further simplifications would become apparent.
You are right that they move the complexity somewhere else, but when abstractions are done right, you don't have to worry how they are implemented when you use them. That is huge benefit.
I believe that premature abstraction is terrible. But in some cases good abstractions may help with handling complexity and may make testing easier. I have a feeling that because authors say that this class should not be changed / refactored, they failed to introduce a good abstraction. This also implies that their tests are inadequate.
A file is an abstraction that hides an incredibly complex process, possibly even involving quantum effects. I have never found a reason to care about the actual details and managed to do quite a significant amount of work using just the abstraction.
As much as I like the style where the code logic is commented very thoroughly, it also falls in the trap of comments no longer matching the current code, I assume some variable renaming happened.
In the function (line 320 of https://github.com/kubernetes/kubernetes/blob/ec2e767e593953...)
the variable "pvc" seems to have been renamed into "claim" and "pv" into "volume", judging from the code/comment mismatch. Comments in the lines 339, 358, 360, 370, 380, 395, 411, 422, 427 point to the old names. Furthermore in line 370 the comment reads:
} else /* pvc.Spec.VolumeName != nil */ {
while the matching if is:
if claim.Spec.VolumeName == "" {
So not only the variable name mismatches, but also the comment is wrong. The VolumeName seems to be a string, so is never nil, the else comment should specify that the VolumeName is non-empty.
The more verbose and detailled the comments are - the more work needs to be spent in ensuring that they are correct.
> Space shuttle style is meant to ensure that every branch and condition is considered and accounted for…
FTFY: …hopefully!
Only if they’ve used a language with Algebraic Data Types support, the compiler would enforce that “every branch and condition is considered and accounted for.” The only PL with ADT that I’ve used was Haskell, but I’ve heard that Rust has them too “enums”.
People are arguing that “code is what computer executes, comments don’t ensure anything!” and so on, but besides being executed, (this Go) code does not ensure anything either. It’s human eyes that skim through all the cases, and look for a matching branch for every one of them that “ensures.”
In my humble opinion, this so-called “space shuttle style” is just one of the many workarounds to deal with Go’s by-design limitations (the most famous one being lack of generics), a language that’s designed only 9 years ago.
> It’s human eyes that skim through all the cases, and look for a matching branch for every one of them that “ensures.”
This a thousand times. The praise in this thread is disturbing.
The absurdity of this code is the logical conclusion of ignoring decades of PL advances in favor of Go's "simplicity."
When you insist on "space shuttle" era language design, is it any surprise when you're reduced to "space shuttle" era programming? I can't imagine anything more fitting.
I agree with this, but snarky comments like these neglect that ADTs (or generics) are not the only nor remotely the most important factor in choosing a programming language. Go certainly bests Haskell and Rust in many important areas even if it loses in safety.
Two years ago I’ve moved one of my biggest projects from Python 3 to Go because my program was inherently concurrent, and at the time -and I think still- Python has 3 competing approaches for concurrency: threading, multiprocesses (for parallelism), and asyncio.
Although it’s nice to have a variety of options, I think this balkanisation affected the community in negative ways because the options are not compatible with each other (see the emerging “SansIO” libraries for asyncio).
This is actually the reason why Python gets so much praise: thanks to its standard library, there is often a single canonical way to do something (if you need sets use `set`, if you need matrices use NumPy, …), which means that the vast majority of libraries are interoperable* with each other. Same goes for Go when it comes to concurrency, and that’s what guided my choice.
>It’s human eyes that skim through all the cases, and look for a matching branch for every one of them that “ensures.”
Worse, they've made an exception for simple error checking, and the result is that the majority of the if statements at the top of the file have no else condition. Quickly scanning by eye doesn't help me determine if someone screwed up and missed a scenario.
But this gargantuan pile of garbage warns that it can not and should not be refactored and it is like the shuttle code (may their souls rest in peace), it is so solid and perfect as is. It is also written in the sacred language of Go.
So if these conditions are met, then yes above praises will be and must be sang.
The jazz music analogy implies that it breaks the rules, but do so artfully and intelligently -- that wouldn't necessarily be a valid defense for any old block of poorly engineered code.
Also, poorly engineered code is seldom irreducibily complex.
Readit News
A naive look at this and my head is screaming that this file is way too big, has way too many branches and nested if statements, has a lot of "pointless comments" that just describe what the line or few lines around it is doing, and has a lot of "logic" in the comments which could quickly become outdated or wrong compared to the actual code.
Yet at the same time, it's probably a hell of a lot easier to maintain and manage than splitting the logic up among tens or hundreds of files, it contains a lot of the inherently complex work it's doing to this file, and it is so well and heavily commented that it should be pretty easy to ensure that any changes also keep the comments up to date (after all, any change without changing the resulting comments should show up like a sore thumb, and will most likely prompt the reviewer to look into it at the very least).
I kind of see it as the opposite: “space shuttle style” is code that adheres to heavyweight rules that most software development has abandoned in favor of a more improvisational style.
But in either case it illustrates that code style rules are desirable, or not, based on the context; you need to understand what purpose rules serve and what trade-offs they involve to understand which rules to use; there's no one-size-fits-all solution.
The complexity goes somewhere.
It's either into lots tests, or it's into something like shuttle style with lots of comments, or it's into a huge QA department, or it's into the type system / DB schema. It could even be going into the org structure!
But something, somewhere is handling the complexity and it is doing so as a partial function to the economic importance of the software to the stakeholders of the software.
Most software other than that written for space shuttles and other seriously critical applications.
A colleague of mine works for a telco, and changes to software that runs on satellites takes months to approve and goes through verification stages that include running on simulators and duplicate hardware that is on the ground.
I'm only halfway through John Ousterhout's book Philosophy of Software Design but I think it agrees with you on this -- that smallness-of-file or smallness-of-function is not a target to shoot for because it prevents the things you build from being deep. That you should strive to build modules which have deep functionality and small interfaces and should contain their complexity within them so the users don't have to know that complexity.
As for comments, this file is essentially Ousterhout taken to the extreme. Still, I think he would have like it, given how critical this file is. In the book, he encourages writing more comments than the current trends would suggest, pointing out that you can't fully express abstractions in code, so all the things the code doesn't contain - the high-level concepts, the rationale, the caveats - should be documented in comments in appropriate places.
Overall, I'm extremely impressed by the book, and its focus on reducing and mitigating complexity.
Sure, it solves the problem when viewed from the outside. "users" of the software (users being other devs in this case) get a nice small interface and docs that explain how to use it, but internally it's much harder to work with. Having everything in one file like this without breaking it into "sub modules" for various parts of the module means that you need to almost have a complete understanding of the module before working on it.
In this case, I have a feeling that is a pro not a con. Because this file is so core to the system, and has so much complexity, that breaking it up into smaller parts could cause a dev to feel like they understand it only to find out they don't after it's released. And it means that any devs that truly do understand it top-to-bottom would just waste a lot of time switching around files if it were split up.
Putting it all in the same file here nudges you to really make sure you understand it top to bottom before making any big changes. It's intimidating and scary for a reason, because at its core it is a complex piece of code, and dressing it up in "simple code's clothing" won't help.
this. interfaces are not the most important thing in a software but definitely one of the most. you can live with a crappy implementation but your interfaces must cater to the usecase and should only hcange when core premises change.
Similar to UX design principles. Just like google or other successful sites present a very simple interface (in case of google the home page is quite clean) but behind it lies very complex code.
IMO, encapsulating complexity and handling all edge cases also lends to total functional programming [1] even without using pure functional languages. [1] https://softwareengineering.stackexchange.com/a/334876/38285
I disagree. Unless you are methodical and know what you are doing (like NASA or the authors of Kubernetes), it is hard to create large functions and files by keeping levels of detail consistent and not repeating code.
How do you test a function that has 100 unique outcomes? How do you safely maintain it to ensure it won't break? How do you even know it's working?
With code that far down in the inner loops of mission critical code, the dominant time cost isn't really writing or even understanding the code, it's ramping up on the nuances of the scenarios you serve, checking the direct effects of your changes, and then checking the second order and third order effects of your changes.
When your code is so far down that every change you make has third order effects, it's a lot easier to maintain code that is meticulously commented and written for maximum thoroughness explicitly baked into the file in front of you, because you're much more likely to miss the nuances if you just finished piecing together ten different files to figure out how the happy path works.
I am all for code that is concise and whose syntax/naming is expressive, but sometimes comments are necessary to clearly spell out the logic or business use case. Expressive code can only go so far. Well-crafted comments significantly reduce the amount of time required for other developers to dive in and become productive with an unfamiliar code base.
The key is keeping the comments up to date. There's nothing worse than an inaccurate comment. One's code review process must include a review of the comments accompanying the modified lines.
It's probably because reading comments only is worse than reading code without comments, that some devs developed an aversion towards outdated comments and thus comments in general.
Comments are additional information and no source of truth, always take them as that and read code and comments.
I warned him at the beginning that he needed to spend a couple of hours wrapping his head around the code before trying to modify it.
I warned him that the code arrived at its current state after long and painful experience.
I warned him that other developers had worked on the code, had exactly his initial reaction, and eventually admitted they couldn't find a way to improve it.
I warned him, after he confidently declared that it just needed to be "more object-oriented," that I had written a lot of object-oriented code, was open to writing it in an object-oriented style if it would improve things, and could not think of any way to do so that would not make things worse.
But he could not even bring himself to read the existing code. He never did. He worked on it for three weeks, wrote thousands of lines of code, and even tried to deploy his own version without ever spending a single contiguous, focused, hour-long block of time reading the existing code. He was "refactoring" the whole time.
The code he produced was exactly how you imagine it. The code went from several hundred lines in a single file with no classes (just a containing class acting basically as a namespace) to thousands of lines scattered over half a dozen files with at least that many classes. The guy kept adding classes and kept adding test after test after test. He couldn't get his code to pass the test cases I had written, so several times he declared that my test cases were wrong and changed or removed assertions. He also claimed that the existing code couldn't "pass" his tests because he had hundreds of lines of tests for components that only existed in his code. According to him, this proved that there was a ton of "hidden untested functionality" in the existing code, which was therefore "dangerous."
I was pretty busy with other things, and this was his project now, so if he had been a little bit more clever he probably could have made his abomination stable enough to replace the existing version over my objections. Thankfully it was never good enough to survive in production.
I actually understood it immediately when I read it. There's something to be said when you have a decent roadmap for debugging critical code segments. It's not just software maintenance here, it's for the "holy crap something went wrong and we can't understand how we got here!" moments.
That stuff is lovely and terse when it works. And when it doesn’t you are at a total loss.
The beauty of imperative code is there’s a beginning and an end, and at every step you can easily see what follows and without too much work what came before.
Once two functions are too far apart to be on screen at the same time, jumping back and forth between two functions in the same file doesn't seem any easier than switching between different files. If anything, switching between two different files is easier since they each get an editor tab.
On the other hand, being wary about extracting functions (which moves things logically together further apart) makes more sense.
I started out writing low-ish level code. Motion control, image processing, digital imaging, and the application level code that coordinated it all. I've steadily moved up the abstraction tree over the last 13 years and there's one thing I hate about it; too many fundtions and classes which do far too little. Abstraction for abstraction's sake.
It's a bit subjective of course and not everyone has their own style, but there is a Starbucks difference in philosophy between the two types of groups.
As for the "space shuttle" term, it's more of a euphemism for "do not even attempt to refactor this mess, it's so complex that you'll surely fuck up if you do, and you'll make it harder for the one guy who can actually understand this"
This is just brilliant, not only because as educational exercises explains perfectly what's doing but the business/thinking process/context.
Never read 1k lines so quickly before, never enjoyed so much.
If Anthony Braxton wrote software...
the positive renforcement cheerleading to start off so that people dont get demoralized in the hateful comments, while effective also seems fake.
It still baffles me. Every programmer has to start from scratch dealing with a new codebase, and it makes improving any non-trivial program impossible unless one is willing to spend hours of archaeological examination. To open-source developers: if you want to get people contributing to a project (and make everyone's effort much more enjoyable!), these sorts of comments are essential. Not to mention they'll save everyone boatloads of time; it's a shame that every programmer has to piece together knowledge from scratch, rather than being 'tutored' by their peers' comments.
This is not to say that there aren't reasons to write large comment blocks, or architecture documents, but that they are often better written not while the system is being first built, but later, in a maintenance cycle, when someone already had wished for the comments, and has regained the knowledge the hard way. By then, it's clear which part of the system are dangerous, suspicious and unclear. Where there's more need for high quality error handling, and where thousands of lines of error handling never get hit, because the failing case that was originally considered doesn't really happen in this dimension anymore.
Writing code so that someone, even a future you, can pick it back up and improve it when it's needed, while still delivering the code at a good pace is a kind of skill that many just don't learn, either because they are always in greenfield teams that never pay for their mistakes, or have an approach to maintenance that involves not becoming intimate familiar with a system, and instead either hack or rewrite.
But nobody looks great in a resume by saying that they are a specialist in software maintenance.
I agree with your point, and I will be benefit from this style if it were the standard, too. But don't you think a good community culture can make people maintain a good git history for this purpose?
My daily job is a Linux kernel developer. I found that source codes are only the "What" part, git comments can and should state the "How/Why" part, and if all those still make no sense to me, I look for the original mailing list for the deeper "Why". Most of the time the information is sufficient.
Currently the only person in-office over Christmas on my first software dev job. Debugging a 50k LOC COBOL beast that digs into three other beast programs and ends up in a final 20k LOC uncommented piece where things are supposed to happen and be returned back.
Nothing is commented, the programs are huge and one can only debug one program at a time, requiring me to submit untested changes in one of them to the shared dev environment since I'm making changes in two, before debugging the other program.
If it just said somewhere what half of the stuff is I would save an insane amount of time getting to know the system.
I think this type of summary should not be per source file but per package/folder/module/project. A high-level developer overview with sufficient depth will also help stop repetition of philosophy in subsequent files.
In this case, I do agree with its existence though because of both the length and complexity of the file. For many files, neither are the case.
https://svn.python.org/projects/python/trunk/Lib/heapq.py
If you need to read comments to understand code, you don’t know how to read code. Comments can lie, only code is the source of truth, once you gain experience you won’t bother with comments.
It's an odd feeling, having the best of both worlds.
https://golang.org/doc/effective_go.html#commentary
Solution: write clear, simple, modular code that is self-documenting and does not need extensive commenting.
Key trick is writing comments as close as possible to the code they affect. Then it's hard to miss affected comments if one's not doing a shoddy job.
But here are a couple of Martin Fowler quotes (from his Refactoring book) I tend to follow:
“A heuristic we follow is that whenever we feel the need to comment something, we write a method instead.”
“Whenever I have to think to understand what the code is doing, I ask myself if I can refactor the code to make that understanding more immediately apparent.”
https://www.goodreads.com/author/quotes/25215.Martin_Fowler
Deleted Comment
Stepping back and whilst we can all overlook it, good code comments can make an enormous difference in productivity - both for an individual, a team and indeed a business. It aids repository knowledge (something that is easily lost between current and prior teams/individuals), which shouldn't be mistaken for the intelligence of someone looking at code...
I've spent far too much time personally and otherwise attempting to reverse-engineer code written by someone with no comments or explanation. At times, super experienced programmers/developers will take performance-enhancing shortcuts that the less-experienced don't understand; They'll compress routines and functions that are a result of their explicit knowledge of a language and/or domain but without explanation...
On a basic level, comments should: inform, educate, outline and help others understand the sometimes complex routines and functions that we all create and often under an enormous amount of pressure.
There are those that believe good code shouldn't need explanation and to some degree that's true, but you can't apply that brush to every codebase. Code can become complex, awkward, spaghetti-like and almost unfathomable at times.
I've always strived to teach less experienced developers to comment well, efficiently and with a little humor/humour (where possible); Something that allows us to understand code quickly, appreciate the efforts of those before us and smile/grin at the complexity of a challenge.
Personally, I don't really care about the code/comment ratio - It's a complete red herring. At times, code comments can be worth more than the code itself. At other times, they just help you get your job done; quickly, efficiently, no fuss, just great code.
I have yet to find a codebase that couldn't be made clear as soon as a programmer actually put some effort into doing so. Far too often adding a comment is used as an excuse to give up on making the code readable before you've even started.
1. Write some piece of code
2. Now write a comment about it
3. Is the comment adding more information, making the code more clear?
If Yes: Put that information into the code. Rename variables. Pull out code into subroutines.
If No: Delete the comment.
You’ll be amazed at how often this practice works. Doing it all the time will make your code more readable.
We have a second, enforced practice at work, thanks to code review. The question in your head is simply: “Am I gonna get a comment about this at review time?” If yes, you gotta make the code clearer/simpler/better. Because you’ll have to answer and address the comment, and that’s just gonna slow you down more than if you just fix the problem now.
Usually codebases are a mess, so were people commenting more on what's the goal then refactoring it would be easier.
Software is very susceptible to the "perfect is the enemy of done" mantra. Software works as soon as it works for the first time. And then it gets a shipped. It's 1.0, even if it looks like a mess internally. Because IT development is expensive, there is rarely budget to go that extra mile (which is again usually would cost a lot more than the first few miles).
> On a basic level, comments should: inform, educate, outline and help others understand the sometimes complex routines and functions that we all create and often under an enormous amount of pressure.
Take the time to simplify the complex routines, clean them up and make them readable and don't waste your precious time in writing "good comments".
> Code can become complex, awkward, spaghetti-like and almost unfathomable at times.
The solution for this is not "comment more". It's "clean up the mess".
> At times, code comments can be worth more than the code itself.
That doesn't make any sense. If the code has no value, you can remove it.
> I've always strived to teach less experienced developers to comment well, efficiently and with a little humor/humour (where possible); Something that allows us to understand code quickly, appreciate the efforts of those before us and smile/grin at the complexity of a challenge.
Please don't do that. Of course it's your code base, but usually it's best to find better venues to express your humor than code comments. Also: teach the junior developers to write clean code first.
Comments too often are used as a bad device to fix code. A developer first writes a horrible mess of code and stops. He may then realize that perhaps no-one will understand it. But if we now teach them not to clean up the code but rather just "write some funny comments", do you think it's any better?
I've seen too many code bases written with this attitude. The comments usually don't help at all, they distract the reader and they distract the author. It's too often useless noise. Many developers hide the comments from these code bases so that they can concentrate on what ACTUALLY is relevant: the code.
What if the underlying logic changes? Do I need to update the original witty comment as well?
How do I know if the comment is still relevant? No compiler nor tests will tell me that and now I'm left with a task of parsing a language with much more degrees of freedom (English), without any support from the IDE. It becomes even harder in international teams.
It is so frustrating to hear when someone thinks commenting more and adding humour is somehow cleaning up the code. But to then hear that they are teaching this to juniors just hurts so much.
Simpler advice to juniors would be to read Clean Code by Robert C Martin, apply some of those techniques and then politely ignore “more experienced” developers who think writing humourous essays as code comments is a good thing.
I’ve even heard phrases banded around like “the more comments the better” - I mean seriously WTF.
I like the analogy of...
When in an unfamiliar city, having no map at all is much better than a map that you have no idea whether you can trust or not. Code comments are absolutely that untrustworthy map, doesn’t matter how many code reviews or convoluted PR process takes place, you still need to read the code to know the truth so more effort making the code communicate is the key to maintainable code. Simple things like good variable names and grouping behaviour into functions at the same level of abstraction can completely negate the need for comments.
Comments have their place, but should be the exception not the norm.
One of my pet peeves is JS projects. I am not sure why but the front-end developers refuse to add any comments at all. This trend, oddly enough, started with ES6. Pre-ES6, JSDoc style comments at least, were quite common.
Just because some popular Javascript project doesn't have comments doesn't mean yours shouldn't. Straight-forward code may not need comments but most of these projects definitely need to explain why or how are things supposed to work or why things are done a certain way.
Code comments usually are not the best place to elaborate on "why things are done a certain way". One can also write documentation about the architecture, design etc.
This is a feature of several (mostly functional) programming languages, e.g. Haskell. Fun to see that often people figure out that these types of concepts are a smart way to write your code. Too bad it usually means many people reinvent the wheel instead of learning about computer science history and other languages.
Functional programming advocats, especially for the "pure" ones like Haskell, always strike me as odd. It seems that all the beauty of those languages make people obsess over that beauty and purity while keeping them from being productive.
Meanwhile, people with simpler languages like Go just get stuff done that is useful and makes people happy. Now if I _ever_ came across a useful Haskell product, I'd be happy to test drive it, shouldn't be a problem by now with all the container technology. But the closest I ever came to using a functionally developed product was RabbitMQ (written in Erlang). That one was _such_ a pain to use and operate — must have been the developers still dreaming in the purity of its code instead of writing some installation docs. I moved on to Kafka later and didn't regret it a minute.
Rant off.
Functional programming, at its heart, is about using self-imposed constraints to avoid certain classes of programming mistakes.
If your application domain doesn't have big consequences for these classes of programming mistakes, then it can seem like functional purity can be a luxury or frivolous. However, if your application domain suffers greatly from those classes of programming mistakes, such as distributed systems, then it may be worth it to consider what functional programming might buy you.
So yes, just because you use a functional programming language won't help you sell your widgets or make a great product. And you can spend lots of time fucking around with it for its own sake and still not sell widgets or make a great product. However, if you understand what its constraints buys you, then you can make your job of building these things in certain domains much easier.
There is indeed a lesson here, but which one do you think it is? It's certainly not that McDonald's makes better (or "simpler") hamburgers: once you taste good hamburgers you can never go back to McDonald's (and yes, I make better hamburgers too, even though mine won't win any prizes!). To me, the lesson is that there's more to success than product quality. The established brand matters, the scale at which you can sell a (possibly inferior) product matters, how low you can get away with paying your employees matters, how well you can survive PR disasters matters, etc.
If you had to make burgers, would you rather make cheap and mediocre ones, or would you rather enjoy making premium burgers? :)
Kubernetes' reputation is just the opposite: that far from being a simple and useful thing, it's an overengineered, overcomplicated solution to a self-inflicted problem (deploying a distributed monolith).
> But the closest I ever came to using a functionally developed product was RabbitMQ (written in Erlang). That one was _such_ a pain to use and operate — must have been the developers still dreaming in the purity of its code instead of writing some installation docs. I moved on to Kafka later and didn't regret it a minute.
Erm, Kakfa was developed in Scala, whose advocates far more of a reputation for purist pontification than Erlang developers do. Maybe all that beauty and purity is actually good for something?
This disparity in numbers is in turn is primarily because golang/Python/C is inherently much more approachable than Haskell because the average programmer has cut his/her teeth writing imperative code for a significant portion of their early programming career. The jump to functional way of thinking requires a certain leap of the mind, that most people don't want to take the trouble doing because they seem to be happy "getting things done" in golang or Python.
However, that is not to say that we shouldn't be striving for correctness in code that Haskell fosters, or the inherent simplicity that it forces on you because you are forced to separate your IO and effect-full code from the pure bits. These are things worth striving for. These are broad principles worth emulating even when you are coding in an imperative language.
To turn your McDonalds analogy around: sure, a McD will let you just get your food requirements out of the way quickly and cheaply (i.e. just "get stuff done"). But in the long run, it's bad for you.
Haskell is a healthy salad to an [insert favourite imperative language] burger.
Err, RabbitMQ is one of the easiest to setup and stabler queues/messaging systems. And I'm no user/fan of Erlang...
That said, there do exist useful products written in Haskell. For example, Facebook writes a lot of spam fighting code in Haskell. https://code.fb.com/security/fighting-spam-with-haskell/
That said, lots of things are functional: chunks of Facebook, Twitter, and Microsoft (and I assume Google) are written in OCaml, Haskell, Reason, and F#. Jet is built in F#. Jane Street famously uses OCaml, and Scala is becoming the standard language for hedge funds. Spark apps are usually written in Scala. Etc.
There's at least git-annex and pandoc
Furthermore, why are you using the user interface as a yard-stick to judge language paradigms with?
Because I'm not in the business of making hamburgers nor am I interested in doing so.
Struggling to see this business coach's point. Is he saying that McDonalds makes better burgers than me because I'm not selling a million burgers a day? Because that's a load of crap.
- the Flow and Hack compilers
- Facebook's Messenger app
- Jane Street's entire trading infrastructure (and everything else they do)
- XenServer
- some parts of Docker, including the TCP/IP networking stack on OS X and Windows
Good examples of this translating to huge gains for the overall community are React + Redux.
I'm quick to admit that functional language ecosystems are not as mature, which might lead to lesser organizational productivity but no need to rat on functional programming in general.
https://pandoc.org
Much of theoretical physics is beautiful. Only when one takes it off the blackboard and into the real world does it turn ugly.
Haskell is a highly opinionated reasearch language. This makes it a great languages to talk about. People could make many of the same points with, say Scala or Ocaml, but because those languages arent as opinionated, it is harder to use them as a basis for discussion than a language which is heavily opinionated about the subject in question.
An example would be head, which takes the first element of a list and throws an error on an empty list.
I really love Haskell but it seems as if they are going for something different here than what Haskell provides.
In all seriousness though, Rust does get a little closer - it requires exhaustive matching (or explicit opt out) and deliberate error handling (or explicit opt out). There are still ways around it, but the happy path in Rust is handling errors.... well, maybe not happy. It is a little verbose.
That said, Haskell still isn't actually an example, chiefly because exceptions can be thrown by any code, and are reasonably often used in practice.
Deleted Comment
Unfortunately the odds of this ending up in a mainstream language this decade is pretty low: the extreme focus on terse code and DRY means the average dev balks at the verbosity (thus the coment in the linked piece of code being necessary). It's a shame, as it's objectively superior by many metrics.
This is just one of a common practice of programming, not a feature of functional languages. These practices are not even "reinventing the wheel".
I mean, this is obvious in many areas: from implementing complex logics like Kubernetes to making hardware drivers in C. Programming languages themselves can't automagically ensure every single conditions because these often happen outside of the program (e.g. targeting hardware state). We need to cover and test all cases by hand anyway.
On one project I work with guy with significant railway signalling experience and this is one of the issues that I'm somewhat unable to explain to him. Probably because our product's design target is not to fail-fast-and-safe but fail-secure.
I didn't see that.
In fact this speaks to me as mission critical software like this needs to be as tediously documented as possible to eliminate surprises. Those branches and conditions are collected through a huge pool of trial-and-errors, implying Haskell can provide those valuable use cases out-of-box is misleading, no it can't.
Model checker like TLA+ or Pluscad might do the trick.
Deleted Comment
Not only does this add the stricture of a compiler enforcing correct return types for all conditionals, it is also idiomatic of any language that supports this.
Having said all that, changing languages is rarely an option. Maybe this is the best solution given the tool available. If so, it demonstrates that maybe this tool isn't right for this job.
What in this <2k LOC file suggests strict types help anything whatsoever? It's not massive. It's large, but fold this and it immediately becomes more readable. Any attempts to simplify this would simply be reasoning about it differently.
In this case right here, what's your counter-example, or what would you use instead of their specific approach?
No, or at least not often enough to be worth thinking about. It is of course possible to use abstractions badly, but the problems that business software has to solve are always fairly simple because they're human problems; human business processes were never that complex. So if a program looks really complicated, the overwhelmingly likely cause is failing to use appropriate abstractions.
> In this case right here, what's your counter-example, or what would you use instead of their specific approach?
As others have said, a result type would greatly simplify this code without sacrificing any safety. No doubt after such a simplification made the code shorter and easier to comprehend, further simplifications would become apparent.
The more verbose and detailled the comments are - the more work needs to be spent in ensuring that they are correct.
FTFY: …hopefully!
Only if they’ve used a language with Algebraic Data Types support, the compiler would enforce that “every branch and condition is considered and accounted for.” The only PL with ADT that I’ve used was Haskell, but I’ve heard that Rust has them too “enums”.
People are arguing that “code is what computer executes, comments don’t ensure anything!” and so on, but besides being executed, (this Go) code does not ensure anything either. It’s human eyes that skim through all the cases, and look for a matching branch for every one of them that “ensures.”
In my humble opinion, this so-called “space shuttle style” is just one of the many workarounds to deal with Go’s by-design limitations (the most famous one being lack of generics), a language that’s designed only 9 years ago.
This a thousand times. The praise in this thread is disturbing.
The absurdity of this code is the logical conclusion of ignoring decades of PL advances in favor of Go's "simplicity."
When you insist on "space shuttle" era language design, is it any surprise when you're reduced to "space shuttle" era programming? I can't imagine anything more fitting.
There are no perfect languages, that is certain.
Two years ago I’ve moved one of my biggest projects from Python 3 to Go because my program was inherently concurrent, and at the time -and I think still- Python has 3 competing approaches for concurrency: threading, multiprocesses (for parallelism), and asyncio.
Although it’s nice to have a variety of options, I think this balkanisation affected the community in negative ways because the options are not compatible with each other (see the emerging “SansIO” libraries for asyncio).
This is actually the reason why Python gets so much praise: thanks to its standard library, there is often a single canonical way to do something (if you need sets use `set`, if you need matrices use NumPy, …), which means that the vast majority of libraries are interoperable* with each other. Same goes for Go when it comes to concurrency, and that’s what guided my choice.
*: high cohesion, low coupling
Worse, they've made an exception for simple error checking, and the result is that the majority of the if statements at the top of the file have no else condition. Quickly scanning by eye doesn't help me determine if someone screwed up and missed a scenario.
https://youtu.be/eEBOvqMfPoI?t=859
- "It's the "jazz music" of software development."
- "...breaks all the "rules" but does so purposefully..."
- "this is irreducibly complex, and cannot be split into multiple files"
- "that smallness-of-file or smallness-of-function is not a target to shoot for"
I am wondering: can't all the above statements be said in defence of any poorly engineered, gargantuan single page code?
Also, poorly engineered code is seldom irreducibily complex.
No.
Yes.
https://youtu.be/cRzDEdY8FL4?t=214