Readit News logoReadit News
OminousWeapons · 5 years ago
> If people are trying to assign blame to a bug or outage, it's time to move on.

This is one of my favorite excerpts. I once worked in a lab where we would have frequent catastrophic failures because there was never any disaster planning or contingency management plan. I personally triaged 3 such incidents alone or with people who happened to be there when the problem arose and attempted to disseminate some suggestions for how to prevent similar problems in the future. No one was interested. People were primarily interested in tearing my head off because I hadn't handled the problem the way they would have done it (of course, they were out drinking beers or sleeping while I was dealing with the issue at 12 AM or on a weekend).

After the third time I said fuck it, the next time there is an issue I am going to insure my own projects are safe and then I'm going home and turning my phone off. Let someone else deal with it. That is the not the culture you want to be promoting.

giantg2 · 5 years ago
I was on call as a new developer on a system. I was not given any procedures or trouble shooting documents. I got a call at 1 am, missed it, and waited one minute to see if there was a message. I did see a voicemail, so I started listening and logging on. Before I could even get halfway through, the person called again (why not leave the voicemail on the final attempt?). So I'm looking for the issue/fix for 5 minutes and they tell me they know who the SME is for the functionality, so they will call them. Why even call me if you're just going to call the SME without giving me time to look at it? I got negative feedback from my manager about the way I handled it. So, I asked how I should have handled it without any training or documentation. They said I should have called the SME. Well, I didn't know who the SME was and there's no documentation or list of what who is the SME for which part of the system, nor was I instructed to immediately call the SME. Again, why not just call the SME first if they knew who it was and the SME didn't create documentation because they are "too busy".
tolbish · 5 years ago
How was the interview process before you got hired on? Any warning signs that seem obvious now in retrospect?
mr_toad · 5 years ago
> SME didn't create documentation because they are "too busy".

Because they keep getting shoulder tapped to put out fires. Because they’re the only one who knows the system. Because there is no documentation…

lazyant · 5 years ago
Quoting what a HN user said in another post a few months back "if a person can break a system, the system was broken to start with".
kbenson · 5 years ago
Eh, that's a nice thing to say, but it only makes sense at certain scales, and no matter what, there's always a person that can break it.

If any random person can break it, it's already broken.

If any employee can break it, it's probably broken (there are very small scales where even this doesn't apply. Ever worked for a company with less than ten people? There's probably something any employee can break).

If any employee that's an engineer, sysadmin or developer can break it, well now you're at least reducing the problem to a more specific set of people.

If only the people on a specific team responsible for a system can affect the system, now you've reached fairly good point, where there's separation of concerns and you're mitigating the potential problems.

If only a single person can break the system, you've gone to far. That effectively means only a single person can fix or work on the system too, and congratulations, you've engineered yourself into a bus-factor of one. Turn right around and go back to making sure a team can work on this.

Finally, realize that sometimes the thing only one team can break is an underlying dependency for many other things, and they may inadvertently affect those. You can't really engineer yourself out of that problem without making every team and service a silo that going from top to bottom. Got a shared VM infrastuture, whether in house or in the cloud? The people that administer that can cause you problems. Don't ever believe they can't. Your office IT personnel? Yep, they can cause you problems too.

Some problems you fix by making it so they can't happen. Other problems you fix by making it hard to happen and putting provisions in place that mitigate the problems if they do.

ctvo · 5 years ago
What systems are you working on? Many are held together by ritual, and deviating from the ritual causes outages. They’re very fragile in some form (deployment, change, infrastructure, dependencies, etc.). They won’t break if you follow the happy path, but to say they’re so robust that an active attempt at breaking won’t bring them down is ... naive? Not sure if that’s the word I’m looking for.

I say this as someone who’s worked at large tech companies that are “internet scale”.

lostcolony · 5 years ago
Maybe. I've seen the opposite, where no one takes responsibility for anything, and it's also bad. In fact, the situation you describe could also be a lack of anyone else taking responsibility for disaster planning and etc.

I think what is needed is a culture of -ownership-. That's basically people saying "I'm responsible". Not one where everyone tries to avoid responsibility, and not one where peopel point fingers.

thethethethe · 5 years ago
Why does someone need to take responsibility when you can have a culture of blameless postmortems where everyone focuses on making sure what ever happened never happens again instead? In blameless postmortem culture, everyone is responsible by default
vbezhenar · 5 years ago
What does it mean to be responsible? Just to say it? Responsibility should be accompanied with fines corresponding to the damage or something like it. Otherwise those are just words. I'm responsible, but I'm not getting any fines if something goes wrong, so whatever, but I'm responsible. Fire me if you want, I'll find new work in a few days, but I was responsible.

It's business owner who's responsible, because ultimately he's getting all the expenses when critical event happens, client leaves, client sues the company, and so on. Other people are not really responsible, they just pretend to be.

nojokes · 5 years ago
So how did it play out? Please, do not let me hanging here.
OminousWeapons · 5 years ago
Honestly, I don't know; I unknowingly followed the author's advice. About half a year after the last incident, a friend who I went to school with called me up and offered me a job at his fledgling biotech. I accepted and never looked back.
frereubu · 5 years ago
This, and the current “sober” posts on r/ExperiencedDevs, makes me think of Herodotus describing the way the Persians made important decisions - once sober, once drunk, and if the drunk and sober decisions were the same they knew it was a good one.
gautamnarula · 5 years ago
Reminds me of Hemingway's "Write drunk, edit sober."
edoceo · 5 years ago
Not for code tho. I've never seen booze help make good code
rewgs · 5 years ago
I find “write sober, review drunk” to be much better advice.
nojokes · 5 years ago
Write code drunk, write tests sober?
TwelveNights · 5 years ago
The Persians also have an expression for "drunkenness and truthfulness", masti o rasti. مستی و راستی
dhosek · 5 years ago
There’s also Pliny the Elder, in vino veritas (in wine there's truth, for the non-classicists in the audience).
wiml · 5 years ago
I think Tacitus had a similar anecdote about the Germans.

(Even without getting drunk, I've always found it useful to consider a hard decision once analytically and once intuitively, and if I don't agree with myself, think about it some more.)

rcpt · 5 years ago
https://en.wikipedia.org/wiki/In_vino_veritas Is the phrase. Useful if you want to appear cultured at a company party.
frereubu · 5 years ago
This is right for the drunken dev thoughts, but not for the Herodotus example. "In vino veritas" - in wine there is truth - means that people expose their true thoughts when they're drunk rather than the filtered version they might present when sober, but the story of the Persians is more about the fact that there's value in considering both drunk and sober reactions, particularly when they tally.
Demeno · 5 years ago
In Hebrew we have "נכנס יין - יצא סוד", loosely translates to "Wine entered - secret came out".
lazyant · 5 years ago
or "children and drunks tell the truth"
vageli · 5 years ago
> The most underrated skill to learn as an engineer is how to document. Fuck, someone please teach me how to write good documentation. Seriously, if there's any recommendations, I'd seriously pay for a course (like probably a lot of money, maybe 1k for a course if it guaranteed that I could write good docs.)

I agree but think it is more than just _documentation_: effectively communicating ideas through text was one of the most underrated skills in software engineering. I say "was" as I think there is much more focus on it now with remote work and video call fatigue becoming the norm for many.

NortySpock · 5 years ago
The first thing to learn is that there are four types of documentation:

learning-oriented tutorials

goal-oriented how-to guides

understanding-oriented explanations or discussions

information-oriented reference material

https://www.writethedocs.org/videos/eu/2017/the-four-kinds-o...

0xbadcafebee · 5 years ago
I would suggest you avoid thinking in too general of terms like this. There are dozens of kinds of docs. It's better to think about the document's specific purpose, audience, what you need to convey, what the audience wants to know, and how they want to absorb it. Then write, then read it as that intended audience, see if that person can make sense of it, and if it provides enough information. If you can't put yourself in their shoes, have the audience proofread it.

Two important lessons I learned:

1. Formatting and direct communication is very useful. It can make the difference between someone stopping and noticing critical information, or skipping it because they're lazy readers.

2. You probably don't know the correct way to convey information, and the audience probably doesn't know how to tell you how to convey it either. You need to listen for when the docs fail: when somebody says they read the docs but still don't know something or don't do something right. That means your doc has a "bug" in how it gets through to the reader. Experiment, change things around, add/remove information, etc until the bug is gone.

mikepurvis · 5 years ago
Agree with the sibling comment, but a starting point is also just to write docs for your future self, which is usually going to be type 3 or 4. Most of us who have been programming for a few years have had the experience of being mystified by something we ourselves wrote in the past, so it eventually becomes fairly straightforward to predict what kinds of things future-me will need a hand in piecing back together.

And it turns out those kinds of docs are pretty useful to my colleagues in piecing it together also.

themulticaster · 5 years ago
In contrast to the current sibling replies, I think this is a very fitting categorization of documentation. Off the top of my head, I can think of several examples where one type of documentation is excellent but others are very lacking, for example:

* Rust Library Documentation: Most libraries have complete and up-to-date reference documentation, but are lacking even basic introductions (tutorials/guides) on how to use the library. This is totally just my personal experience so maybe I've been looking at the wrong crates, but with most of the crates I spend several minutes looking trough all the modules in order to find that all the juicy functions are hidden in the Connection struct, or something similar.

* Linux Kernel Documentation: The Linux kernel has excellent in-depth explanations on several high-level concepts, but on the other hand a little more systematic reference documentation on the supporting library code would help a lot.

* While I can't think of a good example right now, a lot of projects have a few basic getting-started tutorials but don't explain advanced concepts or high-level design at all, leaving you to wade through the sources yourself in order to understand how to actually use them.

dehrmann · 5 years ago
I end up reading the source half the time, anyway; documentation is often incomplete, dated, and possibly incorrect. For code, I'd prefer the time go into designing a cleaner interface and making what calls do obvious.

That said, I find high-level documentation for larger systems to be very valuable. I also find Python's docs to be lacking compared to Java's; I'm often left wondering about the definition of what type is returned, exactly what parameters should be, and which exceptions are raised. Java's docs are very explicit about all these things.

treis · 5 years ago
>end up reading the source half the time, anyway; documentation is often incomplete, dated, and possibly incorrect.

I'm a pretty firm believer that documentation has to live next to the code. Otherwise it's nearly guaranteed to be out of date and/or incomplete

npteljes · 5 years ago
I agree. I'd like my documents to contain overview, intent, and exceptions. The actual implementation I can look up in the code. Also generated stuff is appreciated, like Swagger.
ahaferburg · 5 years ago
You should be reading the source all of the time. The point of documentation is to tell you what the code doesn't. If it tells you the same, you should delete the documentation.
ChrisMarshallNY · 5 years ago
I just use headerdoc-type stuff. Been doing it for decades.

It works very well, and doesn't really add any overhead to my work.

I write about how I do documentation here: https://littlegreenviper.com/miscellany/leaving-a-legacy/

It's a long read, because it's a big topic.

GordonS · 5 years ago
That's useful, but only covers documenting the code, and API usage.

Depending on the project, various other documents may be required, e.g. installation guide, user guide, operations manual, architecture diagrams, networking diagrams, module/component diagrams, information flow diagrams, high-level design, low-level design, docs at various "views" (such as "business view", "information view", "technology view"), design decision tracker, ontology...

hnedeotes · 5 years ago
One can hope, but sadly I don't think it's going to be an easy shift.

People managing work, from what I've seen, still prefer to babble over their scribbled 5 basic points than taking the time to do their job and create relevant textual information. Then you listen, you take notes, and then you go and produce whatever documentation of the objective is required to at least understand if it's going to work. Of course you still will have gaps in your understanding, so then more calls, and repeat. In the end, unnecessary/missing features, a whole bunch of time wasted in crap, deadlines missed, burnt time, all of which could have been avoided if someone just had taken the time to do their supposed job - this is not to say it wouldn't have to be discussed, or that there's no need for back and forth and calls, etc, it's just instead of starting halfway you start from -50% or something.

Even in outsourcing platforms there's been a shift contrary to that. At least two years ago and before that, video calls or calls weren't really usual unless you were in some months long collaboration - now even there everyone expects video calls on the interview... It doesn't matter if it's a $100 one time job or whatever.

Tempest1981 · 5 years ago
I've seen lack-of-documentation worn as a badge of honor. "I'm moving so fast, I can't waste my time on documentation. That's the next guys problem."

And management is usually/always ok with this short-term optimization.

b3morales · 5 years ago
Also "if the code is clear, it documents itself". Which in my opinion completely misses the point. Good documentation doesn't tell you what the code is doing, it tells you why it's not doing something else.
withinboredom · 5 years ago
We had a guy that always opened PRs with no description. Guy always said “read the code”. Manager wouldn’t do shit and he kept doing it until we all just stopped reviewing his code and then he couldn’t merge.

Like dude, tell us why we should read the code in the first place.

dijit · 5 years ago
But there are circumstances where documentation would be outdated before it is finished.

Trying to write ops documentation on an unfinished project can be this way.

mancerayder · 5 years ago
Exactly. That's not a documentation problem, that's a writing problem. A lot of tech people don't enjoy writing, but also they're sometimes not very good at predicting or empathizing with the future reader of their writing. Sometimes that's also manifested in speaking, and failing to context frame concepts, arguments and ideas before jumping into excruciating detail. Senior managers notice this, and this limits your career.

So I argue that the issue is writing skills, of which technical documentation is a subset speciality of writing skills. I will add, similar to math problems or programming, writing wants you to do it over and over so it can get better.

routerl · 5 years ago
> A lot of tech people don't enjoy writing, but also they're sometimes not very good at predicting or empathizing with the future reader of their writing.

Agreed, wholeheartedly. Will hitchhike on your comment to recommend two things: Brett Victor's pdf stash [1] and, specifically, the Walter Ong essay "The Writer's Audience is Always a Fiction"[2].

Long story short, we form our audiences by subjecting them to our writing. In writing software documentation, we are implicitly informing the next generation's thought by the simple power dynamic that underlies all technical documentation: "you must understand this in order to do your job properly".

It is no wonder that "form", "inform", and "information" are such closely related words.

We dictate the level of rigor and intelligibility we expect out of our technical documentation, when we write technical documents. It almost sounds like a tautology when put this way, but "bad docs" are exclusively the result of a professional culture that puts up with the existence of bad docs. I've been there; too tired and overworked to care about writing something properly, or wanting to avoid writing a doc enough that I setup some autodoc thing and called it a day. We literally don't get paid for writing documentation.

But good documentation is what made us into good developers (if we are good developers). We should get paid for doing that...

[1] http://worrydream.com/refs/

[2] http://worrydream.com/refs/Ong%20-%20The%20Writer's%20Audien...

jjice · 5 years ago
> effectively communicating ideas through text was one of the most underrated skills in software engineering

Absolutely. So many hour long meetings could be shortened by better communication skills (just more targeted), or even a small email chain.

Communicating in short form confidently is a skill. Many people, including myself (something I've been working on) struggle with saying something that was a complete idea in a meeting and not stopping because they feel like they need to say more. Short and sweet is the way to go pretty much whenever you can, for technical work that is.

tharkun__ · 5 years ago
And many many people don't do it well. On both ends. Reading is a skill too.

I find 'small email chains' don't help at all. Too many people just don't read past the first sentence or paragraph. And email is slow.

I usually 'escalate' quickly. Support didn't understand a ticket comment I made on how this isn't a bug or why there really is a workaround for it?

I send them an IM trying to coax it out of them/get them to understand for a few minutes. Doesn't work? Quick call and do screen sharing. Problem usually solved after a few minutes. Problem is they might stay longer than that took just to 'catch up' ;)

This is really not that different from when we were all at the office just that the last part would be walking over to their office and looking at the computer together. In some situations that makes it even easier nowadays because I don't need to take an elevator down 20 stories and back up again after.

Having asynchronous means of communication is great. But as soon as the back and forth is more than maybe twice on each side, there's probably a miscommunication happening somehow that will be much easier to get solved with a really short feedback loop. Some people you have to call right away coz they just type soooooo slowly ;)

ridethebike · 5 years ago
Not only in writing but also in plain English. I see situations like described below over and over.

Example 1 - too much details. Morning stand up. Manager: what's the status of that new feature. Senior Engineer: well I tried to call that service but it was timing out so I spoke to Bob and he said to check with DBA on why that stored procedure but it's so slow and turns out index is missing so we tried to add it but mysql and varchar something fckn something...

Dude couldn't you just tell it's delayed due to DB and then expand if needed.

Example 2 - insufficient details I return from the meeting and discover avalanche of emails, chat messages and urgent meeting invite, all with same topic - "Blah service fails, we are blocked" but no details apart from that. On the call I get description of the problem - blah service fails and how everyone is blocked and how infinitely critical it is and what ETA for resolution would be. What endpoint? How does it fail - timeout, connection aborted, 503 response, 200 response but with error message?

arendtio · 5 years ago
I like documentation that starts with a minimal but functional example, followed by the most common additions to improve the solution and finally a complete documentation of all functions.

Having links to actual code, like GoDocs have it, is something I appreciate too.

personjerry · 5 years ago
Effective communicating is a crucial part of every job. I think in software engineering, a lot of us are introverts who want to deprioritize this soft skill, but the truth is still that people matter more than the code.

And communicating gets even more important the higher up you move in your seniority.

ljm · 5 years ago
Documentation can also be anything. Is it your initial RFC or ADR, or is it a spec? Is it a set of product requirements? Is it inline with your code so it produces a manual when you build it? Is it a set of articles or tutorials written after the fact? Is it a README?
jtouri · 5 years ago
Effectively communicating through text amplified due to WFH. If you can avoid syncing up on video chat and resolve something with a couple of back and forth messages. That's a win
tobesure · 5 years ago
I find documentation to be relatively straightforward to write. The issue for me is sitting down and slogging through the activity, which seems almost antithetical to writing code. It's like doubling the work in a far less rewarding way. And then you have to go back and update it any time the code changes. It's a sort of necessary evil I suppose...I think most of the problem is that devs just can't be fucked to take the time, and I'm often guilty.
daenz · 5 years ago
>Don't meet your heroes. I paid 5k to take a course by one of my heroes. He's a brilliant man, but at the end of it I realized that he's making it up as he goes along like the rest of us.

I thought they were going to go the direction of "he's an asshole" and was ready to accept that, but this particular criticism is actually disturbing. People with strong visions can often appear to be "making it up as they go along," when really they are just subpar communicators.

Short story, I am helping my current company switch from a monolith to service-oriented architecture, and in the process have built a framework for spinning up new fully-deployable services from scratch that gets engineers 90% of the way there (minus the business logic). I have a strong vision for how it works and had a dozen-page RFC accepted by the engineering team for it. Yet there are engineers who think I am making it up as I go (I have been asked this indirectly), without any vision guiding the pieces into place. I have chalked up this feedback to me needing to improve my communication of the vision.

So the post's response of "I realized that he's making it up as he goes along like the rest of us" is disturbing because it makes me realize just how difficult communicating a vision is... if this hero that the poster paid $5k to go see can't even convince one of their fans, what chance do normal people like you and I have of convincing people that we're not making it up as we go?

>EDIT: I realize that I am posting under the assumption that the person's hero does in fact know what he's doing. If he truly is making it up as he goes, the above doesn't apply.

swader999 · 5 years ago
What is wrong with making it up as you go? I mean everything I've ever built I had a notion of what I was doing but most of the real work was in the details. Anyone could say I was making it up as I go. I'd be like yeah, if I knew completely how to do it, I'd already be finished it.

Then there's the times where you think you know exactly what you're doing and after going down a road you realize it's the wrong way. Failing to learn and see the signs and make the embarrassing declaration that you got lost and need to turn around is never good. But some people just keep driving. It's hard though when there's a big line of cars behind you that think you actually know where you are headed.

daenz · 5 years ago
Nothing wrong with making it up as you go, and I didn't mean to sound like I was knocking it, if I did. Sometimes everyone fumbles around trying to find solutions that work...it's a totally valid way to approach some problems.

Sometimes it's a hybrid of knowing what you are doing but not knowing the implementation specifics. You know you need to connect high-level pieces A, B, and C with specific constraints, but it won't be until you get into the low-level implementation that you'll know if it is indeed possible. I think that's an example of both having a vision and also improvising as you go.

I am concerned about how to effectively communicate visions to people, because it gets everyone rowing in the same direction. If nobody thinks that you have a vision, when you do, there is no reason they should choose your direction vs just do their own thing.

jtolmar · 5 years ago
> What is wrong with making it up as you go?

Nothing if you're good at it. But if you're hoping to learn something from someone, it is pretty disappointing. How to make it up as you go along is far less teachable.

the-pigeon · 5 years ago
It really depends.

I take this really just to mean "everyone has faults".

People often idealize heroes and think of them as beyond human. If you do that and met your hero then your illusion will often be shattered. But the problem is just that you were putting them on an unreasonable pedestal.

Of course some people are frauds and some people have no idea what they are doing but manage to make people think they do. But I didn't read this as being one of those situations. Just someone they saw as beyond human being only human.

bitexploder · 5 years ago
I like the phrase “kill your heroes”. Not literally, of course. But in your mind. They are just flawed people like everyone else that happen to have been mythologized. Learning more about your heroes often leads to disappointment.
tester756 · 5 years ago
I have different take on this

I "know" guy who's conference speaker, so he knows other conference speakers, drinks vodka with them and so on.

He says there's significant amount of bullshit aka things that work nice on slides, things that only work cool in theory, but in practice they arent as great

I think that's what OP meant

booleandilemma · 5 years ago
A common meme I've seen recently is "no one knows what they're doing".

I think people like to believe this because it helps them cope with impostor syndrome, or maybe they think it puts them on even ground with people who do in fact know what they're doing.

mortenjorck · 5 years ago
My model for people who "know what they're doing" is that they tend to have a well-organized hierarchy of rules. At the base are principles; at the top are opinions.

The foundation tends to be pretty simple, deeply held, and unchanging, while the higher levels are increasingly fluid and specialized. The higher you get on this stack, the more "making it up as you go along" it becomes, but every improvised part is perched on something more stable.

They key to "knowing what you're doing" is organizing this hierarchy well, having the right supports in place to successfully guide improvisation and course-correction while steadily fortifying the foundation.

WJW · 5 years ago
It's clearly not true in all circumstances. You can bet that an airline pilot has a very clear idea of what they are doing, and so will your dentist. Closer to home there are plenty of sub-fields in software where I'd be completely lost but when (say) we have to add a new endpoint to the webservice I work with, I absolutely don't have to make it up as I go along.

Your assessment of why people like to believe this seems spot on.

andreilys · 5 years ago
Even worst is the meme that “programming is just copy+pasting from stack overflow”
toss1 · 5 years ago
The criticism may depend on different values of "making it up as you go along", i.e., it may not mean so much "just wing it in ignorance" vs something like "even if you have many answers you don't yet have all of them, and new answers generate new questions exponentially...". So, perhaps less "everyone's ignorant" vs "we're all living in a land of many unknowns". But, yeah, he did find it disillusioning, and maybe is over-generalizing from a one-off experience (in contrast, I've done similar and was nothing but impressed, finding it is extremely valuable to learn from the best in a field).

How do you distinguish between what you do and "making it up as you go along"?

staysaasy · 5 years ago
IME most people will generally appear to be making things up as they go – even if they have significant relevant experience. Every situation is unique, and experience tends to look more like having a list of techniques with varying degrees of expertise, rather than having a playbook for every situation. You have to look for the expertise rather than raw confidence.

In sports terms it would be something like a baseball pitcher being able to throw a great curveball, a great fastball, and an all right slider, and knowing roughly what situations to use them in. There will still be a high degree of randomness and mistakes will be made.

antod · 5 years ago
Agreed. What experience and talent gives you are instincts that improve the chances of whatever it is "you're making up as you go" working well.

I would much rather work with people who have a good track record of making it up as they go as opposed to people coming in with a fixed idea of how something should happen and are more likely to misapply whatever lessons led to those views (probably someone elses anyway).

karmakaze · 5 years ago
> built a framework for spinning up new fully-deployable services from scratch that gets engineers 90% of the way there (minus the business logic)

I'm guessing that this was based on first-hand experience building such services and witnessing engineers struggle getting new services up. And not so much that you've had specific training or past experience in developing bootstrapping frameworks. This would be my definition of making it up as you go and is great way to do it. Another way is learning how to make bootstrapping frameworks and applying it wherever you can which doesn't go as well.

dehrmann · 5 years ago
Related thing I'd add: be very careful about taking a dream job; I've seen this happen a few times--it's likely to disappoint. Also dating a minor celebrity crush.

Deleted Comment

tomnipotent · 5 years ago
In your particular instance, I would have collaborated with a single team to work on converting a single service over to the new framework. Once some success was made, it would be much easier to make traction with other projects and teams. Also, a vision or a plan doesn't mean you're not winging things as you go.
thisismyswamp · 5 years ago
Building a new framework should be way at the bottom of your list of things to consider. If you do, please make it a blackbox. It's tiring how many details one often needs to get into before being able to do something they could have summarized in a sentence the whole time. But this is a general issue!
rwmj · 5 years ago
Also which "hero" is charging 5k for courses and what form does the course take?

(I work at Red Hat, and my programming heroes are also my colleagues.)

tayo42 · 5 years ago
I wonder if it was a course at something like a conference? I think those are expensive. I can't imagine anyone is charging 5k that seems absurd
ww520 · 5 years ago
It just means don't blindly trust the experts.
gher-shyu3i · 5 years ago
The problem is idealizing such people in the first place. Sure, they may have had incredible achievements, but they're humans at the end of the day.
OJFord · 5 years ago
> Hacker News and r/Programming is only good to get general ideas and keep up-to-date, the comments are almost worthless

That's a weird one. I don't know anything about that subreddit, but HN comments are frequently great. I submit stuff because I want there to be HN comments on it for me to read. I typically read the comments first and only bother opening the link if they were interesting.

bombcar · 5 years ago
See: https://danluu.com/hn-comments/

>HN comments are terrible. On any topic I’m informed about, the vast majority of comments are pretty clearly wrong. ...

>And yet, I haven’t found a public internet forum with better technical commentary. On topics I'm familiar with, while it's rare that a thread will have even a single comment that's well-informed, when those comments appear, they usually float to the top. On other forums, well-informed comments are either non-existent or get buried by reasonable sounding but totally wrong comments when they appear, and they appear even more rarely than on HN.

ksec · 5 years ago
Pretty much this. HN is still gazillion times better than everything else on the internet. And that is excluding the absolute gold comment from members that were part of those battle stories.

It is also a reason why I dont want to mention or see HN links in mainstream media. Although I think most reporters sort of know this as well and tend to not mention or link to HN as source.

seoaeu · 5 years ago
Obviously HN commenters like us are going to think that. Honestly, I bet even YouTube commenters would say the same about their own community.
runeks · 5 years ago
That’s not obvious to me at all. I comment on both YouTube and HN, and I don’t feel I’m part of any “community”. I just think the quality of the average, highly upvoted HN comment is about 100 times greater than the equivalent YouTube comment.
whateveracct · 5 years ago
I frequently bookmark both an article + its HN thread when the comments are full of related opinions, info, and links.
___luigi · 5 years ago
You should check bitcoin early-days posts
baby · 5 years ago
R/crypto has a great community fwiw
vsareto · 5 years ago
>I don't know why full stack webdevs are paid so poorly. No really, they should be paid like half a mil a year just base salary. Fuck they have to understand both front end AND back end AND how different browsers work AND networking AND databases AND caching AND differences between web and mobile AND omg what the fuck there's another framework out there that companies want to use? Seriously, why are webdevs paid so little.

Full stack compresses two jobs into one. It's purely for cost-savings. They're paid so little because companies revert to "well, you can still only do 8 hours, so you do half as much of each", but really that's just them trying to weasel out of paying for knowledge. They also blur the lines by putting full stack along-side other devs, even though the other devs may not have invested the same time to gain as much knowledge as full stack.

When you take a full stack job, you undervalue your knowledge (and the time invested) and are selling it for roughly half of what it's worth.

titanomachy · 5 years ago
I've been a "full-stack" developer at large tech companies, and my experience is there at least it means "frontend developer who can put together a basic API server". My fellow full-stack developers and I would spend most of our time building out frontends, which was generally regarded by others as challenging and specialized work, and maybe 20% of the time adding API endpoints to fetch or update some data, which was considered straightforward.[0]

Not having to wait for some other engineer to make the backends made us a lot more efficient. It definitely was rewarding to be able to complete products end-to-end.

Hiring standards and pay were the same as for any engineer, at least in FAANG.

[0] Yeah, occasionally we had to optimize some SQL queries or whatever but we're competent engineers, we can figure it out even if it's not what we do every day.

optiomal_isgood · 5 years ago
> my experience [full-stack] means "frontend developer who can put together a basic API server"

This is 100% accurate in my experience too, and it's also true in the other way around: "full-stack" means backend developer who can put a basic SPA using React/Vue.

From the frontend perspective: they call themselves full-stacks for knowing how to spin up a NodeJS HTTP Server powered by Express with MongoDB inserting JSON in the database. But they are missing:

- AWS/cloud computing: not necessarily creating the infrastructure (although that's a must on more senior levels), but how to orchestrate the different components together.

- databases: why SQL/NoSQL, beyond basic SELECTs, knowing how to properly add indexes, debug why queries are slow, modeling, understanding locks implications and transaction levels, and so on.

- tooling: how to set up a bundler, linter and formatter, testing, CI/CD. This overlaps a bit with the responsibilities of the DevOps engineer, but a full-stack should know at least on intermediate level all of those things. I can't say how many times I've seen "senior full-stacks" that had no clue about how webpack worked at all.

From the backend perspective: they call themselves full-stacks for knowing how to spin up a React/Vue app that does basic CRUD operations using forms, powered by a UI framework like Material UI. But they are missing:

- CSS: most will find CSS hard/annoying and won't bother understanding at all how it works even on a fundamental level, will defer to hacks most of the time to make things work, especially when it comes to adjusting for edge cases like responsive design or cross-browser support.

- the DOM: normally they don't understand it at all, or to a very limited extent.

- Web vitals: how to measure and makes things faster and performant—not really including here overly optimized, but just making sure your app is 60fps or close to that most of the time. Usually when things get slow either on the network side or in the app itself, those engineers will blame is the framework/library, not their misusage of it.

--

Those lists are definitely non-exhaustive, as I didn't even mention more advanced stuff like protocols (how HTTP works? most can't answer), caching, etc etc, but you can get the point I'm trying to make:

The problem with the term full-stack is that only very few engineers really are sufficiently great on both sides of the stack and could say that they mastered both, simply because there's just too much to learn! Frontend has become so much more complex with SPAs compared when it was about rendering static HTML with some CSS and basic behavior with jQuery. Same for backend with the advent of cloud computing and several types of databases.

I've been coding professionally for a decade and I've met only a single engineer that I'd consider him full-stack (he checked all those boxes I mentioned and more). I think I would also include myself, because I spent 50% of my career spent as a front-end engineer, became senior, then I transitioned to back-end engineering because it pays the same or more and it's less stressful (for most of the regular companies that most of us work at). My current title is "principal full stack engineer", but in practice I only do backend/devops, I don't actively code front-end but I keep up with the industry by following the new trends and testing things here and there in personal projects.

Ultimately, I believe for being a full-stack engineer you have to be first a front-end (or back-end engineer) then learn the other, what we have today is most people doing the same from the very beginning of their careers and they either go deep on a single one or in none of them.

mrweasel · 5 years ago
There's also an issue with how many full stack web developers who are actually capable of doing all the things he lists.

My experience is that at some scale it works out okay, but beyond a certain point it just falls flat for most. We deal with insanely talented developers, who will trash a database, because it don't understand how it works. Talented JavaScript developers, who don't really understand how HTTP works... or load balancers, or caching... or webservers. Sometimes you get these fantastic software machines as deliverables, complex, you can't monitor them, or configure much, and the it just implements a basic feature of HA-Proxy or Apache, but badly.

My point is that they should be paid poorly, because they fail to be excellent at every part of their job, but rather than: Yes, this should in most cases not even be a job title. If you find someone who can do all of this well, you almost can't overpay, but are you really sure that you want to tie everything up on one person anyway?

risyachka · 5 years ago
It is not 2 jobs in 1.

Full-stack developer knows some frontend, some backend, some sql. They are paid good money because they are convenient, not for their knowledge. A dev who does only one thing knows way more about it that a team of full-stack devs.

Full-stack devs earn a lot and will probably earn only more in the future.

This is mostly a result of tech advancements like cloud infrastructure, tools etc that takes all the hardest things from you - like managing a DB, implementing security, managing infrastructure, deployments etc.

You don't need deep experts because of it, generalists are perfect for quickly shipping new features.

temp329192 · 5 years ago
I would do "full-stack" over a decade ago, when there was less of the notion of "front-end engineer" (which still sound a bit ridiculous to me) - the front-end was mostly HTML and CSS. It was a good experience, to go from requirements gathering to the database schema and back to presentation - it helped me see the whole picture.
arduinomancer · 5 years ago
Why is everyone saying they’re underpaid?

You can make doctor level salary at the ceiling if you move to a FAANG as a full stack dev.

The fact that boot camps even exist shows that it’s not that difficult of a job.

Other industries boot camps don’t even exist.

b20000 · 5 years ago
there is plenty of other software engineering work that is way more complicated than full stack web dev, and there are a lot of web devs.
gentleman11 · 5 years ago
What common position pays better than full stack?
giantg2 · 5 years ago
"Sure, $120k + bennies + pension sound great, but you'll be selling your soul to work on esoteric proprietary technology."

It sounds great because it is great. I don't make that much and I work on boring systems.

I guess those numbers also explain why the author can recommend maxing out the 401k. People supporting a family on less than $100k don't have $19.5k per year to put into it.

alexeldeib · 5 years ago
You have a great point, but let's balance this out with a few of the author's other comments.

- OP uses the phrasing senior engineer.

- Never worked for FAANG. This is relevant because $120k + bonus/benefits is basically a FAANG new grad. Fairly normal for SV tech companies.

- Those numbers likely provide a solid standard of living, but as a senior engineer you are likely underpaid.

- Esoteric and proprietary knowledge means if you choose to go elsewhere, you will be at a competitive disadvantage compared to using industry standard tools. There are of course tradeoffs and lots of general learning that comes with experience, but all other things equal it's a disadvantage.

> I guess those numbers also explain why the author can recommend maxing out the 401k

Yes, but again fairly typical for the target audience I think. There are even startups trying to target this market to optimize the flow of money from salary -> 401k -> post tax contributions/megabackdoor -> other investments, brokerage accounts, etc., e.g. https://www.helloplaybook.com/

Just as a sanity check, BLS suggests the median SWE wages work out to ~$110k.

https://www.bls.gov/ooh/computer-and-information-technology/...

giantg2 · 5 years ago
I'm an intermediate dev, masters degree, 9 years experience, non-FAANG, higher cost of living area (not SV, NYC or NVA), and work with obscure tech and proprietary tools.

It sucks that I suck. Oh well.

ksec · 5 years ago
>This is relevant because $120k + bonus/benefits is basically a FAANG new grad. Fairly normal for SV tech companies.

There were a few post on HN recently about fresh grad asking for / being paid $200K and anything lower they felt they were feeling lowballed.

For those of us outside US we could never quite grasp whether something is true or not. Salary across the pond is just incomprehensible.

gabereiser · 5 years ago
This rings home for me as well. I remember my (now ex) wife saying she was going to quit working and be a stay at home mom the day I hit $100k salary. Effectively cutting it in half economically. I feel like folks who say these kind of recommendations also need to preface the fact that not everyone can afford to. I couldn’t afford to put anything in a 401k for years.
giantg2 · 5 years ago
My prior manager said something about everyone should be contributing the max. Easy for him to say as a manager... with a physician as a wife.
brundolf · 5 years ago
> It's not important to do what I like. It's more important to do what I don't hate.

This one has started to dawn on me. I'm never going to love my job as much as I love my personal projects, so a job that I don't get very excited about but doesn't drain my energy is better than one that I get somewhat excited about but does drain my energy (not that it's impossible to have both, but it's rare)

meowface · 5 years ago
This is a very pessimistic long-term view, in my opinion.

Life is extremely, infinitesimally short. If it's at all possible for you, you should try to spend as much of it as you can doing things you love. I know many people can't, but it's bleak to just give up and permanently settle, I think. (Especially if you don't currently have any dependents who rely on you; it changes the equation if you do.)

brundolf · 5 years ago
I don't look at it that way at all. I'm choosing to conserve my finite energy for the things that matter most. It's all about priorities: of course I would like to have a job that I'm both excited about and not too stressed about, but I can be happy and fulfilled with one that simply doesn't dominate my life (in terms of time or energy, because those are separate things), because my job is not my life. And being uncompromising about having a job that really excites you can force you to make all sorts of other sacrifices (time, stress, risk, maybe even things like money and lifestyle), which is where the prioritization comes in.
aiisjustanif · 5 years ago
This is a very western view on life.
tharkun__ · 5 years ago
This is a really really big one if you ask me.

It seems to me that many if not most people I meet fall into one of two camps.

The folks that just don't care at all and do the minimal amount of work they can get away with not to get fired. The other side is folks that care so much about it that they spend so much energy on as to ultimately be unhealthy for them and also making it worse for the rest of us (not taking vacation, unpaid overtime etc.

It is very rare to find people that care 100% while at work. But after ~40 hours each week that's it. Sure I'll stay later for that one meeting that's sorta important and needs to start at 4:30p.m. But you better not try that every day from now on. Want me to be on call for a fixed rate that has nothing to do with my salary? I can't log an hour for a 5 minute call at 1am and I can't take time off in lieu? Good luck getting me on that rotation!

sbarre · 5 years ago
I made this exact choice about 5 years ago and my life has dramatically improved.