My career record (US Navy) for cost savings was something over $50MM. Every time I did something I had to do PowerPoints, present to Flag Officers, etc. Hell, once I almost got hugely punished because I didn't let my boss take the credit (he had no desire to because he had zero idea what it was I even did).
Note that some of that was as a Lean Six-Sigma Black Belt doing Enterprise projects (I hate every single bit of terminology in that entire godforsaken sentence), where it was literally my JOB to save the DOD as much money as possible with the least disruption possible. Those were the absolute worst years of my career. That period of my career was my reward for just going rogue and fixing things that saved millions.
I'll echo the last part of that post: Beware of doing good things at work; the reward is rarely compensation and is usually more work for the same pay.
At the wrong gig, with the wrong bosses. I have the misfortune of being at one of those bad places now. I was at a much better place before, where if I went out of my way and saved a couple million, I'd actually be congratulated; boss would even let have the credit.
Never stay at a job where the culture is toxic. Even if the money is better. Nothing eats at the soul like being held down by feckless, petty careerists and control freaks.
I agree that getting out of a toxic culture is a great idea but you can also just check out, give them what they want and save as much money as you can. Especially if your goal is to eventually start your own company.
Ideally, you should be working for a company that pays well and that actually rewards you for doing things that are good for the business. But it does seem like virtually every company has some kind of dysfunction.
This. 1000x this. Even if it's a FAANG, do it, there's no shame. I had a good run at one, then the culture went to crap due to one of those long boring stories that still can't communicate how...easy it is for things to be awful, even when they shouldn't be.
When they are, just move on. Life's too short. Whatever blows you up with $2 million in savings would blow you up at $500K
I had an industrial engineering professor who did a full career in the military, focused on energy efficiently.
Paraphrasing, "The great thing about the US Government is it's so big, if the optimal solution would save 30%, but you find a suboptimal solution that only saves 29%, nobody will notice because you're still saving tens or hundreds of millions of dollars of savings."
One of his big projects was on energy efficiently in remote military camps. The cost of fuel per gallon was around $100 delivered to parts of Afghanistan. They had tons of portable generators running at ~20-40% of capacity. If memory serves, ~70% is most efficient. By some combination of building a small or connecting a few tents to a single generator, they were able to very significantly reduce the fuel required and improve the quality of service.
> The cost of fuel per gallon was around $100 delivered to parts of Afghanistan
This was one of the most deeply insane parts of the whole twenty-year war. Of course, wars have unlimited budgets, so trucking gasoline thousands of miles through hostile territory with occasional losses and then using it to aircondition tents was going to happen. While back inside the US teachers were having to buy school supplies with their own money, because only wars have unlimited budgets.
Sometimes I think of joining government org (3 letter agency) to find areas to improve $ efficiency on, or make decent direct contribution to US as a thank you for accepting me as an immigrant.
I'm an engineer in a FAANG, who worked directly in money flow, and have had experience in diverse areas where it could come handy.
Then I start thinking finding the right person to work with & right area to start at is probably 95% of the job, then give up.
You'd want to start with the United States Digital Service (USDS). They're basically a consultancy inside the executive branch to build tech solutions for the various 3 letter agencies. You basically do a "tour of duty" for a year or so (however long you want), tackling one of these solutions. And they go out of their way to hire folks like you, from FAANG and other tech companies, so you work with good people.
As someone who does this sort of work (but posting personally, not officially), you're 100% right. There are groups of already-networked folks you can join like USDS or 18F that are already connected to the problems. The post reminds me of some of the stuff you might see in gov, though I've never seen someone save money and be punished like this.
There are a lot of ways to make an impact in National Defense outside of direct government employment. It’s tricky to find a good contractor, but they exist.
The direct employment option might get better with the recent executive order on AI. USG pays market wages for VA doctor/s; eventually they’ll do the same for tech roles.
I'm so triggered by this phrase. The worst (to work for and performing) company I was at hired SSBBs like crazy and none of them really did anything or made a damn bit of difference, but if you took the courses you were fast-tracked for success as far as promotions went.
I was not at all popular among my "peers" in that world. I had too much common sense, and didn't make changes just to make changes. I was extremely effective, but since I didn't play by the rules I was ostracized internally.
That said, it did give me the opportunity to get the hell out of that community and back into IT/CyberSec.
In all honesty, there are absolutely good concepts and tools in that world. The issue is that the people who gravitate to that world have no ability to provide any value themselves so don't actually understand how to USE those tools properly.
Everything is a nail when all you have is an ENTIRE TOOLBOX.
It’s interesting because I found Mech design engineers with green belts to usually be really sharp. I only met like one black belt design engineer and he pivoted careers a decade ago. But most black belts I met in QA were managers. Most of the technicians that did the work weren’t certified but lived and breathed the products for decades.
This is the source of so much despair for me, I have lived it in two jobs. I want the stability of nice salaried gig but I know that if I do well, my managers are going to see that as slack and I am going to be overworked to a breaking point. And the greedy fucks would never agree to a rev-share or profit-share, no just here's your annual 5% increase now shut the fuck up and do your job.
A few jobs ago I was asked to list the names of projects I was actively maintaining for the company, that list stretched out to two entire screens in Excel at default font size! I was burnt out to the point of severe depression.
Even worse is I hate the bullshit ritual of submission that is job hunting.
I can relate. The feeling of being undervalued and taken advantage of is poison for the soul. It's gotten to the point where I optimize for financial independence, maximum income, minimum hours worked and maximum free time.
> I'll echo the last part of that post: Beware of doing good things at work; the reward is rarely compensation and is usually more work for the same pay.
Let’s rephrase this in a less nihilistic way: Understand your organization’s values and culture. And beware of doing good (or bad) things in a way that goes against your organization’s values and culture.
Speaking as someone who spent over 20 years in the DOD:
There isn't a way to make things better in the DOD that doesn't go against the culture. Period. My literal JOB was to make things better and it was the worst time of my career. That was with Flag Officer backing and independent authority. I got a chest full of shiny candy and a pathological distaste for it all after a few years of that.
You do have the option of becoming self employed. If you do, then you might discover a very different perspective. I've been doing it for 23 years now, with two partners and around 20 odd employees. IT company, you wont have heard of us but you will know some of our customers.
I'm still sort of working now (its ~0030 here) keeping an ear to the ground - HN, r/sysadmin (int al), The Register and a few others are some of my canaries. I listen to their songs. I also have a BirdNetPi doing the same for actual birds too.
If my company cocks up it will probably be fatal. I/we don't get the luxury of blaming someone else and I certainly don't get to mumble some sort of bollocks about "Your x is very important to us, soz we failed, lol and that - here have some free credit rating checker thingie".
Why not try running a small company and living on your own wits with no mother to land on - you are mother!
one day, maybe. Today, I personally lack the savings buffer nor desirable kinds of skills to pull it off properly. Wasn't born with a silver spoon, so I guess I gotta pay my dues.
Even then, my kind of self work is just as unstable as my industry. I may in fact be good at it and still fail.
>Note that some of that was as a Lean Six-Sigma Black Belt doing Enterprise projects
>(I hate every single bit of terminology in that entire godforsaken sentence.
Checks out. :).
A super convenient way to do cool stuff like that in the DoD is to do it for the most senior Flag you can get interested in it directly. Projects at SECDEF or SECNAV offices work...differently than outside, as there isn't really an 'up' left for most of the folks involved (in my experience - most are extremely focused on getting the job done and/or the geo-strategic problems).
During the time I worked in that position I was one office removed from COMFRC, which was the Flag I generally reported to via my CO (who was the one who put me in that position and was highly supportive, and who ended up as COMFRC in later years).
Anything that was directly interfacing with that office was great; as soon as I was detailed to do something downline from that office it got painful despite reporting to that Flag.
I wasn't in a position to go higher without spending more time in the service, and that was a non-starter for me.
In the military it’s extremely common for someone above you in the Chain of Command to get the credit; it may be your boss, or several levels up.
Some of that credit may or may not spill over to you, but the general thought process is that they are responsible for what happens in the command so they get the credit. Good ones avoid that and push the credit to where it belongs, but then they don’t promote. This leads to an incentive to take credit from those under you in order to get promoted.
Presidents routinely take credit for all sorts of things that happen without their involement or knowledge. It's part of the tradeoff of expecting them to be omnipotent.
> I'll echo the last part of that post: Beware of doing good things at work; the reward is rarely compensation and is usually more work for the same pay.
Likewise with chip software tooling; despite it being standard to outsource tooling to large EDA vendors, we got a lot of mileage out using our own custom tools, generally created or maintained by one person, e.g., while I was there, most simulator cycles were run on a custom simulator that was maintained by one person, which saved millions a year in simulator costs (standard pricing for a simulator at the time was a few thousand dollars per license per year and we had a farm of about a thousand simulation machines). You might think that, if a single person can create or maintain a tool that's worth millions of dollars a year to the company, our competitors would do the same thing, just like you might think that if you can ship faster and at a lower cost by hiring a person who knows how to crack a wafer open, our competitors would do that, but they mostly didn't.
Have worked in EDA and .. yeah, the software is bad but the buyers are conservative, precisely because it's a risky and expensive business.
He talks about a "cocktail party version of the EMH". The credentialed econ version of "nothing works" is https://en.wikipedia.org/wiki/The_Market_for_Lemons , which discusses the role of information and its asymmetry in markets. The EMH fails precisely because total knowledge is impossible and adverse selection is real.
There are some companies that represent the opposite of what OP describes. Once you work with one of them, I've found it impossible to settle for one of the bad places.
Lol, if I wasn't given a raise, I'd agree to everything, and at the start of my presentation say
"hi everyone, I wanted to walk you how we got to a half-a-million dollar savings, basically I spent a day looking at how terrible the original infrastructure was deployed and removed a code test feature that was causing the problems. This was just complete oversight from every aspect of the development, management, testing, and everything. Overall this code is as bad as it can possibly get, and we just launched it. And basically I was told not to say any of this because it makes everyone look bad, so I was to roll this out gradually to make it seem like managers were doing some sort of work."
Then drop the mic and walk off stage.
Honestly the amount of give-a-fucks I would have lost would have been a lot. And this is coming from someone who's done this for almost 2 decades and cares about his job because bills to pay, kids to feed.
Back in the day I worked at a company that if you came up with some long term cost saving measure, they gave you a bonus of 10% net savings for the first year.
I worked for a large telco that operates very, very similarly to the company in the article.
About once a year or so one of the stand-out engineers that had the weight of the world on their shoulders would get burnt out and frustrated enough to do exactly what you suggest above.
Literally everyone would just look around awkwardly, leave the meeting and never talk about it again. All of middle management already know all of this, the only way they keep their jobs is by never talking about it, and just ignoring anyone that does. The VPs and President only know what those below them feed them.
"Back in the day" I identified a way to save prob a quarter million dollars a year on my project, funded with your tax dollars. Created a spreadsheet model for the cost, documented it all, and gave it to my boss. He said "I'll look it over". I asked his two weeks later if he'd been thru it. His response was "It looks correct. Nice job." I asked if were going to trial it.
Golden response: "No, we don't get paid to save money; we get paid to spend money."
And that is when I truly understood Cost+Award Fee contracting with the gov't.
Anecdotally I have worked in government roles and have been rewarded extremely well for saving taxpayer money, to the point where I felt it was too much at times. It really all dials down on the common denominator, which is the boss or manager. I've learned over the years in both the private and public world that if a manager is not willing to go out of their way to reward their great employees, it's time to move on to the next role.
Maybe my friends are all just bad devs but I don't hear of people getting treated well at FAANG anymore. (Though I haven't heard anything about Netlifx recently.)
At least at fang you can write a mostly plaintext document instead of PowerPoint. Downside is you would probably have to do it before making the changes, have it reviewed by the entire team and get “alignment”
You didn't accidentally save half a million, you deliberately and intentionally saved them half a million, but now you regret it. That's not the same thing.
Large organisations are so woefully inefficient that I'm surprised they're able to compete at all, but they have a ton of money and economy of scale and all that, and along the way there's more than enough money to waste millions on stupid nonsense and inefficiency and nobody really cares.
> Large organisations are so woefully inefficient that I'm surprised they're able to compete at all
It's easy to understand: large orgs are competing against other large orgs that are similarly inefficient, because efficiency in a large org is a really, really, really hard human problem that we simply haven't solved.
And remember they're still providing a valuable service/product. For all their inefficiency, they're still more efficient than not existing at all. This is why they would exist if there weren't competition.
And you might ask, what about competition from small orgs? Well, a small org has less efficiencies of scale, but if they are efficient otherwise, they can sometimes compete effectively with the large org. But then they grow into a large org and will wind up with all of the large org inefficiencies.
It's just how things are. It's not that nobody cares -- to the contrary, company owners care hugely. It's that literally nobody has any idea how to solve it.
> It's easy to understand: large orgs are competing against other large orgs that are similarly inefficient, because efficiency in a large org is a really, really, really hard human problem that we simply haven't solved.
We used to break up companies that became Too Large. Either they became so large that it was impossible to compete against them (as Large Company can demand way better pricing on volume than Small Startup can or has way better access to financial instruments due to better ratings, or like many automotive companies even run their own goddamn bank), or because they could use a wildly profitable business to price-dump competitors out of existence or because they became so large that they felt free to extort their customers to the tune the people would complain about it in the media/their congresspeople too much, or because they became so large their sheer size represented too much economic risk ("too big to fail").
We should begin doing that again. Something as big as Google/Alphabet, Apple, Amazon, Meta/Facebook, Walmart, the Big Four consultancy shops, virtually all major banks - there's absolutely no reason these should be allowed to exist at their current size. Or, if these companies still wish to exist at their size / their existence as one platform, they at least have to be regulated to mitigate the threat originating from that scale.
Competition is usually eliminated by one or more "moats": huge capital upfront investment, patents, market lockin agreements (e.g. the way Windows is sold to OEMs), vertical integration (e.g. Apple), compliance (it's genuinely hard to start a bank or a hospital), buying out competition, buying out the staff of the competition (why FAANG have so many highly paid staff doing work that gets cancelled) or even less legitimate means.
Large companies tend to end up with similar failure modes to authoritarian state-controlled companies. It's interesting how well China has been able to ride the line between control and growth on this one.
They typically don't. When the government doesn't enforce it's own rules, these companies just buy up competition. They strangle the market and they get more inefficient all at the same time.
In my mind, there is where inflation really comes from. For every wage paid that wasn't useful, the resulting product becomes more expensive / less profitable. If we were able to optimize out all the waste we'd probably have deflation while computers and business processes keep getting more efficient
Inflation is actually very simple: More money is added to the system than value produced. That is, money is printed faster than value is created through labor. Stop and think for a second. How do we have trillions of USD? Banks create money. And they're doing it faster than ever before.
Now you can absolutely have price gouging at the same time, but the two are independent of each other, even though combined the affects are worse for the price gouged.
I mean waste is probably a source of inflation but lots of it just comes from unrestrained profit-seeking behavior. If a company believes it can raise its prices without impacting sales, it will do that. That's why so many massive corporations posted record-breaking profits during the recent bouts of inflation.
Except that generalized deflation is one of the largest and widespread economic disasters that can be brought about; you are disincentivizing immediate consumption to on the basis of future larger consumption, you're disincentivizing investing in the economy instead of just holding capital - both things that will make a economy stop working in the long term.
It's the reason why most central banks really want some minimal level of inflation going on at all times.
Another way to look at it, you intentionally saved the company half a million and you got paid what you think was the fair pay for it. It seems like the author might think it's too much money but the irony is that someone is clearly undercharging (helping skew salaries in the entire industry lower, of course) while execs get bonuses. There's nothing to regret if you can make ends meet, great job.
It would be helpful if you'd point out what term is appropriate. Given the reference to "my country", odds are that they're not a native speaker. (I can't tell from the English though, but I'm not a native speaker myself.)
Once upon a time I uncovered a bug that recovered $4MM/year in revenue. It was swept under the rug to protect the team and executives that let the blunder continue for as long as it did. I didn't get a raise, but I made some allies and got to coast for a while.
Very early in my career I discovered some mission-critical network devices at the hedge fund I worked at that would have caused a network loop if either of the two machines were ever rebooted. The two servers were related to some feed data and I had noticed that they were cabled together oddly based on how their function was described to me.
The estimated cost of downtime for these systems was something like $7 million per minute. I had raised the issue to a couple of the staff responsible for the machines and to the networking team but was completely dismissed because "there is no way we would have hooked them up that way" and because I was the FNG.
I then raised the issue again at the weekly group meeting because it seemed important -- somebody was dispatched to check visually and came back to confirm what I said. It was a big deal -- the networking team had about 2 weeks of emergency work to do to resolve the issue cleanly.
EVERYONE was angry at me. Even though I had just averted a catastrophe for the company, I made everyone look bad by doing it and particularly because of my status/position on the team. It was an important lesson learned.
On his book "The Secrets of Consulting", Gerald Weinberg advise against improving more than 10% of performance, and if so, of hoping to have any credit.
Just like the article, his reasoning is that if you improve performance too much, it makes management/the team look bad for not doing it before, while a smaller improvement in performance make management looks good.
>> EVERYONE was angry at me. Even though I had just averted a catastrophe for the company, I made everyone look bad by doing it and particularly because of my status/position on the team.
People can be excessive in both taking credit and placing blame. An appropriate and helpful way to frame this is "the system was configured incorrectly but nobody noticed because the problem never actually happened. It's a good thing the new guy had time to go through things and spotted the problem before it ever happened." No need to crucify the team or exaggerate the value of the new guy.
If you had just let the crisis occur, everyone would get a chance to spring into action like heroes, handshakes and champagne all around on a job well done.
A company of maniacs, but that was a good lesson. Often it’s better to just stay silent and watch. It’s brutal to watch, but better to be the bad guy for pointing at obvious flaws.
Or find and company that values your talent and knowledge.
The Secrets of Consulting", Gerald Weinberg advise against improving more than 10% of performance, and if so, of hoping to have any credit.
Just like the article, his reasoning is that if you improve performance too much, it makes management/the team look bad for not doing it before, while a smaller improvement in performance make management looks good.
I did something similar early in my career but it very much went noticed. The system I wrote to correct the bug was basically a MITM attack against pharmacy rx transmissions, it re-priced prescriptions just before they went to the insurance carrier's system because the pharmacies would never apply price updates (i worked at a small independent pharmacy chain, they didn't have a central dispensing system). In my infinite genius I named the system after my current crush. Corporate liked it so much they made an annual award named after my system... so therefore after my crush but i was too embarrassed to tell them the real origin of the name. That was 25 years ago but, to this day, every year a trophy gets made with the first name of my old crush printed on it and handed out. heh if she ever knew she'd be mortified.
This kind of thing has been going on forever. Once upon a time, I walked into a room full of VT100 terminals, and bored CS students waiting for their compiles to finish. (It was a weekend near the end of term) I took a look at the system, and realized they had pushed all the compiles into a batch queue, but that queue defaulted to BELOW interactive priority, so any keystroke anywhere had higher priority... so all the people checking their position of the compile job in the queue, slowed it down even more.
Over the next 15 minutes, I kept bumping the priority of the top job in the queue up, an hour later everyone had their work done, and went home. I had the room to myself. Rogue sysadmin for the win. ;-)
I saved my company $1 million a year a couple of months ago by noticing that there was an S3 bucket that kept growing and costing 80k a month.
I poked around and realized that there was a system that we weren't using anymore that was copying files to the bucket I reached out to the stakeholders and they turned it off and we deleted the files.
The higher-ups didn't seem to really care, My boss's boss told me to reach out to another team that should've caught this and that was about it.
"For example, you've run 234,745 INSERTs into [table name deleted]_HOURLY and ALL of them were under 1000 rows."
That's an actual quote from our org's snowflake support slack, about pretty much the same problem: lighting up the cluster with a bunch of tiny transactions spread out in time. The product is not made for common use cases like trickle loading.
I'm an actual experienced data warehouse admin, so I'm watching them rediscover my job description one cost overrun at a time. They unironically say things like "It's self-managing, but you need to monitor costs and rewrite loads and queries to run more efficiently". I'm afraid to ask them what they think I do all day.
Readit News
My career record (US Navy) for cost savings was something over $50MM. Every time I did something I had to do PowerPoints, present to Flag Officers, etc. Hell, once I almost got hugely punished because I didn't let my boss take the credit (he had no desire to because he had zero idea what it was I even did).
Note that some of that was as a Lean Six-Sigma Black Belt doing Enterprise projects (I hate every single bit of terminology in that entire godforsaken sentence), where it was literally my JOB to save the DOD as much money as possible with the least disruption possible. Those were the absolute worst years of my career. That period of my career was my reward for just going rogue and fixing things that saved millions.
I'll echo the last part of that post: Beware of doing good things at work; the reward is rarely compensation and is usually more work for the same pay.
Never stay at a job where the culture is toxic. Even if the money is better. Nothing eats at the soul like being held down by feckless, petty careerists and control freaks.
Ideally, you should be working for a company that pays well and that actually rewards you for doing things that are good for the business. But it does seem like virtually every company has some kind of dysfunction.
When they are, just move on. Life's too short. Whatever blows you up with $2 million in savings would blow you up at $500K
Paraphrasing, "The great thing about the US Government is it's so big, if the optimal solution would save 30%, but you find a suboptimal solution that only saves 29%, nobody will notice because you're still saving tens or hundreds of millions of dollars of savings."
One of his big projects was on energy efficiently in remote military camps. The cost of fuel per gallon was around $100 delivered to parts of Afghanistan. They had tons of portable generators running at ~20-40% of capacity. If memory serves, ~70% is most efficient. By some combination of building a small or connecting a few tents to a single generator, they were able to very significantly reduce the fuel required and improve the quality of service.
This was one of the most deeply insane parts of the whole twenty-year war. Of course, wars have unlimited budgets, so trucking gasoline thousands of miles through hostile territory with occasional losses and then using it to aircondition tents was going to happen. While back inside the US teachers were having to buy school supplies with their own money, because only wars have unlimited budgets.
I'm an engineer in a FAANG, who worked directly in money flow, and have had experience in diverse areas where it could come handy.
Then I start thinking finding the right person to work with & right area to start at is probably 95% of the job, then give up.
https://www.usds.gov/
The direct employment option might get better with the recent executive order on AI. USG pays market wages for VA doctor/s; eventually they’ll do the same for tech roles.
I'm so triggered by this phrase. The worst (to work for and performing) company I was at hired SSBBs like crazy and none of them really did anything or made a damn bit of difference, but if you took the courses you were fast-tracked for success as far as promotions went.
That said, it did give me the opportunity to get the hell out of that community and back into IT/CyberSec.
In all honesty, there are absolutely good concepts and tools in that world. The issue is that the people who gravitate to that world have no ability to provide any value themselves so don't actually understand how to USE those tools properly.
Everything is a nail when all you have is an ENTIRE TOOLBOX.
A few jobs ago I was asked to list the names of projects I was actively maintaining for the company, that list stretched out to two entire screens in Excel at default font size! I was burnt out to the point of severe depression.
Even worse is I hate the bullshit ritual of submission that is job hunting.
Let’s rephrase this in a less nihilistic way: Understand your organization’s values and culture. And beware of doing good (or bad) things in a way that goes against your organization’s values and culture.
There isn't a way to make things better in the DOD that doesn't go against the culture. Period. My literal JOB was to make things better and it was the worst time of my career. That was with Flag Officer backing and independent authority. I got a chest full of shiny candy and a pathological distaste for it all after a few years of that.
The takeaway is don't be good at things you don't want to be asked to do.
You do have the option of becoming self employed. If you do, then you might discover a very different perspective. I've been doing it for 23 years now, with two partners and around 20 odd employees. IT company, you wont have heard of us but you will know some of our customers.
I'm still sort of working now (its ~0030 here) keeping an ear to the ground - HN, r/sysadmin (int al), The Register and a few others are some of my canaries. I listen to their songs. I also have a BirdNetPi doing the same for actual birds too.
If my company cocks up it will probably be fatal. I/we don't get the luxury of blaming someone else and I certainly don't get to mumble some sort of bollocks about "Your x is very important to us, soz we failed, lol and that - here have some free credit rating checker thingie".
Why not try running a small company and living on your own wits with no mother to land on - you are mother!
Even then, my kind of self work is just as unstable as my industry. I may in fact be good at it and still fail.
>(I hate every single bit of terminology in that entire godforsaken sentence.
Checks out. :).
A super convenient way to do cool stuff like that in the DoD is to do it for the most senior Flag you can get interested in it directly. Projects at SECDEF or SECNAV offices work...differently than outside, as there isn't really an 'up' left for most of the folks involved (in my experience - most are extremely focused on getting the job done and/or the geo-strategic problems).
Anything that was directly interfacing with that office was great; as soon as I was detailed to do something downline from that office it got painful despite reporting to that Flag.
I wasn't in a position to go higher without spending more time in the service, and that was a non-starter for me.
A part of me finds I might know you. Another part states: this story is way too common.
... I'm mentally tripping over that part. Is that normally expected? ... By recursion, it seems that POTUS should get all the kudos, all the time.
Some of that credit may or may not spill over to you, but the general thought process is that they are responsible for what happens in the command so they get the credit. Good ones avoid that and push the credit to where it belongs, but then they don’t promote. This leads to an incentive to take credit from those under you in order to get promoted.
Preach. 100%.
Dead Comment
Likewise with chip software tooling; despite it being standard to outsource tooling to large EDA vendors, we got a lot of mileage out using our own custom tools, generally created or maintained by one person, e.g., while I was there, most simulator cycles were run on a custom simulator that was maintained by one person, which saved millions a year in simulator costs (standard pricing for a simulator at the time was a few thousand dollars per license per year and we had a farm of about a thousand simulation machines). You might think that, if a single person can create or maintain a tool that's worth millions of dollars a year to the company, our competitors would do the same thing, just like you might think that if you can ship faster and at a lower cost by hiring a person who knows how to crack a wafer open, our competitors would do that, but they mostly didn't.
He talks about a "cocktail party version of the EMH". The credentialed econ version of "nothing works" is https://en.wikipedia.org/wiki/The_Market_for_Lemons , which discusses the role of information and its asymmetry in markets. The EMH fails precisely because total knowledge is impossible and adverse selection is real.
There are some companies that represent the opposite of what OP describes. Once you work with one of them, I've found it impossible to settle for one of the bad places.
- managers asked how it was possible we saved that much without help from them
- asked to prepare slides
- asked many times on how it happened
- had to roll it out slowly to make it look like they did it over time incrementally vs one small toggle
- asked for a raise due to impact and did not happen.
Sir, for your sake, apply to a FAANG or something, you'll be at least taken care of better.
Also please implement Twitter card metadata in your blog so it looks better on twitter :)
"hi everyone, I wanted to walk you how we got to a half-a-million dollar savings, basically I spent a day looking at how terrible the original infrastructure was deployed and removed a code test feature that was causing the problems. This was just complete oversight from every aspect of the development, management, testing, and everything. Overall this code is as bad as it can possibly get, and we just launched it. And basically I was told not to say any of this because it makes everyone look bad, so I was to roll this out gradually to make it seem like managers were doing some sort of work."
Then drop the mic and walk off stage.
Honestly the amount of give-a-fucks I would have lost would have been a lot. And this is coming from someone who's done this for almost 2 decades and cares about his job because bills to pay, kids to feed.
About once a year or so one of the stand-out engineers that had the weight of the world on their shoulders would get burnt out and frustrated enough to do exactly what you suggest above.
Literally everyone would just look around awkwardly, leave the meeting and never talk about it again. All of middle management already know all of this, the only way they keep their jobs is by never talking about it, and just ignoring anyone that does. The VPs and President only know what those below them feed them.
But you would get handsomely paid.
Didn’t Xitter recently remove the display of all external site metadata except the image?
See, for example: https://twitter.com/simonw/status/1717768637799706922
“We’ll need the percentage complete as well and a summary of the savings expected. It needs to be sent out every Friday at noon.”
Deleted Comment
Noooooooooo
Wait, it wasnt the managers that designed the overly complex solution! this is all on the engineers
Large organisations are so woefully inefficient that I'm surprised they're able to compete at all, but they have a ton of money and economy of scale and all that, and along the way there's more than enough money to waste millions on stupid nonsense and inefficiency and nobody really cares.
It's easy to understand: large orgs are competing against other large orgs that are similarly inefficient, because efficiency in a large org is a really, really, really hard human problem that we simply haven't solved.
And remember they're still providing a valuable service/product. For all their inefficiency, they're still more efficient than not existing at all. This is why they would exist if there weren't competition.
And you might ask, what about competition from small orgs? Well, a small org has less efficiencies of scale, but if they are efficient otherwise, they can sometimes compete effectively with the large org. But then they grow into a large org and will wind up with all of the large org inefficiencies.
It's just how things are. It's not that nobody cares -- to the contrary, company owners care hugely. It's that literally nobody has any idea how to solve it.
We used to break up companies that became Too Large. Either they became so large that it was impossible to compete against them (as Large Company can demand way better pricing on volume than Small Startup can or has way better access to financial instruments due to better ratings, or like many automotive companies even run their own goddamn bank), or because they could use a wildly profitable business to price-dump competitors out of existence or because they became so large that they felt free to extort their customers to the tune the people would complain about it in the media/their congresspeople too much, or because they became so large their sheer size represented too much economic risk ("too big to fail").
We should begin doing that again. Something as big as Google/Alphabet, Apple, Amazon, Meta/Facebook, Walmart, the Big Four consultancy shops, virtually all major banks - there's absolutely no reason these should be allowed to exist at their current size. Or, if these companies still wish to exist at their size / their existence as one platform, they at least have to be regulated to mitigate the threat originating from that scale.
Competition is usually eliminated by one or more "moats": huge capital upfront investment, patents, market lockin agreements (e.g. the way Windows is sold to OEMs), vertical integration (e.g. Apple), compliance (it's genuinely hard to start a bank or a hospital), buying out competition, buying out the staff of the competition (why FAANG have so many highly paid staff doing work that gets cancelled) or even less legitimate means.
Large companies tend to end up with similar failure modes to authoritarian state-controlled companies. It's interesting how well China has been able to ride the line between control and growth on this one.
They typically don't. When the government doesn't enforce it's own rules, these companies just buy up competition. They strangle the market and they get more inefficient all at the same time.
Competition is a fairy tale we tell MBAs so they'll start new businesses so it looks like there's competition. They never had a chance.
Inflation is actually very simple: More money is added to the system than value produced. That is, money is printed faster than value is created through labor. Stop and think for a second. How do we have trillions of USD? Banks create money. And they're doing it faster than ever before.
Now you can absolutely have price gouging at the same time, but the two are independent of each other, even though combined the affects are worse for the price gouged.
It's the reason why most central banks really want some minimal level of inflation going on at all times.
The estimated cost of downtime for these systems was something like $7 million per minute. I had raised the issue to a couple of the staff responsible for the machines and to the networking team but was completely dismissed because "there is no way we would have hooked them up that way" and because I was the FNG.
I then raised the issue again at the weekly group meeting because it seemed important -- somebody was dispatched to check visually and came back to confirm what I said. It was a big deal -- the networking team had about 2 weeks of emergency work to do to resolve the issue cleanly.
EVERYONE was angry at me. Even though I had just averted a catastrophe for the company, I made everyone look bad by doing it and particularly because of my status/position on the team. It was an important lesson learned.
Just like the article, his reasoning is that if you improve performance too much, it makes management/the team look bad for not doing it before, while a smaller improvement in performance make management looks good.
People can be excessive in both taking credit and placing blame. An appropriate and helpful way to frame this is "the system was configured incorrectly but nobody noticed because the problem never actually happened. It's a good thing the new guy had time to go through things and spotted the problem before it ever happened." No need to crucify the team or exaggerate the value of the new guy.
If you had just let the crisis occur, everyone would get a chance to spring into action like heroes, handshakes and champagne all around on a job well done.
Or find and company that values your talent and knowledge.
Over the next 15 minutes, I kept bumping the priority of the top job in the queue up, an hour later everyone had their work done, and went home. I had the room to myself. Rogue sysadmin for the win. ;-)
I poked around and realized that there was a system that we weren't using anymore that was copying files to the bucket I reached out to the stakeholders and they turned it off and we deleted the files.
The higher-ups didn't seem to really care, My boss's boss told me to reach out to another team that should've caught this and that was about it.
That's an actual quote from our org's snowflake support slack, about pretty much the same problem: lighting up the cluster with a bunch of tiny transactions spread out in time. The product is not made for common use cases like trickle loading.
I'm an actual experienced data warehouse admin, so I'm watching them rediscover my job description one cost overrun at a time. They unironically say things like "It's self-managing, but you need to monitor costs and rewrite loads and queries to run more efficiently". I'm afraid to ask them what they think I do all day.