Tangential, but it seems to me that as organizations grow, more and more resources are poured into everything other than what made them successful in the first place. Bureaucracy grows, hierarchies increase, teams upon teams organize, things are envisioned and realized and KPId, volumes of messages and emails shift back and forth, endless hours are met in meetings...
At the same time, productivity is reduced, actual communication diminished, gatekeepers slow everything and everyone down, fiefdoms form with their territorial turf wars, naked emperors run amok fanned by yes-men. On average three people out of a hundred are doing something actually useful, while the company slowly loses its grip on whatever niche monopoly has allowed it to so grotesquely exist thus far.
Everyone else is gradually PTSD'd into a corpo version of Homo Sovieticus, filling out time sheets and RTO attendance records while duly marching towards V17 in the most recent two-year plan, aligned with the corporate values writ large on the HR site's main banner.
All companies start small. Only the absolute best survive, so there is tremendous survivorship bias for the remaining companies to comprise of fabulous people.
Naturally, when the company starts growing, it is insanely hard to be able to keep on recruiting only such fabulous people. If nothing else, it would slow the growth, snd as such is usually not prioritized.
That’s when the regression to the mean starts. Then, you start needing more process and more bureaucracy so that the middling people have any chance of keeping up.
Unfortunately, there is no deus ex machina at the end of this path, like in TFA.
I do wonder if there is a U shape here - ie at some point the company get's so large that the central control breaks down and innovation starts to happen in the cracks.
My experience of very large companies is there is an official way stuff get's done, which is different from how stuff actually get's done - which is through peer networks.
it's like... we go like gangbusters in our lives doing the things that positively need to be done.
Then when we get a moment to breathe... we do the things we want to do more than the things we need to do.
Things work better with a slight sense of urgency, but less than full-on burnout (although fighting along up to burnout is probably pretty ruthlessly efficient)
In my experience, this was likely entirely driven by one person, my guess would be two levels above the author in the org chart. It's sometimes frighteningly easy to convince business leaders that the dev teams are wasting a ton of time, doing the wrong thing, etc. It's even easier when that direction is coming from a consultant (might not be in this case, but I've seen it happen a few times).
Someone who was supposed to be advocating for their team (maybe the author's boss) wasn't, or was being out-advocated by others, and that led to breakdowns. As a manager, I keep a lot of KPIs and do a lot of postmortems (lean), because you need to be able to counter the gut feeling of "development should be faster."
Did that have any effect? Usually people lower in the org chart have less power than those closer to the root, so I'd expect that complaint to be filed away an no-op'ed on.
Nothing in this article pertains to actual research - development has always included elements of design. Interesting article otherwise though.
I've been in an organisation that was actively winding down the research side of R&D. Lots of chemists and physicists let go, or at least not replaced. Projects that had gone nowhere for years canned; people with no output for years canned. More focus on product roadmaps. What's really weird is that every step seemed pretty reasonable, but the overall capability was much less in the end. It's really tricky.
The opposite hasn't really worked out in Detroit, and I don't think it worked particularly well for Boeing either, but I still think organizing R&D under one human is a huge part of the problem. Too much 'efficiency' in research and effectiveness drops to zero. Lack of urgency or excessive urgency (read: burnout) will lead to team after team running out of fucks to give, and one day someone will look up and see nothing has been accomplished with the last several millions of dollars.
I believe having different people running 'competing' groups that report to the board instead of a tall narrow org chart with clear choke points may not fix the problem entirely, but it would slow down the cycle. Worst case you disband one of the orgs and create a new group to replace it, run by someone from one of the more efficient divisions, or new blood from outside.
Often I see this happen, and the result is the company loses out somehow. I think maybe metrics for “output” are wrong in many cases, and you’ve just canned someone who had a useful or even critical role you didn’t know about. A lot of people who are important to company operations are invisible!
One of the challenges is that there are always more smart people outside the org than in - so the chances of the same company out innovating the world again and again is slim.
On the other hand, if your management decide to cut R, and just buy in the innovation at the right time to develop it - if you don't have any internal R people trying to do the same sort of things you will find it really difficult to make the right acquisitions at the right time.
So you need both - and treading that line is really tricky.
I also think that having people with an R mindset is important - people who are interested in pushing the envelop, doing things better etc.
Having people like that in the organisation means you are less likely to be blindsided by a shift in technology that could kill the company.
I know this story is about company scale RnD. It can also be applied at any level. Research lives on a gray scale. At its core is growing understanding of your area so that you can do things you didn't know how to do before. I've always gravitated to the hardest problems to be solved so that I can learn something and make something that no one else had the vision or perseverance to make. So almost all my jobs have been RnD though only a few formally.
The most fun I'd say I've had was recognizing something ineffective and making (software) tools for it. Now that I think about it one of the first programs I made on my Atari 400 as a kid was Room, which let me move/rotate my to-scale bedroom furniture outlines around to see what layouts were possible and may be good to actually move the furniture.
For making it really exact to the actual measurements(which presumably was the goal) grid paper isn't going to work, not without large sheets and drafting tools. Illustrative drawing can be done freehand and allowed to slop and distort things a bit, but for a practical interior design you need to get down to centimeters of space.
I have a new one. PM had determined that their work load is diminished if a project is killed. So they deliberately recommend that projects be terminated and or do things that would cause the likelihood of termination to increase.
Research is one of those things that feels like “work” for me. My least favourite part of grad school. I just want to dive in and touch stuff and prototype. I find myself often jumping to the prototype phase as a way to justify skipping research. Maybe I’ll review a few related libraries and some blogs and such.
It’s definitely something I’d like to work on while not losing the practicality of not being caught in research hell like some peers have in the past. Their end products ended up late and no better than my third iteration of the same thing.
It's interesting thinking about this. In my career, I would not think nearly anything I've done resembles research. Just pumping out development tasks.
The one thing I can think of that was like research was really enjoyable.
I should think about how to get more of this in my career. Even making personal projects isn't exactly "research".
I was on a moonshot team in a previous role. Research is a lot of fun to get paid for certainly doesn’t necessarily imply academic (being a DS lends to a bit more of this than typical SWE). In my experience it’s big open problems that no one really expects you to solve, and rarely would there be any top down direction on how to do so. And those problems aren’t always e.g. mathematical. It could be figuring out how a new product could enter a market, quantifying demand for some product, testing out a new algorithm, or doing a greenfield rebuild of something that exists but could only be meaningfully improved by starting over.
I think what is satisfying about this is the fact that your day to day is largely self directed and open ended. It’s not the type of thing that lends itself to backlogs and well defined tickets, and typical productivity methodologies like whole/scrum tend to fall flat in teams like this for this reason. You just sort of dive deep on a problem, put together prototypes, figure out how to quantify their utility, and keep trying new things. There also tends to be less pressure on deadlines because of the lack of top down.
This sounds right up my alley. Any suggestion on roles/titles/companies to keep an eye out for? I’ve been a SWE for 20+ years and have a background in mechanical engineering
In your average job instead of “research” it’s really “discovery”. which is trying to decipher what some business guy at your company or a customer really wants
If people are asking you to do something that doesn't make sense, there's still nothing to discover. There are only a finite number of holistic social needs, simply do all of them.
I've seen a surprisingly low rate of research conducted 'R&D' roles through my admittedly short career. The research segment of any work had been limited to testing of ideas that are highly likely to work, the bulk of the work is product or prototype development. The R&D technologists employed tend to act as rapid response personnel for tasks not predicted by project managers or systems engineers.
Readit News
At the same time, productivity is reduced, actual communication diminished, gatekeepers slow everything and everyone down, fiefdoms form with their territorial turf wars, naked emperors run amok fanned by yes-men. On average three people out of a hundred are doing something actually useful, while the company slowly loses its grip on whatever niche monopoly has allowed it to so grotesquely exist thus far.
Everyone else is gradually PTSD'd into a corpo version of Homo Sovieticus, filling out time sheets and RTO attendance records while duly marching towards V17 in the most recent two-year plan, aligned with the corporate values writ large on the HR site's main banner.
All companies start small. Only the absolute best survive, so there is tremendous survivorship bias for the remaining companies to comprise of fabulous people.
Naturally, when the company starts growing, it is insanely hard to be able to keep on recruiting only such fabulous people. If nothing else, it would slow the growth, snd as such is usually not prioritized.
That’s when the regression to the mean starts. Then, you start needing more process and more bureaucracy so that the middling people have any chance of keeping up.
Unfortunately, there is no deus ex machina at the end of this path, like in TFA.
My experience of very large companies is there is an official way stuff get's done, which is different from how stuff actually get's done - which is through peer networks.
it's like... we go like gangbusters in our lives doing the things that positively need to be done.
Then when we get a moment to breathe... we do the things we want to do more than the things we need to do.
Things work better with a slight sense of urgency, but less than full-on burnout (although fighting along up to burnout is probably pretty ruthlessly efficient)
Deleted Comment
Someone who was supposed to be advocating for their team (maybe the author's boss) wasn't, or was being out-advocated by others, and that led to breakdowns. As a manager, I keep a lot of KPIs and do a lot of postmortems (lean), because you need to be able to counter the gut feeling of "development should be faster."
Someone two, three levels above us in the org chart.
I've been in an organisation that was actively winding down the research side of R&D. Lots of chemists and physicists let go, or at least not replaced. Projects that had gone nowhere for years canned; people with no output for years canned. More focus on product roadmaps. What's really weird is that every step seemed pretty reasonable, but the overall capability was much less in the end. It's really tricky.
I believe having different people running 'competing' groups that report to the board instead of a tall narrow org chart with clear choke points may not fix the problem entirely, but it would slow down the cycle. Worst case you disband one of the orgs and create a new group to replace it, run by someone from one of the more efficient divisions, or new blood from outside.
Often I see this happen, and the result is the company loses out somehow. I think maybe metrics for “output” are wrong in many cases, and you’ve just canned someone who had a useful or even critical role you didn’t know about. A lot of people who are important to company operations are invisible!
On the other hand, if your management decide to cut R, and just buy in the innovation at the right time to develop it - if you don't have any internal R people trying to do the same sort of things you will find it really difficult to make the right acquisitions at the right time.
So you need both - and treading that line is really tricky.
I also think that having people with an R mindset is important - people who are interested in pushing the envelop, doing things better etc.
Having people like that in the organisation means you are less likely to be blindsided by a shift in technology that could kill the company.
The most fun I'd say I've had was recognizing something ineffective and making (software) tools for it. Now that I think about it one of the first programs I made on my Atari 400 as a kid was Room, which let me move/rotate my to-scale bedroom furniture outlines around to see what layouts were possible and may be good to actually move the furniture.
It’s definitely something I’d like to work on while not losing the practicality of not being caught in research hell like some peers have in the past. Their end products ended up late and no better than my third iteration of the same thing.
There’s a balance I’m still fighting to find.
The one thing I can think of that was like research was really enjoyable.
I should think about how to get more of this in my career. Even making personal projects isn't exactly "research".
I think what is satisfying about this is the fact that your day to day is largely self directed and open ended. It’s not the type of thing that lends itself to backlogs and well defined tickets, and typical productivity methodologies like whole/scrum tend to fall flat in teams like this for this reason. You just sort of dive deep on a problem, put together prototypes, figure out how to quantify their utility, and keep trying new things. There also tends to be less pressure on deadlines because of the lack of top down.