> few nuggets scattered on the internet regarding how Stripe does things (ex. #1, #2, #3) and in general the conclusion is that they have a very demanding but very advanced engineering culture
The comments so far are surprising. Yea counting PRs and lines of code isn’t impressive, and yes you may also do them at your own company. Any engineer will tell you, if you push code often and continuously move it to production, regression is inevitable. In finance, at a scale that stripe operates, not making mistakes is very critical. Being able to do what the articles describes is very impressive in any engineering organization. Being able to do that as Stripe is even more impressive.
In finance, not making mistakes is not at all critical, and mistakes happpens all the time. Regulatory compliance is critical, as it provides a thorough legal defense even in case of failure.
The lack of efficiency in finance (or pharma for that matter) is not driven by a wish for quality, but purely from a fear of stepping outside regulatory compliance with no individual wanting to be responsible for any process optimization that could later be seen as non-compliant.
Younger companies on the other hand might realize that compliance controls are, in fact, something the company defines themselves and can structure and optimize however they'd like, allowing for both high throughput and compliance. It's just hard to implement in older companies where half the staff would fight back as their role exists purely due to poor processes and their overhead.
If it's not impressive why is the article so impressed (it's even in the headline)?
And no, not making any mistakes isn't critical, ... some are. You can have a million of UI mistakes and regressions (where is the stat for how many regressions there are?) and what not in the UI of you Stripe Dashboard app without any of them being critical. The "finance" aura doesn't permeate literally everything a finance company does, raising it to the critical level
I thought the article was going somewhere, it ended up being a very vapid "Impressive, right? Reflect what you can do to accomplish this in your organization".
Well the actual ending is, "Subscribe to my newsletter!".
My previous company averaged 2 PRs (and deploys) per engineer per day across a small team. At my current company I'm averaging about 2.5 CLs per day (they're a bit smaller changes). Stripe is good at this, but this is very achievable.
Often the problem is that we put too much into a single change. Smaller changes means easier reviews, better reviews, less risky deploys, forces better tooling for deployments and change management, often lower latency, and even often leads to higher throughput because of less WIP taking up head space and requiring context switching. The benefits are so compounding that I think it's very undervalued in some orgs.
That's sort of what I mean by small changes being a forcing function. The tooling we have available rarely makes this level of small changes untenable, it's just clunky. When you send 1k PRs a day though you'll notice things that are too clunky and fix them, and then that makes it easier to get to and maintain that level of productivity.
The article seems surprised that you can do so many changes at scale, but IMO that's the wrong perspective. The larger the scale you have, the easier it must be to ship a change.
Yes, regressions will be more painful if you manage trillions of dollars, but it also means shipping a fix for such regressions needs to be easy and fast, which you can only do if you have a good and frequent CI/CD pipeline.
See also "The Infallible five-tier system for measuring the maturity of your Continuous Delivery pipeline". You should live on "Friday".
Incorrect, shipping a fix quickly is relatively irrelevant, being able to roll back a busted change to a working configuration is critical.
The exception is security issues. But these usually require actual thinking to be fixed, so no, you're not getting volume in the first place.
Preferably not breaking things while doing your mostly cosmetic or preparatiry changes rather than patching afterwards limits the scope of this kind of fix churn.
And how to know you didn't? Proper functional and integration tests is how.
Having minute of downtime per year is quite a big tradeoff.
It makes sense for Stripe, they would lose lot of money when not operating. But in smaller companies you can choose to take more downtime to reduce engineering time.
Instead of skirting around doing gradual data transformations on live data, you could take service offline and do all of it once.
Diminishing returns at some point, but, I think the counterpoint to this is companies who are doing a single huge manual deployment every month (or less) which is scheduled into a 4-hour outage window where many if not all services will be down for this period.
I do agree there isn't a lot of delta between a company doing 2 or 10 deploys a day and a company doing 1,200, but, there's a huge engineering gap between companies who do approximately .03 deploys per day and those doing multiple.
Open source projects can be the worst at this stuff. I realise it's all volunteer run so I'm not complaining too much. But so often they end up pushing a versioned release once a year. So you end up finding a bug, going to report it and see it was fixed 8 months ago but still broken in the published package. And then they get afraid to push a new version because it's been so long since the last one that everything has changed.
Readit News
#3 is "What I Miss About Working at Stripe" (https://every.to/p/what-i-miss-about-working-at-stripe) reminiscing about 15-hour days, missing vacations, and crying at work.
discussed here; https://news.ycombinator.com/item?id=32159752 (131 comments)
Deleted Comment
The lack of efficiency in finance (or pharma for that matter) is not driven by a wish for quality, but purely from a fear of stepping outside regulatory compliance with no individual wanting to be responsible for any process optimization that could later be seen as non-compliant.
Younger companies on the other hand might realize that compliance controls are, in fact, something the company defines themselves and can structure and optimize however they'd like, allowing for both high throughput and compliance. It's just hard to implement in older companies where half the staff would fight back as their role exists purely due to poor processes and their overhead.
And no, not making any mistakes isn't critical, ... some are. You can have a million of UI mistakes and regressions (where is the stat for how many regressions there are?) and what not in the UI of you Stripe Dashboard app without any of them being critical. The "finance" aura doesn't permeate literally everything a finance company does, raising it to the critical level
Well the actual ending is, "Subscribe to my newsletter!".
https://news.ycombinator.com/item?id=32165794
Often the problem is that we put too much into a single change. Smaller changes means easier reviews, better reviews, less risky deploys, forces better tooling for deployments and change management, often lower latency, and even often leads to higher throughput because of less WIP taking up head space and requiring context switching. The benefits are so compounding that I think it's very undervalued in some orgs.
Googler identified.
I'm sure most people are working on some settings UI, fraud or risk tools, etc.
Yes, regressions will be more painful if you manage trillions of dollars, but it also means shipping a fix for such regressions needs to be easy and fast, which you can only do if you have a good and frequent CI/CD pipeline.
See also "The Infallible five-tier system for measuring the maturity of your Continuous Delivery pipeline". You should live on "Friday".
https://qntm.org/cd
The exception is security issues. But these usually require actual thinking to be fixed, so no, you're not getting volume in the first place.
Preferably not breaking things while doing your mostly cosmetic or preparatiry changes rather than patching afterwards limits the scope of this kind of fix churn. And how to know you didn't? Proper functional and integration tests is how.
It makes sense for Stripe, they would lose lot of money when not operating. But in smaller companies you can choose to take more downtime to reduce engineering time.
Instead of skirting around doing gradual data transformations on live data, you could take service offline and do all of it once.
I do agree there isn't a lot of delta between a company doing 2 or 10 deploys a day and a company doing 1,200, but, there's a huge engineering gap between companies who do approximately .03 deploys per day and those doing multiple.
Deleted Comment