Readit News logoReadit News
sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
bob1029 · 11 hours ago
Monolith is definitely what you want to start with.

Being able to ~instantly obtain a perfect list of all references to all symbols is an extraordinarily powerful capability. The stronger the type system, the more leverage you get. If you have only ever had experience with weak type systems or poor tooling, I could understand how the notion of putting everything into one compilation context seems pointless.

sethammons · 5 minutes ago
/me raises hand. Any system that passes querysets around for one. Can't know who is using what.
sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
AlotOfReading · 8 hours ago
There's a lot of middle ground between "deploy to production 20x a day" and "deploy so infrequently that you forget how to deploy". Like, once a day? I have nothing against emergency fixes, unless you're doing them 9-19x a day. Hotfixes should be uncommon (neither rare nor standard practice).
sethammons · 7 minutes ago
Org size matters. A team of 500 should be deploying multiple times per day.
sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
_pdp_ · 13 hours ago
In my previous company we did everything as a micro service. In the company before that it was serverless on AWS!

In both cases we had to come up with clever solutions to simply get by because communication between services is of a problem. It is difficult (not impossible) to keep all the contracts in sync and deployment has to be coordinated in a very specific way sometimes. The initial speed you get is soon lost further down the path due to added complexities. There was fear-driven development at play. Service ownership is a problem. Far too much meetings are spent on coordination.

In my latest company everything is part of the same monolith. Yes the code is huge but it is so much easier to work with. We use a lot more unit tests then integration tests. Types make sense. Refactoring is just so easy. All the troubleshooting tools including specialised AI agents built on top of our own platform are part of the code-base which is kind of interesting because I can see how this is turning into a self-improving system. It is fascinating!

We are not planning to break up the monolith unless we grow so much that is impossible to manage from a single git repository. As far as I can tell this may never happen as it is obvious that much larger projects are perfectly well maintained in the exact same way.

The only downside is that build takes longer but honestly we found ways around that as well in the past and now with further improvements in the toolchains delivered by the awesome open-source communities around the world, I expect to see at least 10x improvement in deployment time in 2026.

Overall, in my own assessment, the decision to go for a monolith allowed us to build and scale much faster than if we had used micro services.

I hope this helps.

sethammons · 32 minutes ago
My experience is the opposite. I worked at SendGrid for a decade and we scaled the engineering org from a dozen to over 500 operating at scale sending billions of messages daily. Micro services. Well, services. The word micro messes people up.

I have also worked at half a dozen other shops with various architectures.

In every monolith, people violate separation of concerns and things are tightly coupled. I have only ever seen good engineering velocity happen when teams are decoupled from one another. I have only seen this happen in a (micro) service architecture.

Overall, in my own assessment, the decision to stick with a monolith has slowed down velocity and placed limits on scale at every other company I have been at and require changes towards decoupled services to be able to ship with any kind of velocity.

The place I just left took 2 years, over 50 teams, and over 150 individual contributors to launch a product that required us to move over an interface for sending messages from ORM querysets to DTOs. We needed to unlock our ability to start rearchitecting the modules because before it was impossible to know the actual edges of the system and how it used the data. This was incredibly expensive and hard and would never have happened but for the ability to reach into other's domains and making assumptions making things hard.

Don't couple systems. Micro services are the only arch I have seen successfully do this.

sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
echelon · 11 hours ago
> If you must to deploy every service because of a library change

Hello engineer. Jira ticket VULN-XXX had been assigned to you as your team's on call engineer.

A critical vulnerability has been found in the netxyz library. Please deploy service $foo after SHA before 2025-12-14 at 12:00 UTC.

Hello engineer. Jira ticket VULN-XXX had been assigned to you as your team's on call engineer.

A critical vulnerability has been found in the netxyz library. Please deploy service $bar after SHA before 2025-12-14 at 12:00 UTC.

...

It's never ending. You get a half dozen of these on each on call rotation.

sethammons · an hour ago
My experience doesn't align with yours. I worked at SendGrid for over a decade and they were on the (micro) service train. I was on call for all dev teams on a rotation for a couple of years and later just for my team.

I have seen like a dozen security updates like you describe.

sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
philwelch · 8 hours ago
If there’s any shared library across all your services, even a third party library, if that library has a security patch you now need to update that shared library across your entire service fleet. Maybe you don’t have that; maybe each service is written in a completely different programming language, uses a different database, and reimplements monitoring in a totally different way. In that case you have completely different problems.
sethammons · an hour ago
Everyone needing to update due to a security thing happens infrequently. Otherwise, coding and deploying may be happening multiple times a day.

We have had shared libraries. Teams updated to them when they next wanted to. When it was important, on call people made it happen asap. Zero issues.

sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
imtringued · 4 hours ago
I don't see the problem?

There's at least one employee per micro service so there should be zero problems preventing just bumping the version of the library.

sethammons · an hour ago
This Segment team was 3 people and 140 services. Microservices are best at solving org coordination issues where teams step on each other. This is a case of a team stepping on itself.
sethammons commented on Why Twilio Segment moved from microservices back to a monolith   twilio.com/en-us/blog/dev... · Posted by u/birdculture
sethammons · 11 hours ago
I left Twilio in 2018. I spent a decade at SendGrid. I spent a small time in Segment.

The shitty arch is not a point against (micro)services. SendGrid, another Twilio property, uses (micro)services to great effect. Services there were fully independently deployable.

sethammons commented on Computer animator and Amiga fanatic Dick van Dyke turns 100    · Posted by u/ggm
ks2048 · 16 hours ago
I just saw a tweet saying his birth was closer to the death of Thomas Jefferson (1826) than it is to today. Wow.
sethammons · 16 hours ago
The grandson of the 10th US president died this year. The US is barely three generations old. That guy could say his grandfather shook hands with Thomas Jefferson.
sethammons commented on Microservices should form a polytree   bytesauna.com/post/micros... · Posted by u/mapehe
jayd16 · 2 days ago
It's about the same for most code all the way down to single threaded function flow.
sethammons · 2 days ago
Yes! This is not unique to microservices.

If you look at this proposal and reject it, i question your experience. My experience is not doing this leads to codebases so intertwined that organizations grind to a halt.

My experience is in the SaaS world, working with orgs from a few dozen to several thousand contributors. When there are a couple dozen teams, a system not designed to separate out concerns will require too much coordinated efforts to develop against.

sethammons commented on Is it a bubble?   oaktreecapital.com/insigh... · Posted by u/saigrandhi
sp4cec0wb0y · 4 days ago
> In many advanced software teams, developers no longer write the code; they type in what they want, and AI systems generate the code for them.

What a wild and speculative claim. Is there any source for this information?

sethammons · 4 days ago
At $WORK, we have a bot that integrates with Slack that sets up minor PRs. Adjusting tf, updating endpoints, adding simple handlers. It does pretty well.

Also in a case of just prose to code, Claude wrote up a concurrent data migration utility in Go. When I reviewed it, it wasn't managing goroutines or waitgroups well, and the whole thing was a buggy mess and could not be gracefully killed. I would have written it faster by hand, no doubt. I think I know more now and the calculus may be shifting on my AI usage. However, the following day, my colleague needed a nearly identical temporary tool. A 45 minute session with Claude of "copy this thing but do this other stuff" easily saved them 6-8 hours of work. And again, that was just talking with Claude.

I am doing a hybrid approach really. I write much of my scaffolding, I write example code, I modify quick things the ai made to be more like I want, I set up guard rails and some tests then have the ai go to town. Results are mixed but trending up still.

FWIW, our CEO has declared us to be AI-first, so we are to leverage AI in everything we do which I think is misguided. But you can bet they will be reviewing AI usage metrics and lower wont be better at $WORK.

u/sethammons

KarmaCake day8569July 29, 2012
About
I enjoy working on, creating, and learning about highly available, scaled, distributed systems. I have fun mountain biking and doing power and olympic lifting. I have a soft spot for anything that deals with helping people learn, especially mathematics. I can be reached at my first name dot last name at gmail.com.

https://www.linkedin.com/in/sethammons/

View Original