I would really like to send this article out to all the developers in my small company (only 120+ people, about 40 dev & test) but the political path has been chosen and the new shiny tech has people entranced.
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
If you have workloads with different shapes, microservices make sense.
If not, do the monolith thing as long as you can.
But if you're processing jobs that need hand off to a GPU, just carve out a service for it. Stop lamenting over microservices.
If you've got 100+ engineers and different teams own different things, try microservices. Otherwise, maybe keep doing the monolith.
If your microservice is as thin as leftpad.js and hosts only one RPC call, maybe don't do that. But if you need to carve out a thumbnailing service or authC/authZ service, that's a good boundary.
There is no "one size fits all" prescription here.
>> Microservices make sense in very specific scenarios where distinct business capabilities need independent scaling and deployment. For example, payment processing (security-critical, rarely updated) differs fundamentally from recommendation engine (memory-intensive, constantly A/B tested). These components have different scaling patterns, deployment cycles, and risk profiles, which justify separate services.
I wonder, at which point is a service getting called microservice? The team-sized service advocated by the usual argument does not sound that "micro" to me - but is most of the times the right size.
I started making the case for organizational efficiency rather than a technical argument. Demonstrating where the larger number of people and teams necessary to make a decision and a change and how that impacts the amount of time to ship new features has been more effective IME.
It's less about how old microservices are and more that ZIRP being over, there is now finally pressure to improve software development efficiency, and to a certain extent, optimize infrastructure costs. Developers are now riding the new wave.
IMO, Engineering mindset is a huge challenge when it comes to 'do you do microservices'
And by that, I mean that I have at times seen and/or perhaps even personally used as a cudgel - "This thing has a specific contract and it is implicitly separate and it forces people to remember that if their change needs to touch other parts well then they have to communicate it". In the real world sometimes you need to partition software enough that engineers don't get too far out of the boundaries one way or another (i.e. changes inadvertently breaking something else because they were not focused enough)
The BEAM ecosystem (Erlang, Elixir, Gleam, etc) can do distributed microservices within a monolith.
A single monolith can be deployed in different ways to handle different scalability requirements. For example, a distinct set of pods responding to endpoints for reports, another set for just websocket connections, and the remaining ones for the rest of the endpoints. Those can be independently scaled but released on the same cadence.
There was a long form article I once read that reasoned through this. Given M number of code sources, there are N number of deployables. It is the delivery system’s job to transform M -> N. M is based on how the engineering team(s) work on code, whether that is a monorepo, multiple repos, shared libraries, etc. N is what makes sense operationally . By making it the delivery system’s job to transform M -> N, then you can decouple M and N. I don’t remember the title of that article anymore. (Maybe someone on the internet remembers).
> The BEAM ecosystem (Erlang, Elixir, Gleam, etc) can do distributed microservices within a monolith.
This ain't new. Any language supporting loading modules can give you the organization benefit of microservices (if you consider it a benefit that is - very few orgs actually benefit from the separation) while operating like a monolith. Java could do it 20+ years ago, just upload your .WAR files to an application server.
> Java could do it 20+ years ago, just upload your .WAR files to an application server.
Erlang could do it almost 40 years ago.
It can be used to upgrade applications at runtime without stopping the service. That works well in Erlang, it’s designed from the ground up for it. I know of a few places that used that feature.
What BEAM offers isn’t modularity, but concurrency. Every genserver is operating concurrently, all within the runtime. Up until actor models came to Java, and more recent Java with lightweight threads, Java was incapable of this. Java is still missing things that BEAM and OTP provides.
I mean sure but one could also argue that VB6 can do the same analogue of Java so long as ASP is involved... And yes I've seen it done; you have a basic interface similar to an actor, but really more like a 'Take a key-value in for input, do processing on it, return a key-value to go to the next stage' and then the other necessary glue to handle that. The way the glue worked, it was minimal ceremony to get a new module in.
NGL It was clever enough that every few years I think about trying to redo the concept in another language....
Yup, good point on the BEAM. The joke we used when microservices were hot was that the BEAM is already ahead with nano-services: a gen_server is a nice lightweight, isolated process. You can define a callback API wrapper for it and deploy millions of them on a cluster.
Yeah, the isolation provides a fault tolerance not seen in wide use until Kubernetes.
Although it would be neat to implement some of the benefits of a service mesh for BEAM — for example, consistently applying network retry/circuit breaker policies, or dynamically scalable genservers.
I'm helping a company get out of legacy hell right now. And instead of saying we need microservices, let's start with just a service oriented architecture. That would be a huge step forward.
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
> And instead of saying we need microservices, let's start with just a service oriented architecture.
I think the main reason microservices were called “microservices” and not “service-oriented architecture” is that they were an attempt to revive the original SOA concept when “service-oriented architecture” as a name was still tainted by association to a perceived association with XML and the WS-* series of standard (and, ironically, often systems that supported some subset of those standards for interaction despite not really applying the concepts of the architectural style.)
Service oriented architecture seems like a pretty good idea.
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
That's the right approach. This is what the article suggests:
>> For most systems, well-structured modular monoliths (for most common applications, including startups) or SOA (enterprises) deliver comparable scalability and resilience as microservices, without the distributed complexity tax. Alternatively, you may also consider well-sized services (macroservices, or what Gartner proposed as miniservices) instead of tons of microservices.
I'm curious, and the specific list of problems and pain points (if--big if!--everyone there agrees what they are) can help more clearly guide the decisions as to what the next architecture should look like--SoA, monolithic, and so on.
While not a complete rebuttal, allow me the following. I manage a team of 4 scrum masters each with 5-6 engineers. We provide services via a user interface we'll call the console, as would be fairly familiar to any B2B or B2C service provider. The backends of this portal are split up by functional area, so we have a compute management service providing CRUD apis for dealing with our compute offerings, a storage service for CRUD on our storage offerings, a network service for interacting with networks etc. all sharing a single, albeit sharded, underlying data store.
My teams pick up a piece of work, check out the code, run the equivalent of docker compose up, and build their feature. They commit to git, merge to dev, then to main, and it runs through a pipeline to deploy. We do this multiple times a day. Doing that with a large monolith that combines all these endpoints into one app wouldn't be hard, but it adds no benefits, and the overhead that now we have 4 teams frequently working on the same code and needing to rebase and pull in change, rather than driving simple atomic changes. Each service gets packaged as a container and deployed to ECS fargate, on a couple of EC2 instances that are realistically a bit oversubscribed if all the containers suddenly got hammered, but 90% of the time they don't, so its incredibly cost effective.
When I see the frequent discussions around microservices, I always want to comment that if you have a disfunctional org, no architecture will save you, and if you have a functional org, basically any architecture is fine, but for my cases, I find that miniservices if you will, domain driven and sharing a persistence layer, is often a good way to go for a couple of small teams.
> we have 4 teams frequently working on the same code and needing to rebase and pull in change, rather than driving simple atomic changes.
You have to pull in changes either way. Either there are contract changes between teams or there aren't. If there aren't, you don't need to rebase just do squash and merge. If there are, then you're going to either find out about the changes now or you're going to find out about them in production when your container starts throwing errors.
I’m always shocked when engineers smitten with the microservices bug try to insist that converting a simple, reliable, in-process function call to a network hop plus serialization plus retry and back off and circuit breaker logic is going to be faster. I’m sure that there are situations where microservices are appropriate, but I’ve never seen one. Mostly I see engineers playing in a sandbox of complexity and then shipping buggy code late. It reminds me of Billy Bean’s question in the movie Money Ball: "If he's a good hitter, why doesn't he hit good?"
Every service boundary replaces a function call with a network request. That one choice cascades into distributed transactions, eventual consistency, and operational overhead most teams don't need. ¯\_(ツ)_/¯
You need multiple services whenever the scaling requirements of two components of your system are significantly different. That's pretty much it. These are often called micro services, but they don't have to actually be "micro"
That's the most nonsensical reason to adopt microservices imo.
Consider this: every API call (or function call) in your application has different scaling requirements. Every LOC in your application has different scaling requirements. What difference does it make whether you scale it all "together" as a monolith or separately? One step further, I'd argue it's better to scale everything together because the total breathing room available to any one function experiencing unusual load is higher than if you deployed everything separately. Not to mention intra- and inter-process comm being much cheaper than network calls.
The "correct" reasons for going microservices are exclusively human -- walling off too much complexity for one person or one team to grapple with. Some hypothetical big brain alien species would draw the line between microservices at completely different levels of complexity.
At this point, I'm convinced that too many people simply haven't built software in a way that isn't super Kubernetes-ified, so they don't know that it's possible. This is the field where developers think 32 GB of RAM isn't enough in their laptop, when we went to the moon with like... 4K. There is no historical or cultural memory in software anymore, so people graduate not understanding that you can actually handle 10,000 connections per second now on a five-year-old server.
When people talk about scaling requirements they are not referring to minutiae like "this function needs X CPU per request and this other function needs Y CPU per request", they are talking about whether particular endpoints are primarily constrained by different things (i.e. CPU vs memory vs disk). This is important because if I need to scale up machines for one endpoint that requires X CPU but the same service has another endpoint requiring Y memory whereas my original service only needs Z memory and Y is significantly larger than Z then suddenly you have to pay a bunch of extra money to scale up your CPU-bound endpoint because you are co-hosting it with a memory-bound endpoint.
If all your endpoints just do some different logic and all hit a few redis endpoints, run a few postgres queries, and assemble results then keep them all together!!!
EDIT: my original post even included the phrase "significantly different" to describe when you should split a service!!! It's like you decided to have an argument with someone else you've met in your life, but directed it at me
Another case I’ve seen is to separate a specific part of the system which has regulatory or compliance requirements that might be challenging to support for the rest of the larger system, eg HIPAA compliance, PCI compliance, etc.
Operationally, it is very nice to be able to update one discrete function on its own in a patch cycle. You can try to persuade yourself you will pull it off with a modular monolith but the physical isolation of separate services provides guarantees that no amount of testing / review / good intentions can.
However, it's equally an argument for SOA as it is for microservices.
There are some other benefits like having different release cycles for core infrastructure that must never go down vs a service that greatly benefits from a fast pace of iteration.
Literally one function per service though is certainly overkill though unless you're pretty small and trying to avoid managing any servers for your application.
I came here to say the same. If you’re arguing either for or against microservices you’re probably not thinking about the problem correctly. Running one big service may make sense if your resource needs are pretty uniform. Even if they’re not you need to weight the cost of adding complexity vs the cost of scaling some things prematurely or unnecessarily. Often this is an acceptable precursor to splitting up a process.
You can still horizontally scale a monolith and distribute requests equally or route certain requests to certain instances; the only downside is that those instances would technically waste a few hundred MBs of RAM holding code for endpoints they will never serve; however RAM is cheap compared to the labor cost of a microservices environment.
I was once working on a project where we had a gRPC server that inserted data into the DB for another service.
This split was probably a mistake, as the interface we exposed resulted in us making twice as many DB calls as we actually needed to.
One of the stored procs needed a magic number as a parameter, which we looked up via another DB query.
One of the other Devs on the team tried to convince me to write a separate gRPC server to run this (trivial) query.
"We're doing microservices, so we need to make everything as small as possible. Looking up this value is a separate responsibility from inserting data."
Luckily our tech lead was sane and agreed with me.
I like goldilocks services, as big or as small as actually makes sense for your domain/resource considerations, usually no single http endpoint services in sight.
Once upon a time, that's what a microservice was. A monolith was the company's software all in one software package.
I think what changed things is FAAS came along and people started describing nanoservices as microservices which created really dumb decisions.
I've worked on a true monolith and it wasn't fun. Having your change rolled back because another team made a mistake and it was hard to isolate the two changes was really rough.
Readit News
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
If not, do the monolith thing as long as you can.
But if you're processing jobs that need hand off to a GPU, just carve out a service for it. Stop lamenting over microservices.
If you've got 100+ engineers and different teams own different things, try microservices. Otherwise, maybe keep doing the monolith.
If your microservice is as thin as leftpad.js and hosts only one RPC call, maybe don't do that. But if you need to carve out a thumbnailing service or authC/authZ service, that's a good boundary.
There is no "one size fits all" prescription here.
>> Microservices make sense in very specific scenarios where distinct business capabilities need independent scaling and deployment. For example, payment processing (security-critical, rarely updated) differs fundamentally from recommendation engine (memory-intensive, constantly A/B tested). These components have different scaling patterns, deployment cycles, and risk profiles, which justify separate services.
Dead Comment
And by that, I mean that I have at times seen and/or perhaps even personally used as a cudgel - "This thing has a specific contract and it is implicitly separate and it forces people to remember that if their change needs to touch other parts well then they have to communicate it". In the real world sometimes you need to partition software enough that engineers don't get too far out of the boundaries one way or another (i.e. changes inadvertently breaking something else because they were not focused enough)
there are of course microservices for things like news feed etc, but iirc all of fb.com and mobile app graphql is from the monolith by default.
The BEAM ecosystem (Erlang, Elixir, Gleam, etc) can do distributed microservices within a monolith.
A single monolith can be deployed in different ways to handle different scalability requirements. For example, a distinct set of pods responding to endpoints for reports, another set for just websocket connections, and the remaining ones for the rest of the endpoints. Those can be independently scaled but released on the same cadence.
There was a long form article I once read that reasoned through this. Given M number of code sources, there are N number of deployables. It is the delivery system’s job to transform M -> N. M is based on how the engineering team(s) work on code, whether that is a monorepo, multiple repos, shared libraries, etc. N is what makes sense operationally . By making it the delivery system’s job to transform M -> N, then you can decouple M and N. I don’t remember the title of that article anymore. (Maybe someone on the internet remembers).
This ain't new. Any language supporting loading modules can give you the organization benefit of microservices (if you consider it a benefit that is - very few orgs actually benefit from the separation) while operating like a monolith. Java could do it 20+ years ago, just upload your .WAR files to an application server.
Erlang could do it almost 40 years ago.
It can be used to upgrade applications at runtime without stopping the service. That works well in Erlang, it’s designed from the ground up for it. I know of a few places that used that feature.
Besides: Erlang predates Java.
NGL It was clever enough that every few years I think about trying to redo the concept in another language....
Although it would be neat to implement some of the benefits of a service mesh for BEAM — for example, consistently applying network retry/circuit breaker policies, or dynamically scalable genservers.
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
I think the main reason microservices were called “microservices” and not “service-oriented architecture” is that they were an attempt to revive the original SOA concept when “service-oriented architecture” as a name was still tainted by association to a perceived association with XML and the WS-* series of standard (and, ironically, often systems that supported some subset of those standards for interaction despite not really applying the concepts of the architectural style.)
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
[1] https://courses.cs.washington.edu/courses/cse452/23wi/papers... [2] corresponding HN thread from 2011 https://news.ycombinator.com/item?id=3101876
>> For most systems, well-structured modular monoliths (for most common applications, including startups) or SOA (enterprises) deliver comparable scalability and resilience as microservices, without the distributed complexity tax. Alternatively, you may also consider well-sized services (macroservices, or what Gartner proposed as miniservices) instead of tons of microservices.
I'm curious, and the specific list of problems and pain points (if--big if!--everyone there agrees what they are) can help more clearly guide the decisions as to what the next architecture should look like--SoA, monolithic, and so on.
The biggest obstacle is changing people's mindsets from legacy programming to modern DevOps workflow. Making them believe it's worth the effort.
My teams pick up a piece of work, check out the code, run the equivalent of docker compose up, and build their feature. They commit to git, merge to dev, then to main, and it runs through a pipeline to deploy. We do this multiple times a day. Doing that with a large monolith that combines all these endpoints into one app wouldn't be hard, but it adds no benefits, and the overhead that now we have 4 teams frequently working on the same code and needing to rebase and pull in change, rather than driving simple atomic changes. Each service gets packaged as a container and deployed to ECS fargate, on a couple of EC2 instances that are realistically a bit oversubscribed if all the containers suddenly got hammered, but 90% of the time they don't, so its incredibly cost effective.
When I see the frequent discussions around microservices, I always want to comment that if you have a disfunctional org, no architecture will save you, and if you have a functional org, basically any architecture is fine, but for my cases, I find that miniservices if you will, domain driven and sharing a persistence layer, is often a good way to go for a couple of small teams.
You have to pull in changes either way. Either there are contract changes between teams or there aren't. If there aren't, you don't need to rebase just do squash and merge. If there are, then you're going to either find out about the changes now or you're going to find out about them in production when your container starts throwing errors.
Every service boundary replaces a function call with a network request. That one choice cascades into distributed transactions, eventual consistency, and operational overhead most teams don't need. ¯\_(ツ)_/¯
Consider this: every API call (or function call) in your application has different scaling requirements. Every LOC in your application has different scaling requirements. What difference does it make whether you scale it all "together" as a monolith or separately? One step further, I'd argue it's better to scale everything together because the total breathing room available to any one function experiencing unusual load is higher than if you deployed everything separately. Not to mention intra- and inter-process comm being much cheaper than network calls.
The "correct" reasons for going microservices are exclusively human -- walling off too much complexity for one person or one team to grapple with. Some hypothetical big brain alien species would draw the line between microservices at completely different levels of complexity.
When people talk about scaling requirements they are not referring to minutiae like "this function needs X CPU per request and this other function needs Y CPU per request", they are talking about whether particular endpoints are primarily constrained by different things (i.e. CPU vs memory vs disk). This is important because if I need to scale up machines for one endpoint that requires X CPU but the same service has another endpoint requiring Y memory whereas my original service only needs Z memory and Y is significantly larger than Z then suddenly you have to pay a bunch of extra money to scale up your CPU-bound endpoint because you are co-hosting it with a memory-bound endpoint.
If all your endpoints just do some different logic and all hit a few redis endpoints, run a few postgres queries, and assemble results then keep them all together!!!
EDIT: my original post even included the phrase "significantly different" to describe when you should split a service!!! It's like you decided to have an argument with someone else you've met in your life, but directed it at me
(To clarify, I’m not disagreeing with you!)
Operationally, it is very nice to be able to update one discrete function on its own in a patch cycle. You can try to persuade yourself you will pull it off with a modular monolith but the physical isolation of separate services provides guarantees that no amount of testing / review / good intentions can.
However, it's equally an argument for SOA as it is for microservices.
Literally one function per service though is certainly overkill though unless you're pretty small and trying to avoid managing any servers for your application.
This split was probably a mistake, as the interface we exposed resulted in us making twice as many DB calls as we actually needed to.
One of the stored procs needed a magic number as a parameter, which we looked up via another DB query.
One of the other Devs on the team tried to convince me to write a separate gRPC server to run this (trivial) query.
"We're doing microservices, so we need to make everything as small as possible. Looking up this value is a separate responsibility from inserting data."
Luckily our tech lead was sane and agreed with me.
I think what changed things is FAAS came along and people started describing nanoservices as microservices which created really dumb decisions.
I've worked on a true monolith and it wasn't fun. Having your change rolled back because another team made a mistake and it was hard to isolate the two changes was really rough.