Is SetOps using Kubernetes (EKS) or ECS?
Also you have some potential for saving money by over-provisioning your workloads on the actual compute instances, which is, as far as I can see, is not possible with Cloud Run.
I can share a slide of one of our presentations right now which roughly shows the inner workings of SetOps in your AWS account: https://static-media.setops.co/infra/aws-components.png
Is this not desirable for others ? All the solutions that I see are focussed on containerizing (I get that to an extent). But I would personally want a service on top of AWS that abstracts away setting up EC2, load balancers, auto scaling, RDS etc etc. Does it have to be kubernetes ?
* Understanding which workloads share a node's memory/CPU, and isolating certain workloads for security reasons
* Running specific workloads on specific instance types (e.g. with GPU or extra CPU)
* Configuring network policy between workloads
* Airgapping certain workloads
* Setting priority levels for different workloads, so some scale more rapidly while others have to wait for a new node to be provisioned
* Customized scaling behavior (e.g. based on the depth of a queue or latency metrics)
* Multi-region support for DR
I could probably go on :)
Although some of these requirement, like running specific workloads on specific instance types, could easily be implemented.
I have really mixed feelings about this response.
On the one hand, I 100% agree - vanilla k8s is not prod-ready, and you need to do a _lot_ of work to figure out some things, especially around persistent storage (but load balancing and certs are a pretty solved problem).
But the line "you don't need to care how we run containers" bugs me. Maybe your two-person start up doesn't need to know, but eventually you will grow to the point that you _do_ need to care how things are running, and need control over it. This is why so many companies end up outgrowing Heroku and have to go through an expensive migration.
What I'd love to see is a "batteries-included Kubernetes", which allows me to slowly take control over more and more of the stack, until I'm a 1000 person company and ready to run my own clusters.
And there are a lot of companies which do not become the next Unicorn and need an easy way to manage their container workloads.
SetOps currently uses ECS since it comes with no additional overhead costs for the management plane/API and does the container management job well enough. However this is not a definite decision and ECS could be replaced in the future. The main point is that there is a simple abstraction for users managing the workloads and that the "backend" is interchangeable.
You're welcome.
[0] https://github.com/microsoft/azurechatgpt
[1] https://web.archive.org/web/20230814080150/https://github.co...