Ask HN: How different is AWS/GCP/Azure in everyday work

I have been running a DevOps agency for the last 8 years and while each Clouds basically offers the same things at this point the two things that always trip me up are networking and IAM.

Some things I noticed as I have done work on AWS, Azure and Google in terms of IAM:

  - Azure seems to have so many different types of IAM permissions it is sort of hard to get your head across each one as they seems to be imported from Azure, Active Directory, etc. 
  - Google differentiates between service accounts and user accounts which takes a bit of getting used to as each is different and the specific service policies that need to be granted are much harder to figure out than AWS.
  - AWS now has three different IAM configurations including IAM, AWS IAM Identity Center, and roles. The complication is that AWS was not built with the Google nomenclature of projects in mind so it is a weird add on that causes all sorts of weird issues.

In terms of networking:

  - AWS for me the simplest to grok but I have also been doing it for the longest so there may be a bias here. Everything is tied to a VPC. It also seems that AWS provides the lowest level primitives for networking versus the other providers which tend to abstract away quite a bit.
  - Google's VPC (i.e network) is global across all regions which is nice for data locality as you can use the same VPC and put subnets across regions.
  - Azure is similar to AWS but does seem to have a lot of hidden features that you need to read the docs to enable espcially around AKS+video streams.

ecshafer · 2 years ago

IAM is the biggest miss that all of the cloud providers suck at. I think Google's is the best, but it really isn't a great experience. This seems like something that is so critical it should be rock solid and extremely clear, but too often I see things that get into these weird situation that's hard to predict exact access rules.

sam2426679 · 2 years ago

I’ve done lots with AWS and really only ever used GCP to configure Google SSO. I was really surprised by how much button clicking is required in GCP vs. AWS. In AWS, you create the root account, provision a service account, and then all AWS resources are managed through terraform. In GCP, you have to verify a domain via CNAME records, etc., in order to create a root account, and then manipulate the organization policy to provision the service account. While you can create the IAP brand within terraform (as long as you use the root account and not the service account), you can only externalize the brand by clicking buttons in GCP. Laughably, there is an open issue/ticket from more than a decade ago requesting a programmatic way to externalize a brand.

Dead Comment

Terretta · 2 years ago

Really good answer in terms of how they "feel" to use.

Just one note, since there's a design decision Google and AWS made differently that feels nice but makes availability more precarious:

> "Google's VPC (i.e network) is global across all regions which is nice for data locality as you can use the same VPC and put subnets across regions."

It's also not uncommon seeing your entire global footprint go down when there's a network plane issue.

AWS — for the longest time — was fanatical about keeping services uncoupled across regions, leading to far fewer "global" outages.

Sadly, many customers complained, wanting services to be cross region, instead of having to replicate environments across regions. Fifteen years in, AWS is accommodating, allowing you to build services that span a couple regions and go down if either region is down.

If uptime is critical to you, in AWS leverage at least 3 AZs in each of at least 2 regions, and be sure you're using region-only services or a cross-region service that's really single region with a consistency scheme. You'll stay online through most regional issues.

Also note that the three define "region" quite differently. The AWS definition generally includes a variety of availability and resilience constraints, such as at least 2 AZs with enough physical separation to survive local physical outages. Looking closely comparing across them, you'll find AWS's resilience story is more stringent, the other two are somewhat more oriented to putting a pin on the map and call things regions that may be more like single POPs (points of presence).

All that said, it's becoming "less true" in both directions, as large customers complain when any two CSPs don't work similarly. The "voice of the customer" is asking for feature parity rather than exploiting the differentiation.

From our point of view, they're still differentiated enough a firm should consider using each for what it's best at, say AWS for lego blocks, Azure for business integration, and Google for scale-out analytics feeding ML/AI. Again, each is trying to shore up what the others already have in their DNA, but it's harder to copy something when it hasn't been your in-house bread and butter or you didn't invent it.

Deleted Comment

thr0w · 2 years ago

> Everything is tied to a VPC

bzt

AWS: C/C++

Azure: C#

Google Cloud: TypeScript

---

GCP is by far the easiest to use, IMO. AWS is the most mature and the most "powerful" but also the most complicated. The core pieces are largely similar at a 1000 foot level, but there are a lot of differences when you look closely at the capabilities of the congruent pieces and how they are operated.

One example that really stands out for me is how AWS handles serverless containers (ECS Fargate) versus Google Cloud Run. Both nominally fit the same needs (not really because Fargate doesn't scale to 0), but because of AWS's more "legacy" platform, working with ECS Fargate is slower and more complicated (IMO) than Google Cloud Run which is literally "throw a container here and run it". AWS Copilot CLI is a way to mitigate this, but because it still fundamentally deploys CloudFormation, it always feels clunky compared to GCP.

Another example is how GCP seemingly has HTTP built-in to many of the different services. For example, in AWS, if you want a timer driven job that hits an endpoint, you'll need to send an event to a Lambda to push an HTTP request. In GCP, you'd use Cloud Task Queues, Pub/Sub, or Cloud Scheduler -- all of which support HTTP-based targets without the need to deploy a Function. Simplifies the overall design of the system, IMO.

I almost see it as a kind of second mover advantage where GCP and Azure had an opportunity to learn from AWS and build certain things more ergonomically and with less low-level finagling required to get it to cover the 90% use case.

I wrote a two part series on this specifically focused for startups:

https://itnext.io/aws-vs-azure-vs-google-cloud-for-saas-star...

j45 · 2 years ago

Azure has the most Fortune 100-500 clients already with them as well I heard.

chrisweekly · 2 years ago

I wonder how much of that market penetration stems directly from those companies being all-in with Microsoft.

fragmede · 2 years ago

Is that Azure or "Azure" aka Microsoft Office 365?

devoutsalsa · 2 years ago

If you say "I barely know AWS, we mostly used GCP", that will be a turn off for anyone screening for heavy AWS experience. If you say "we use the cloud extensively & 100% of our infrastructure the made extensive use of AWS and GCP using a multi-cloud solution", you'll be fine.

A recruiter won't know the difference unless they really ask, and I seriously doubt they will.

If an engineer or engineering manager asks, ask them what problem they are trying to solve using AWS and then tell them how to solve it. If your answer is good enough, you won't have any problems.

ein0p · 2 years ago

+1. Early screening is basically just grep for the right verbiage. As a rule, recruiters are not technical. Some of your interviewers also won’t be technical, might also be worth keeping that in mind.

abhiyerra · 2 years ago

nosefrog · 2 years ago

We moved everything to Azure. Still mad about it because things in general don't work on Azure lol. Especially networking features. Avoid any feature that sounds even a little complicated to develop, because it probably doesn't work. Even basics like the CDN have showstopping bugs that Azure blames on Chrome for some reason (update an Azure blob and access it via a cdn, it will fail ~1% of the time).

I hope the DOJ investigates Azure next, because I can't believe such a garbage platform would get to 2nd place without an abuse of Microsoft's monopoly power. Specifically, using their office product monopoly to create a cloud monopoly by signing 365 discounts with customers that require all cloud services used by those customers to be on Azure.

llamaLord · 2 years ago

The amount of bugs in Azure that break genuinely critical parts of the workflow (like... We're talking, the ability to run a deployment on their K8's PaaS at all) which they seem to be totally fine with not fixing for long periods of time genuinely amazed me when we were trying to use them.

nunez · 2 years ago

Or how Entra ID (Azure AD) breaks silently damn near daily and just spectacularly blows up several times per year.

Spooky23 · 2 years ago

Or the “I see you have a license compliance issue, buy some Azure credits, and all is forgiven.”

Also avoid or approach with extreme caution anything that's "In Preview," which will feel like just about everything (or it did when I used Azure heavily in 2020/2021). Unless you like extremely limited documentation, unresolved GitHub issues or some of the most complicated-looking .NET exception stacks you've ever seen.

CharlieDigital · 2 years ago

tmsh · 2 years ago

There's a lot of edge cases. Since we spend so much time in this part of the tech stack compared to 10-15 years ago, I think it'd be similar to knowing Java v. C++ 10-15 years ago. There's a lot of devil in those details (what was: what's your opinion on template metaprogramming in C++? Or how do you optimize garbage collection in Java? Or do you know Effective Java v. Effective C++? -- is now, how do you deal with cold starts in AWS Lambda? How do you represent infrastructure as code in a nimble way? When you're setting up a load balancer what kind of things get stuck? How do you ensure blue/green deployments are optimal?) If this part of the stack weren't so complicated for many sets of applications would there not be companies like Vercel that attempt to abstract these layers and make a lot of money on top of that? So yes the general patterns are similar. But everything about the implementation is different. And there are smooth parts (with ML colabs etc in GCP) in one, and smooth parts in others (QuickSight, OpenSearch integration, AWS SDK once you understand the building blocks, etc.).

greatgib · 2 years ago

From my experience, Google works but service are badly finished at the edge. It is lien them being constantly in Beta, having users as beta testers. But works well enough for the most common cases.

Azure is the worst. No one in his sane mind would have a reason to go there. Seriously. Things does not work regularly without a good reason. Randomly. Even their own administration portal randomly crash, randomly fail to apply changes, consume a lot of memory in your browser. Have stupid rough edge everywhere. Like it is cloud but still you can't "migrate" a webapp install to another plan or region. Just impossible. You have to manually copy it. And by manually I mean that it is not even good at exporting a complete config like yaml or json to reload it somewhere else.

Azure as so much business just because they give free credits or pay consulting for customers to switch, and also have managed to have big corporation to sign contracts with them. The one that use office 365.

And when you are a dev or manager in such a big corp, it is easy to request a subscription with the corp account that requesting the company to contract with AWS. So you go with the flow to get the job done.

AWS is the nice in between between the 2. They have services that are rarely upgraded with better features but have a lot of services that just work correctly mostly. And administrating your account through the interface or code logic is basically guaranteed to work.

Azure works well enough when Microsoft's footing the bill!

hoofhearted · 2 years ago

If you have an understanding of basic computer science, full-stack development, and the type of different services required to run an app; then you should be fine if you have experience with one versus the other.

Each cloud provider has their different features and quirks, but their underlying services are pretty much the same when it comes to managed servers for computing, data storage, caching, queuing, notifications, email, network policy, etc.

Basically what I am saying is that if you have a deep understanding of AWS or GCP, what each service does, and how billing works; then it just comes down to what each provider names their service, such as RDS or S3.

If you are trying to jumpstart your career with AWS and already have a deep understanding of GCP, then watch some quick AWS cloud practitioner guides on YouTube. If you feel comfortable enough with what they are talking about, then you can go take the cloud practitioner certification test in person next week and add that certification on your resume for less than $200.

swozey · 2 years ago

I've worked with and on GCP and with AWS for at least 15 years combined. I've worked with Azure maybe 10-15 times and am almost completely lost by its interface each time. Whether that's just inexperience or their UX I couldn't tell you. AWS has some frustrating UX. GCP is the simplest comparatively.

I don't have a real solid reason why, maybe I'm just the lack of comfort/experience but I have no interest working with any Azure environment. Maybe I just don't have / need to learn another since I can always find work using those. You will absolutely find less GCP jobs than Aws or Azure.

SRE w/ about 20 years exp in cloud/distsys/datacenters