Readit News logoReadit News
Posted by u/klinskyc 3 years ago
Tell HN: CloudFront Is Having an Outage
Sites hosted via Cloudfront

(e.g https://app.circleci.com/) are having an outage and returning 421s and 500s currently

akhayam · 3 years ago
AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
NullInvictus · 3 years ago
Short of something within a few hundred feet of Bezo's desk being actually on fire, AWS's health page never seems to actually report issues.
scrum-treats · 3 years ago
Bezos is no longer the CEO. It's Andy Jassy.

I agree, AWS (and Amazon Alexa + Ring) seem to be skirting and/or choosing active silence on that "Lambda" outage that took out us-east-1. Alexa had at the very least two full scale outages this year, and Ring was successfully hacked as well.

Amazon's response: "looks like a Lambda... more later" then nothing, nothing, and "it must be a 3rd party skill."

AWS health page: GREEN.

AWS Post-event summaries: non-existent after 2021.

This is quite troubling when you consider what AWS holds in its coffers, along with the number of customers AWS currently has. No transparency, not even strategic transparency, is a definite red flag.

pgn674 · 3 years ago
Got posted a minute ago.

Operational issue - Amazon CloudFront (Global) Service Amazon CloudFront Severity Informational RSS Elevated Error Rates Jul 18 10:26 AM PDT Between 9:37 AM and 10:13 AM PDT, we experienced elevated error rates for request serviced by the CloudFront Origin Shield and Regional Edge Cache in the US-EAST-1 region. The issue has been resolved and service is operating normally.

ssabev · 3 years ago
"elevated error rates" is now my favourite euphemism for a downtime
temp0826 · 3 years ago
FYI, it takes some pretty high-level approval at AWS for someone to make a change on their "status" page (signaling SLA breaches and money stuff). Not an actual live status page.
snacktaster · 3 years ago
All our HTTPS (ACM) sites are returning:

> 421 ERROR

> The request could not be satisfied.

> The distribution does not match the certificate for which the HTTPS connection was established with. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.

akhayam · 3 years ago
Lots of users seem to be impacted for a short time period: https://isdown.app/integrations/aws/amazon-cloudfront
ssabev · 3 years ago
Well, that explains wtf is happening. You guys seeing this on us-east-1 or across?
neuronexmachina · 3 years ago
As the saying goes: Friends don't let friends rely on us-east-1. (It was the very first region, and it shows)
messe · 3 years ago
Having worked at AWS: us-east-1 is probably the easiest region to deploy in (it's old, it's popular, every AWS service is available there), but might just be the worst region to rely on (it's old, it's popular, and every AWS service is available there—but only most of the time).

I vividly remember when AWS Fault Injection Simulator first launched, posting a meme in the #aws-memes slack channel posting a drake meme with the first panel being "Using AWS Fault Injection Simulator", and the second "Deploying in us-east-1".

hinkley · 3 years ago
I will never understand why anyone in our infrastructure team thought for a moment that putting our backup datacenter in us-east-1. And why nobody else tried to get him fired.

That's like buying a backup generator from a guy that wants to meet you in the Walmart parking lot.

flangola7 · 3 years ago
Seeing it across 3 regions at minimum
Aaronstotle · 3 years ago
saw it across us-east-1 mainly
romanhotsiy · 3 years ago
us-east-1 only for us
dave4420 · 3 years ago
My employer’s sites are hosted via CloudFront and seem to be fine. The underlying buckets are in Dublin, in case that makes any difference.
gregmfoster · 3 years ago
Seeing 421 on our AWS amplify splash page as well
klinskyc · 3 years ago
It looks like they are back up now
sgammon · 3 years ago
Seems stable now