The issue started occurring intermittently at 08:26 US/Pacific. The issue is impacting Google’s Backbone network and may impact various services when accessing them from a different region or from the internet. Impacted services include Cloud Services (Workspace, Firebase, GCP) as well as other Google properties.
Connectivity within a zone should not be impacted.
Our engineering team has implemented a mitigation and is now monitoring the effectiveness of the change.
Why the most GCP outages are global? Even though it says issue started in US/Pacific, I'm not sure if it affects other regions and the status page of them does not make it clear.
Because Google engineers think that they are the smartest people in the world which means they don't actually engineer for minimization of a blast radius.
You should just see how Google builds the prefix lists that it allows customers to advertise to Google via PNIs. You would think they are building it off some registry lists because why else would Google insists that the routes are registered? Oh no, that's for small people. At Google they just bring up a session to you, let you advertise and take whatever you advertised to them during the setup as the allow list because everything is done via custom automated software!
Does Cloudflare use Google Cloud? I know they have a ton of their own hardware, but maybe they outsource some things?
I had my first ever Cloudflare outage earlier today. It's just a free site that has a single page rule that redirects to a different domain, but it started refusing connections for a little while today.
Deleted Comment
Connectivity within a zone should not be impacted.
Our engineering team has implemented a mitigation and is now monitoring the effectiveness of the change.
You should just see how Google builds the prefix lists that it allows customers to advertise to Google via PNIs. You would think they are building it off some registry lists because why else would Google insists that the routes are registered? Oh no, that's for small people. At Google they just bring up a session to you, let you advertise and take whatever you advertised to them during the setup as the allow list because everything is done via custom automated software!
I wish I could say that to my customers...
I had my first ever Cloudflare outage earlier today. It's just a free site that has a single page rule that redirects to a different domain, but it started refusing connections for a little while today.
https://cloud.google.com/press-releases/2020/0124/abk-and-go...
Edit: things started working now.