A database migration caused the www.cloudflare.com name to get directed (along with a small number of other names). Didn't affect api.cloudflare.com or dash.cloudflare.com, etc.
Totally our doing and not a hack of any kind. Team is writing this up and we'll blog about what went wrong.
Thanks. Curious: Were DNSSEC validating clients failing to redirect to clickfunnels.com too, or was the misconfiguration on Cloudflare's end such that they weren't?
If it wasn't DNS, then I guess it must have been the reverse-proxy? In that case, if clickfunnels had uploaded their own public CA (or hosted it on pages.dev, which doesn't add sni.cloudflaressl.com to common-name/dns-name), would the redirection still have been without browser errors?
Oh wow. I'm pinging the folks I know at CF, but it looks like they got owned. A quick dig shows that cloudflare.com points to the same server running clickfunnels.com; not sure what that indicates about the attack.
EDIT: scratch that, different IP.
EDIT 2: A CURL to www.cloudflare.com gives an error page. A CURL with the FF agent header gives the redirect.
EDIT 3: Word is, not a hack. Configuration mistake.
Something similar happened at the CDN I work for about 10 years ago… a custom configuration we made for a customer had a typo in it (it was on the host matching regex… the regex had a trailing |, which caused it to match every host)… this caused every request for any customer to be sent to that one customer. It quickly overwhelmed their origin and caused an outage (the largest one we have ever had, before or since). We wrote a system that is still in place today that loads all new configurations and sends test traffic to it, to make sure the results are as we expect.
They did, but I'm not sure it's referencing this incident. Doing a cursory look back through their Twitter timeline, the account seems to tweet twice every business day.
I don't believe for one second that clickfunnels would have done this on purpose. Not that I know them personally, just that it would be a huge backfire if they had. My money's on clickfunnels being a cloudflare client that just happens to be the recipient of some description of forwarder misconfiguration.
But more realistically, I imagine that if this was deliberate they would be sued into oblivion and criminal charges might even be pressed. Don't mess with other rich people.
Readit News
Totally our doing and not a hack of any kind. Team is writing this up and we'll blog about what went wrong.
Sorry about this error.
If it wasn't DNS, then I guess it must have been the reverse-proxy? In that case, if clickfunnels had uploaded their own public CA (or hosted it on pages.dev, which doesn't add sni.cloudflaressl.com to common-name/dns-name), would the redirection still have been without browser errors?
EDIT: scratch that, different IP.
EDIT 2: A CURL to www.cloudflare.com gives an error page. A CURL with the FF agent header gives the redirect.
EDIT 3: Word is, not a hack. Configuration mistake.
Still, not a good look, is it?
But more realistically, I imagine that if this was deliberate they would be sued into oblivion and criminal charges might even be pressed. Don't mess with other rich people.
[1] https://twitter.com/NateSmoyer/status/1485750837322215424
https://twitter.com/Yank/status/1485763736103096320 claims to be a Cloudflare community person:
> It was a configuration issue that was limited to the marketing site. The dashboard and all customer sites and services were not impacted.
wget cloudflare.com --2022-01-24 23:21:32-- http://cloudflare.com/ Resolving cloudflare.com... 104.16.132.229, 104.16.133.229, 2606:4700::6810:85e5, ... Connecting to cloudflare.com|104.16.132.229|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://www.cloudflare.com/ [following] --2022-01-24 23:21:32-- https://www.cloudflare.com/ Resolving www.cloudflare.com... 104.16.123.96, 104.16.124.96, 2606:4700::6810:7b60, ... Connecting to www.cloudflare.com|104.16.123.96|:443... HTTP request sent, awaiting response... 302 Found Location: https://www.clickfunnels.com?aff_sub=domain_redirect&utm_cam... [following] --2022-01-24 23:21:33-- https://www.clickfunnels.com/?aff_sub=domain_redirect&utm_ca... Resolving www.clickfunnels.com... 104.16.16.194, 104.16.14.194, 104.16.15.194, ... Connecting to www.clickfunnels.com|104.16.16.194|:443... connected. HTTP request sent, awaiting response... 200 OK