Readit News logoReadit News
zzzcpan commented on Turns out half the internet has a single-point-of-failure called “Cloudflare”   easydns.com/blog/2020/07/... · Posted by u/StuntPope
snowwrestler · 5 years ago
Unless you're sending all your traffic back to physical machines that you own locked into a cage in a datacenter, you are probably letting someone MITM your SSL traffic. For example if you are hosting on AWS, Amazon has access to your keys. If you are hosting on a hardware server leased from Hetzner, Hetzner has access to your keys.

When a 3rd party has access to your keys, their responsibilities to you are spelled out in your contract with them. That's true for CDNs as well as hosting companies.

zzzcpan · 5 years ago
It's more complicated.

For most websites today if someone can intercept traffic somewhere close to the server they don't even need the keys, they can just fake responses to pass CA validation and issue valid certificates with their own keys and MITM like there is no encryption.

And coldboot attacks performed by a hosting provider staff of dumping memory and finding keys isn't that realistic of a threat, just like putting servers into a locked cage on someone else's property isn't much of a protection.

zzzcpan commented on Fact-check of viral climate misinformation quietly removed from Facebook   popular.info/p/fact-check... · Posted by u/MaysonL
gh02t · 5 years ago
There is no real debate about whether climate change is real nor whether or not it is a major threat, but there is still debate on policy-relevant points. I'm not a climate scientist myself, but I do work with a bunch of them and from what I gather from talking to them there is still a lot of work and debate over consequences, remediation, and urgency. The latter mostly ranging from "extremely urgent but we still have time to avoid the most severe consequences" to "it's too late to prevent widespread disasters and we need to try to minimize the worst."
zzzcpan · 5 years ago
Right, there is plenty of confidence that climate change is real and caused by humans, but pretty much zero confidence in the long term effects of climate change on human population.
zzzcpan commented on Cloudflare outage on July 17, 2020   blog.cloudflare.com/cloud... · Posted by u/tomklein
katzgrau · 5 years ago
... and after you've solved for the truck problem, you have a potentially infinite list of other things to plan for, some of which you will not foresee. And of course, there's probably an upper bound on the time you can spend preparing for such things.

Famous to the point of being a cliche, the titanic was thought to be unsinkable, and I would have a similarly hard time convincing the engineers behind the ship's design to believe otherwise.

The level of confidence you're displaying in predicting the unforeseeable is something you may want to take a deeper look at.

zzzcpan · 5 years ago
You are missing the point. Solving the truck problem is exactly what you shouldn't do, well, at least until your system is resilient. Because it could be something entirely different, it could be law enforcement raiding a data center and your wall around it won't protect it from them. So instead you approach the system in terms of what it has to rely on and all possible states of the thing it has to rely on. Which maps to a very small number of decisions. Like whether a server is available or not. If it's not available it really doesn't matter which of the infinite things that could happen to it or to a data center it is in actually did, you simply don't return it to users if it's not available and have enough independent servers to return to users in enough independent data centers to achieve specific availability. It's really not difficult.

I understand that most of those leetcode corporations don't care much about resilience, likely even incapable of producing highly reliable systems, and may give you a false impression that reliability is something of an unachievable fantasy. But it's not, it's something we have enough research done on and can do really well today if needed, we are not in titanic era anymore.

I have high confidence in these things (not in "predicting the unforeseeable"), because I've done them myself. My edge infrastructure had like half an hour of downtime total in many years, almost a decade already.

zzzcpan commented on Cloudflare outage on July 17, 2020   blog.cloudflare.com/cloud... · Posted by u/tomklein
katzgrau · 5 years ago
> The whole idea behind resilience is to cover unforeseeable risks

Speaking of things that don't make sense... if it's unforeseeable, one will have a difficult time adequately preparing for it

zzzcpan · 5 years ago
It's not difficult, it's just different. It's the difference between predicting that a truck might crash into a data center and building concrete wall around it, and designing a system in a such way that users only ever resolve to servers that are currently available regardless of what happened to some of them in a data center that had a truck crashed into it.
zzzcpan commented on Cloudflare outage on July 17, 2020   blog.cloudflare.com/cloud... · Posted by u/tomklein
katzgrau · 5 years ago
It does make sense, and it's paradoxical, I know.

> If you design for resilience, you get more resilience and you build confidence as you see the evidence how the system works in real world.

You simply can't foresee or eliminate all risk. This is referred to as "the turkey problem." It's not my idea, but one I certainly subscribe to.

https://www.convexresearch.com.br/en/insights/the-turkey-pro...

zzzcpan · 5 years ago
The whole idea behind resilience is to cover unforeseeable risks, the turkey problem just doesn't apply here. I would even say if the system doesn't solve the turkey problem it cannot be called resilient. And high availability without resilience is not practically possible.
zzzcpan commented on Cloudflare outage on July 17, 2020   blog.cloudflare.com/cloud... · Posted by u/tomklein
katzgrau · 5 years ago
And you will still be exposed to being blindsided by something out of your control. It's really only in your control of you can think of and plan for it ahead of time. And there will certainly be things that we don't consider. You can call that a failure but it happens all the time and it's reality.

What if a political event impacts you, for instance? A pandemic? A storm taking out a major data center? A weird Linux kernel edge case that only happens beyond a certain point in time? That only sounds ridiculous because it hasn't happened, but weird things like that happen all the time. There are so many unseen possibilities.

I understand that might sound unreasonable or facetious or like I'm expanding the scope.

The point is, the more confident that you've built something that has no SPOF the more exposed your are to the risk of it, because one probably does exist.

zzzcpan · 5 years ago
Honestly, you are not making any sense. This is not how engineering works. If you design for resilience, you get more resilience and you build confidence as you see the evidence how the system works in real world. Furthermore, with resilience you have to always cover all risks, it's just that you don't immediately reach fine granularity of decisions that don't trigger failover to servers in different countries, you improve granularity as you learn from actual operations and modify your designs accordingly.

I remember when I first deployed DNS routed system it was too reactive, constantly jumping between servers, monitoring was too sensitive, it didn't wait for servers to stabilize to return them into the mix and exponential backoff was taking servers out for far too long. But even given all that it was still able to avoid outages caused by data center failures and connectivity problems.

zzzcpan commented on Cloudflare outage on July 17, 2020   blog.cloudflare.com/cloud... · Posted by u/tomklein
katzgrau · 5 years ago
I'd argue that you're probably holding yourself to a standard that is more/less unachievable in such an interdependent world. It's idealistic and idealism is a square peg in the funny shaped hole of reality.

Taking accountability and having backup plans are extremely important, but you simply can't remove every last shred of dependence. You eventually have to accept that there are things that are out of your control and may take you by surprise despite best efforts.

zzzcpan · 5 years ago
In web and online infrastructure pretty much nothing is out of your control except for two things: ISPs people use and domain name registrar you use for your domain name. And even domain name registrar centralization can be mitigated against by having multiple domains from multiple registrars and promoting different domains to different users and having backup communication channels to inform users about new domains in case something happens.

Other than that it's your choice whether to make your infrastructure dependent on a bunch of unreliable centralized SPOFs from big corporations or build highly available infrastructure relying on servers from many different providers running your own DNS servers with DNS routing, failover, etc. You will definitely beat Cloudflare's availability this way many times over.

zzzcpan commented on It pays to be grumpy and bad-tempered (2016)   bbc.com/future/article/20... · Posted by u/Hnaomyiph
nine_k · 5 years ago
Indeed, to strive to (even more of) perfection, you need to be discontent with the current state of things.

That is, you have to be discontent with things like 99% of the time. Because of this, you'll feel compelled to improve them. But the price of it would be being less than happy most of the time.

zzzcpan · 5 years ago
Wouldn't that make you more happy, not less? Always figuring out how to improve things and having endless possibilities to improve them is almost a definition of fun.
zzzcpan commented on How objectivity in journalism became a matter of opinion   economist.com/books-and-a... · Posted by u/pseudolus
benlumen · 5 years ago
In my opinion, once-objective mainstream journalism is now about writing content to please target audiences and catch trending topics on social media, as well as keep fast cycles going to appease the algorithms on news aggregation sites and search engines.

The names of once-trusted news companies has stayed the same, but it's about the only thing about them that has.

I believe the tipping point was smartphones, and find it very ironic that Steve Jobs showed off iPhone's ability to load up The New York Times in its reveal keynote in 2007.

This was the exact topic of my first substack piece on Monday if anyone is interested. https://benlumen.substack.com/p/thank-god-i-never-went-into-...

I did feel a little vindicated reading Bari Weiss' NYT resignation letter the next day saying that "Twitter has become its ultimate editor".

zzzcpan · 5 years ago
> once-objective mainstream journalism

Objective journalism was never ever a thing. That's why Manufacturing Consent happened and all the works from Edward Bernays and all the way to Noam Chomsky.

What journalism had before though is just more consistency in worldview, because mass media was very centralized and pushed much more consistent propaganda with nothing to oppose it.

zzzcpan commented on Why is the Fessenheim 2 nuclear power plant closing in France?   sustainability-times.com/... · Posted by u/ericdanielski
eqvinox · 5 years ago
I have no idea whether it's even possible to find out how the different reasons have factored into the decision; it might as well just have been due to the age or seismic activity and nothing else...

But, in reality, it was probably a mix of all 3 and some more.

zzzcpan · 5 years ago
It's more like a mix of solar and wind energy investors with fossil burning energy investors, both benefit from shutting down nuclear reactors. Solar and wind investors just want to have their huge returns with nothing wasted, as each kWh is pretty expensive, but lacking nuclear power most of the energy generation will still go to burning fossils, who will profit massively from it. Happened in other countries too, like Ukraine, which was recently forced to temporary stop some reactors to benefit those two groups and of course make things worse for the climate.

u/zzzcpan

KarmaCake day4245December 27, 2011View Original