I find it pretty cool that you went through the length of publicly announcing it and everything for something that everyone else was doing already. That does show some integrity in face of "let's just make it work" people.

None-the-less, as a web developer (which I am less than 5% of my programming time, probably less, so I'd rather not lose my hair over adjacent stuff) this has been extremely frustrating to debug in the past. Would it be possible/make sense to show maybe a small yellow (i) as like a 8x8px icons at the bottom right of the lock to show that something's up? And then if the user clicks "more information" on the "secure connection" (which I believe is deep enough to be more power-user/developer-centered) show a yellow line saying "you're certificate chain's broken dude"?

I haven't hit this issue in several years, and it's unlikely I will hit it again because I'm now systematically using caddy as a front which will do TheRightThing (or so I hope), so maybe my comment is out-dated/irrelevant, in which case, feel free to ignore me.

> Firefox can just ask the root CA to notify them when they issue an intermediate

In fact notifying m.d.s.policy is already required whenever such an intermediate is created

5.3.2 "The operator of a CA certificate included in Mozilla’s root store MUST publicly disclose in the CCADB all CA certificates it issues that chain up to that CA certificate trusted in Mozilla’s root store that are technically capable of issuing working server or email certificates,"

A common way that these work is that 1 browser does it, then if the others don't copy they appear "broken" to users.

IDK what happened in this case but it is pretty easy to imagine Chrome accidentally allowed validation against certificates in its local cache. Maybe it added some sort of validation cache to avoid rechecking revocation lists and OSCP or similar and it would use intermediates from other sites. Then people tested their site in Chrome and it seemed to work. Now Firefox seems broken if they don't support this. So they decided to implement this and do something more robust by preloading a fixed list rather than whatever happens to be in the cache.

Basically no browser wants to be the first to stop supporting this hack.

This is ultimately an application of the "robustness principle" or Poestel's law, which was how people build stuff in the early Internet.

Plenty of people believe these days that this was never a wise guideline to begin with (see https://www.ietf.org/archive/id/draft-iab-protocol-maintenan... which unfortunately never made it to an RFC). However, one of the problems is that once you started accepting misconfigurations, it's hard to change your defaults.

It's Postel's Law being bad advice yet again. No, you should not be liberal in what you accept, because being liberal in what you accept causes even more malformed data to appear in the ecosystem.

Because it wasn’t actually a server misconfiguration, nor was it, as others have speculated, about Postel’s Law.

The way X.509 was designed - to the very first version - was the notion that you have your set of CAs you trust, I have my set, and they’re different. Instead of using The Directory to resolve the path from your cert to someone I trust, PKIX (RFC 2459-et-al) defined AIA.

So the intent here was that there’s no “one right chain to rule them all”: there’s _your_ chain to your root, _my_ chain to my root, all for the same cert, using cross-certificates.

Browsers adopted X.509 before PKIX existed, and they assumed just enough of the model to get things to work. The standards were developed after, and the major vendors didn’t all update their code to match the standards. Microsoft, Sun, and many government focused customers did (and used the NIST PKITS test to prove it), Netscape/later Mozilla and OpenSSL did not: they kept their existing “works for me” implementations.

https://medium.com/@sleevi_/path-building-vs-path-verifying-... Discusses this a bit more. In modern times, the TLS RFCs better reflect that there’s no “one right chain to rule them all”. Even if you or I aren’t running our own roots that we use to cross-sign CAs we trust, we still have different browsers/trust stores taking different paths, and even in the same browser, different versions of the trust store necessitating different intermediates.

TLS has no way of negotiating what the _client’s_ trust store is in a performant, privacy-preserving way. https://datatracker.ietf.org/doc/draft-kampanakis-tls-scas-l... or https://datatracker.ietf.org/doc/draft-davidben-tls-merkle-t... are explorations of the problem space above, though: how to have the server understand what the client will trust, so it can send the right certificate (… and omit the chain)

In my hosting days, we relied on the ssl checker that ssl-shopper has. Browser was never considered a valid test for us. It was final validation, but a proper ssl checker was the real test

As @kevincox says there's a problem where if one browser does it, then users complain a site "works in this-generations-IE" forcing the other browsers to duplicate the behaviour.

But one other problem that happens isn't necessarily "browser 1 fixed this configuration" and browser 2 copied them. It can (and often was) "browser 1 has a bug that means this is broken configuration works", and now for compatibility browser 2 implements the same behaviour. Then browser 1 finds this bug, goes to fix it, and finds that there are sites that depend on it and it also works in other browsers so even if it started off as a bug they can no longer fix it.

That's why there's increasing amounts of work involved in trying to ensure new specifications are free of ambiguities and such now before they're actually turned on by default. Even now thought you still have places where the spec has ambiguity/gaps in the specification where people will go "whatever IE/Chrome does is correct" even if the specification has a gap/ambiguity that allows different behaviour (which these days is considered a specification bug), even if other browsers agree, it's super easy for a developer to say "the most common browser is definitionally the correct implementation".

Back when I worked on engines and in committees I probably spent cumulatively more than a year do nothing but going through specification gaps working out what behaviour was _required_ to ensure sufficiently compatible behavior between different browsers. I spent months on key events and key codes alone, trying to work out which events need to be sent, which key codes, how IM/IME (input method [editor] mechanism used for non-latin text) systems interact with it, etc. As part of this I added the ability to create IMEs in javascript to the webkit test infrastructure because otherwise it was super easy to break random IMEs because they all behave completely differently in response to single key presses.

It's very difficult in practice to shift the blame to the website. Even though the browser would be right in refusing connection, the net effect is that the user would just use another browser to access that website. The proper workaround (Firefox shipping intermediate certificates), doesn't actually damage security. It just means more work for the maintainers. That's a fair tradeoff for achieving more market share.

It's the same reason why browsers must be able to robustly digest HTML5 tagsoup instead of just blanking out, which is how a conforming XML processor would have to react.

Do browsers or is this another OpenSSL Easter egg we all have to live with?

I remember that OpenSSL also validates certificate chains with duplicates, despite that obviously breaking the chain property. That’s wasteful but also very annoying because TLS libraries like BearSSL don’t (I guess you could hack it and remember the previous hash and stay fixed space).

Maybe in the modem days the smaller certificate was considered ideal for connection overhead?

It wasn't long ago when TLS was not the norm and many, many sites were served over plain HTTP, even when they accepted logins or contained other sensitive data. There's a good chance this decision was a trade-off to make TLS simpler to get working in order to get more sites using it.

Browsers have a long history of accepting bad data, including malformed headers, invalid HTML, and maintaining workarounds for long-since-fixed bugs. This isn't really that different.

I was wondering recently if it's better to use wildcard certs because of this. On the other hand, all sub-domains are discoverable through DNS anyways. Does it then make a difference if the sub-domains are logged in the certificate transparency logs?

The title was "The browsers biggest TLS mistake"...

This strikes me as interesting even if it's a field I have very light understanding, and feel like I might fall victim to this. Asking for a friend, but if that friend uses Let's Encrypt to create certs for subdomains on a single vhost, what would that friend need to do to see the information you are seeing?