One approach to guard against Hyrum’s Law is GREASE (aka “Generate Random Extensions And Sustain Extensibility” used in the TLS 1.3 protocol) i.e. behavior randomization to avoid inadvertent dependencies on unspecified behavior: https://textslashplain.com/2020/05/18/a-bit-of-grease-keeps-...
(Which I find a hilarious second order example of Hyrum's law - if you add true randomness to prevent people from depending on the iteration order, they might use it as a way to randomly access items in the map!)
This was mostly annoying because Go doesn't really provide any ordered map in std (and for while didn't give you generics, which are needed to make a general-purpose one that's perfomant). I feel like the best response to people relying on the order of iteration is to recognize that as a valid need an provide it via a separate type.
Matt Kulukundis (sp?) mentions this problem is his talk about Google's rather fancier Swiss tables.
At that point their debug builds would do a "coin toss" with 50.3% chance to be heads during hash table insertion and he expected some idiots would use that to generate random bits then be annoyed if it breaks. I believe in production it's entirely seeded from ASLR, so you're actually leaking address layout info if you try to use these bits as entropy instead of just to make the hash table defeat your bad unit tests.
Our team went with a similar approach when refactoring Protobuf debug APIs (https://bughunters.google.com/blog/6405366705946624/fixing-d...). People were relying on debug output and trying to parse it, so in the new implementation we threw up big warning flags and made the output unstable so that you couldn't make the mistake.
The key lesson to take away is: if you want something to be an implementation detail, make sure to have multiple differing implementations :)
When people don't know which implementation will be used, they tend to stick to the standard or the documentation.
We see this in places like the C language or in Common Lisp. Library writers must take into account that the implementation they use their library on may not be the same as the implementation their users will use.
Telling people who depended on probably internal behavior that wasn’t an explicit part of your contract to fuck off when they whine about you changing something.
When the clients have the power (e.g. a team that contributes to 90% of company's revenue), breaking the contract is often non-option from the beginning.
What are some fun or unusual examples of Hyrum’s Law that you’ve run into? Were you the user, inadvertently depending on some implementation detail? Or were users depending on your implementation’s details?
I reviewed a CL from Hyrum at Google where he was trying to remove a `set_timeout(float)` method in favor of `set_timeout(absl::Duration)` and changed the former to delegate to the latter. It turned out that there was some special handling of inf/nan in the legacy API, despite no mention in the documentation, and his CL broke a number of tests. It was amusing to experience Hyrum's law so directly :)
> I guess the linux kernel still supprorting binaries from ages ago might be too.
Actually it's Linus making sure that kernel changes don't break old binaries from ages ago. And in fact there were many cases of undocumented behavior being relied on by apps, and Linus would send big rants about how you never break userspace, whether the behavior is documented or not.
Don't have any of them on hand right now, unfortunately.
At Google (long time off and on home of Hyrum), they migrated to swisstable from other hash maps. Unordered map iterates through the elements in insertion order. They explicitly couldn't support that without adding overhead to their new table so they randomized iteration order of the map
The migrations to the new table were fun because you would migrate some code and then find a test that assumed the order things were inserted into some seemingly distant protocol buffer array were in insertion order of the map
Usually the ordering was just a defect of the test, but it required actually digging in to be sure
IIRC There's a (CppCon?) talk about this rollout where Hyrum is in the audience so that he can heckle each time they describe a change which "obviously" can't break anybody because of course Hyrum's Law did cause that to break at Google when they did it.
Maybe somebody who remembers better can link it and/or correct my memory of exactly what's going on.
I joined a company once where some APIS were, for some reasons, returning strings instead of booleans (e.g. "false" instead of _false_).
One of the system returned a typo for some edge cases, e.g. "ture" instead of "true". But instead of fixing this, other systems relied on the typo to be returned, so it was difficult to just fix the typo and move on.
I implemented a development-only feature which span up a thread and permanently held a lock on the database, effectively making the app useless until restarted.
A couple months later, I had a user thank me for it. I still don't know why
Let's just say "this HTTP endpoint expects its body to be JSON" and "we use a library that allows invalid JSON" (e.g. trailing commas) aren't exactly a good combination.
Code easily ends up depending on the specific library's quirks and config, regardless of what the actual contract said.
I'm not trying to be overly dramatic, but I think it's precisely when the industry accepted this as a law -- instead of treating it as something that needs be trained out of junior programmers -- is when software quality started to tank.
Consider any other form of engineering. Take some kind of screw. It has a documented specification in terms of torque, material strength and whatnot. Good engineers on the customer side will use the screws in a way that keeps within the spec. And good engineers on the supplier side will find ways to fulfill that specification as cheaply (which usually also means as narrowly) as possible.
There could be a kind of Hyrum's Law at play, if hardware engineers were idiots. Let's say that the screws accidentally overfulfil the specification today by 20%, and a customer measures the material to figure this out, and starts to depend on that. A year later, the supplier finds a cheaper way to produce the screws (or introduces a binning process) and as a consequence, the screws only exceed the spec by 5%, and the customer's product breaks. Who's responsible? The customer. And I should add, obviously.
This is fixed by training engineers. No one in their right mind would start to introduce artificial faults into their screws purely for the reason to prevent users from depending on the excess strength or something. But that's exactly the kind of thing that's regularly suggested to guard against Hyrum's Law.
So whenever I see software developers on my team look at the source code of a library to figure out "whether it's thread-safe" or "whether the sort order is stable" or whatever, I die a little on the inside. The problem is, you almost have to do that, because were two generations of software engineers into this and the practice is so accepted now that libraries no longer bother to document what they do (and don't) guarantee.
And then people wonder why every existing piece of software needs a fully staffed team nowadays just to keep it working. And why as a consequence, even core products built by competent companies (like Google Maps) have regressions in 15-year-old features every other week.
Yes, the relationship between "Hyrum's Law" and having a terrible documentation culture is strong.
The important part of the "Law" is the bit where it says "it does not matter what you promise in the contract".
That's definitely going to be true in a place where neither writing nor reading documentation is taken seriously (and one of the main things this site teaches us is that Google is such a place).
Yeah, no, humans can't hold complicated documentation in their heads. The whole experience of being human is finding a way to get by with a simple model which fits in your head but isn't wrong enough to cause trouble, and that's exactly what Hyrum is reflecting.
What you're getting at is the C++ "Just don't make any mistakes" approach to software engineering, which is a disaster that has cost our civilisation a great deal.
> it's precisely when the industry accepted this as a law -- instead of treating it as something that needs be trained out of junior programmers -- is when software quality started to tank.
And precisely when exactly do you think that happened, when do you think there was this golden age where people actually relied on only documented behavior?
Windows and Linux both have been bending backwards to retain all sorts of undocumented features since early/mid 90s. Glibc was notoriously hampered by emacs relying on some details of its internals. C programs relying on implictly or explicitly undefined behavior has caused endless handwringing as long as there has been a C standard. The list goes on and on. Relying on implementation details has been the modus operandi in computing since day 1.
Readit News
What are other approaches?
https://dev.to/wallyqs/gos-map-iteration-order-is-not-that-r...
(Which I find a hilarious second order example of Hyrum's law - if you add true randomness to prevent people from depending on the iteration order, they might use it as a way to randomly access items in the map!)
At that point their debug builds would do a "coin toss" with 50.3% chance to be heads during hash table insertion and he expected some idiots would use that to generate random bits then be annoyed if it breaks. I believe in production it's entirely seeded from ASLR, so you're actually leaking address layout info if you try to use these bits as entropy instead of just to make the hash table defeat your bad unit tests.
The key lesson to take away is: if you want something to be an implementation detail, make sure to have multiple differing implementations :)
Why bother? We're not even able to guard against well establish contracts from changing and breaking compatibility.
When people don't know which implementation will be used, they tend to stick to the standard or the documentation.
We see this in places like the C language or in Common Lisp. Library writers must take into account that the implementation they use their library on may not be the same as the implementation their users will use.
Deleted Comment
Deleted Comment
And lets be real. There is no contract, and any one that is made will inevitably be broken.
This should get a re-submit on HN (0 comments - https://news.ycombinator.com/item?id=16149534 4 years ago)
I wonder if it would allow for easter eggs again. Would the fabled story between Larry Osterman and Raymond Chen's Exchange POP3 server easter egg fall within GREASE - https://learn.microsoft.com/en-us/archive/blogs/larryosterma...
https://github.com/facebook/react/blob/6a44f352ecdfa93949cad...
https://github.com/pmndrs/its-fine
Git archive generation meets Hyrum's law - https://news.ycombinator.com/item?id=34631275 - Feb 2023 (76 comments)
Hyrum's Law - https://news.ycombinator.com/item?id=33283849 - Oct 2022 (53 comments)
Hyrum's Law - https://news.ycombinator.com/item?id=29848295 - Jan 2022 (36 comments)
Hyrum's Law - https://news.ycombinator.com/item?id=27386818 - June 2021 (5 comments)
Hyrum's Law: An Observation on Software Engineering - https://news.ycombinator.com/item?id=21515225 - Nov 2019 (6 comments)
Hyrum's Law - https://news.ycombinator.com/item?id=19249199 - Feb 2019 (1 comment)
https://twitter.com/pwnallthethings/status/13632600649293620...
I guess the linux kernel still supprorting binaries from ages ago might be too.
Actually it's Linus making sure that kernel changes don't break old binaries from ages ago. And in fact there were many cases of undocumented behavior being relied on by apps, and Linus would send big rants about how you never break userspace, whether the behavior is documented or not.
Don't have any of them on hand right now, unfortunately.
The migrations to the new table were fun because you would migrate some code and then find a test that assumed the order things were inserted into some seemingly distant protocol buffer array were in insertion order of the map
Usually the ordering was just a defect of the test, but it required actually digging in to be sure
Maybe somebody who remembers better can link it and/or correct my memory of exactly what's going on.
Deleted Comment
Needless to say I didn't stick around.
A couple months later, I had a user thank me for it. I still don't know why
Code easily ends up depending on the specific library's quirks and config, regardless of what the actual contract said.
https://en.m.wikipedia.org/wiki/The_purpose_of_a_system_is_w...
Consider any other form of engineering. Take some kind of screw. It has a documented specification in terms of torque, material strength and whatnot. Good engineers on the customer side will use the screws in a way that keeps within the spec. And good engineers on the supplier side will find ways to fulfill that specification as cheaply (which usually also means as narrowly) as possible.
There could be a kind of Hyrum's Law at play, if hardware engineers were idiots. Let's say that the screws accidentally overfulfil the specification today by 20%, and a customer measures the material to figure this out, and starts to depend on that. A year later, the supplier finds a cheaper way to produce the screws (or introduces a binning process) and as a consequence, the screws only exceed the spec by 5%, and the customer's product breaks. Who's responsible? The customer. And I should add, obviously.
This is fixed by training engineers. No one in their right mind would start to introduce artificial faults into their screws purely for the reason to prevent users from depending on the excess strength or something. But that's exactly the kind of thing that's regularly suggested to guard against Hyrum's Law.
So whenever I see software developers on my team look at the source code of a library to figure out "whether it's thread-safe" or "whether the sort order is stable" or whatever, I die a little on the inside. The problem is, you almost have to do that, because were two generations of software engineers into this and the practice is so accepted now that libraries no longer bother to document what they do (and don't) guarantee.
And then people wonder why every existing piece of software needs a fully staffed team nowadays just to keep it working. And why as a consequence, even core products built by competent companies (like Google Maps) have regressions in 15-year-old features every other week.
The important part of the "Law" is the bit where it says "it does not matter what you promise in the contract".
That's definitely going to be true in a place where neither writing nor reading documentation is taken seriously (and one of the main things this site teaches us is that Google is such a place).
What you're getting at is the C++ "Just don't make any mistakes" approach to software engineering, which is a disaster that has cost our civilisation a great deal.
And precisely when exactly do you think that happened, when do you think there was this golden age where people actually relied on only documented behavior?
Windows and Linux both have been bending backwards to retain all sorts of undocumented features since early/mid 90s. Glibc was notoriously hampered by emacs relying on some details of its internals. C programs relying on implictly or explicitly undefined behavior has caused endless handwringing as long as there has been a C standard. The list goes on and on. Relying on implementation details has been the modus operandi in computing since day 1.