And since we're adding random related things, lcamtuf's museum of broken packets is an old, but always interesting read: http://lcamtuf.coredump.cx/mobp/
What is the general lesson we should learn from this?
Postel's law aka robustness principle [1] can easily lead into accumulating complexity when implementations adapt to the bugs in other implementations. How could protocol designers mitigate this problem beforehand?
The lesson is "be strict in what you accept, since the beginning".
For instance, a lesson learned by the Linux kernel developers: when adding a new system call to an operating system, if you have a flags parameter (and you should, which is another lesson they learned), if any unknown flag is set, fail with -EINVAL (or equivalent). Otherwise, buggy programs will depend on these flags being ignored, and you won't be able to use them later for new features.
But it has to be since the beginning; once the protocol is "in the field", you can't increase the strictness without pain.
Otherwise, buggy programs will depend on these flags being ignored, and you won't be able to use them later for new features.
I don't understand why that's true. Just add the new flag and use it for your new feature. Aren't buggy programs who were already sending the flag responsible for their own bugs?
I'd say one of the lessons is that even a supposedly well-standardised system sees hundreds of implementations (or more!) then the accumulated bug baggage can still make it hacky with per-platform code. For comparison consider web browsers, which although are far better these days than they used to be, between just Chrome, Firefox, IE and Safari there's a bunch of quirks and platform-specifics, so I can imagine worldwide TCP deployments are "interesting".
Where possible hacks should be applied only where necessary, e.g. the specific software versions affected only, and exclude fixed versions. Then hopefully in the long run the old buggy versions die out and the hack can be removed... but as the article says, over a network it's not always possible to identify when to apply a hack.
I've to confess I've quite limited knowledge with TCP/IP stack internals, e.g. the way stack extensions work et cetera.
Does anyone know of any available online visual materials/tutorials? I'm particularly searching for tools capable of recording and replaying TCP/IP stack packets with visual representation, references to RFCs and specifications.
Richard Steven's "TCP/IP Illustrated" books are the original works, Unix network programming at its finest. More advanced than Comer's book, and much more applicable. Get used, older editions if cheaper, you will be fine.
Pretty much everything in technology is harder than it looks. Took me a day to configure Apache Solr for the first time the other day while my estimate was just one hour.
SYN-ACK which does not get retransmitted, advertises a zero window and does not have options looks like some implementation of SYN cookies. A nice (and useful in some cases) hack but a violation of the TCP spec, which is why it is disabled by default in most places that implement it.
Readit News
http://revealingerrors.com/google_miscalculator
Postel's law aka robustness principle [1] can easily lead into accumulating complexity when implementations adapt to the bugs in other implementations. How could protocol designers mitigate this problem beforehand?
[1]: https://en.wikipedia.org/wiki/Robustness_principle
For instance, a lesson learned by the Linux kernel developers: when adding a new system call to an operating system, if you have a flags parameter (and you should, which is another lesson they learned), if any unknown flag is set, fail with -EINVAL (or equivalent). Otherwise, buggy programs will depend on these flags being ignored, and you won't be able to use them later for new features.
But it has to be since the beginning; once the protocol is "in the field", you can't increase the strictness without pain.
I don't understand why that's true. Just add the new flag and use it for your new feature. Aren't buggy programs who were already sending the flag responsible for their own bugs?
Where possible hacks should be applied only where necessary, e.g. the specific software versions affected only, and exclude fixed versions. Then hopefully in the long run the old buggy versions die out and the hack can be removed... but as the article says, over a network it's not always possible to identify when to apply a hack.
Does anyone know of any available online visual materials/tutorials? I'm particularly searching for tools capable of recording and replaying TCP/IP stack packets with visual representation, references to RFCs and specifications.
https://www.cs.purdue.edu/homes/comer/netbooks.html
Richard Steven's "TCP/IP Illustrated" books are the original works, Unix network programming at its finest. More advanced than Comer's book, and much more applicable. Get used, older editions if cheaper, you will be fine.
https://en.wikipedia.org/wiki/Christmas_tree_packet
https://en.wikipedia.org/wiki/Ping_of_death