All Beej's guides are fantastic, but if you're interested in Network programming you could do way worse than follow this up with his guide to network concepts. https://beej.us/guide/bgnet0/
If you search Algolia for Beej you'll see his material has been on hacker news numerous times.
Anyone knows of a well-rounded guide that explains the structure of the Internet at large? I know everything listed in the one you linked, but I have a nebulous idea of how anything works after my packet exits my gateway.
So, BGP, autonomous systems, peering agreements and other site-to-site routing protocols that keep the entire Internet infrastructure ticking along.
I don't know of any one resource that gives a overview of everything, I find that browsing Wikipedia for each topic gives a good overview of each topic though, with the overview being the "Internet Architecture" category: https://en.wikipedia.org/wiki/Category:Internet_architecture
Some bookmarks I have regarding BGP and peering that dives into it a bit:
Or, as I just recently learned, you can click on the (beej.us) domain name next to the title at the top of this page and it will take you to a list of all the HN submissions for the site:
I agree with everything you said. Additionally, I think that the ZeroMQ guide is also a great resource to learn a lot of the common networking patterns, even if you don't plan on using ZeroMQ directly.
There is also a C guide [1] which has a pretty interesting email policy...
I’m generally available to help out with email questions so feel free to write in, but I can’t guarantee a response.
I lead a pretty busy life and there are times when I just can’t answer a question you have.
When that’s the case, I usually just delete the message. It’s nothing personal;
I just won’t ever have the time to give the detailed answer you require. [...]
I had a work colleague with a similarly refreshing Out-Of-Office E-mail autoresponder that included something like: "I'm on vacation. When I get back I'm not going to have time to read all my missed E-mails. So your mail is going to be ignored and deleted with the other 2,000 unread mails. If it's important, E-mail me again after I return on [date]."
Here's a product (Gmail/Outlook add-on or so): have your own LLM trained over your public texts (articles, books, blog entries, READMEs, Hacker News comments, and so on), mark the emails you can't respond to with the "GPT" label which puts the mail in a queue and if you don't remove the label in 7 days or so, it sends a response with a polite intro "I didn't have the time to get to your email, but here's what I might think as a response".
I can speak from experience that posting that type of material will generate some interesting email inquiries. I used to get email messages with a 4000-line program, multiple datasets, and "My program doesn't work. Can you tell me what the error is?" The awkwardness of deleting the message is less than the awkwardness of refusing what someone else views as a 30-second request.
When you know you definitly do not have time for an answer deleting it might unload your mental burden. Keeping hundreds of unamswered messages around might feel like an unresolved issue.
There's no functional difference between "not looking at it ever again" and "deleting it".
You should understand when they say "delete" that they mean "remove from their queue", whether it still exists in an "archive" folder, backups, trash, ... is irrelevant.
> "The way I see it, it's the educators who hold all the power in the world. No, stop laughing and hear me out. A good instructor can, over the course of their career, create hundreds of millions or even billions of dollars of GDP. Let's say a teacher teaches 40 students a year for 30 years. And Each of those students goes on to work for 40 years. And, due to the quality of instruction, they earn $10,000 more per year than they would have otherwise. That's $480 million in extra revenue. From one good instructor."
OK but providing value isn't the same as having power. It's almost the opposite thing.
When people talk about having or not having power, they rarely mean magnitude alone. The more important component is control. Can you wield your power to achieve some specific goal of yours? Can you use this power to overall improve your life, and lives of those you care about? Can you use this power to protect yourself from someone else using their power to achieve something at your expense?
Beej's teaching power only answers "Yes" to the first question here - and that's only if your goals have something to do with teaching or improving the world overall, somewhere, for some people.
Yea he's using economic output as a measure.. then equating it to having power, which is laughable.
Anyway, in an age of information abundance and videos, educators are overvalued if they just give instruction alone. It's the whole package, the checks and balances on the student, that possibly provides some sort of value. Then again, there are irregularities, like the programmer who's never been to college who contributes as much or even more.
Posted a million times before, and should be posted a million times again in the future. This is one of the only reference books I've ever used, and it got me through my first real programming job. Wonderful text. Thank you, Beej!
Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories:
1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or
2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it, or for intellectual stimulation.
(Note that in category 2 I count things like "not using protobuf/thrift/etc. for IPC.)
When are these things not true? Genuinely curious! I'm guessing interfacing with existing obscure third parties, or using obscure language environments, but what are examples of that? And what other usages are there?
I don't work with it either. But I find that learning the abstraction below where you're working at can be quite beneficial to understanding the constraints of your layer, debug, and solve problems.
E.g.: Learn the basics Transport Layer protocols (TCP/UDP) if you work with HTTP
2. Speaking unusual protocols for which (good) libraries don’t exist.
3. I think interacting with the sockets api can also be relevant for tweaking various options to get better performance (though often there are Linux defaults that can be tweaked instead)
at knownow i wrote an event-driven nonblocking http server in python with asyncore which could efficiently do comet over hundreds of concurrent http connections; existing http libraries were all blocking at the time
i also wrote a similar event-driven nonblocking http client in c for benchmarking and load-testing both that server and a different compatible one being written by another team in c++. this was nonblocking because it was important for the client to be faster than the server. at the time there was no libevent/libev/libuv so i wrote my own
at satellogic i maintained the upper layers of our cubesat space protocol stack, which provided a socket-like interface to our coap stack, which i also maintained part of
i also had to tweak kernel socket buffering settings for rabbitmq there
for wercam i'm experimenting with different approaches to ipc for efficiency; the data being transferred are pixel buffers so i don't think protobuf/thrift/etc. is going to help
for the mail server i share with some friends i wrote a milter in python to smtp-reject outside email to certain recipients unless it matched a whitelist; milter doesn't speak pb either and i couldn't find a python library
i wrote a multicast file transfer program in python (bccpo/bccpi) for copying files across the lan without having to fiddle around with ip addresses; the data being transferred is just (multicast) the ip and port and (unicast) the filename, file size, and file contents, so again i'm not sure pb would simplify anything. the hard part of getting it working was just multicast
i wrote a chat client for icb in python with asyncore, and again i couldn't find a suitable library
even when i'm using a library, it's often pretty useful to be able to read strace output or use select or poll with the file descriptor it's trying to hide from me
i don't think any of these qualify as either your category 1 or 2
i've also done a huge number of things that fall into category 2, of course, like writing an irc client in bash, writing a web server in assembly, and writing a chat server in three 80-column lines of c, writing a mud, and so on
I often work in a resource constrained environment where my program size is measured in two-digit kilobytes, RAM is measured in the single-digit kilobytes, and it can’t exceed some current measured in mA/h. When I am simply sending a UDP packet, I didn’t need nor can I afford the footprint that these libraries have.
It's useful to understand the socket API so that you can do things like read the output from strace, notice that accept() returns EAGAIN too often, and know what that indicates.
At least on Unix and Windows file read and write methods work for sockets too, so streaming bytes isn't different.
How would you implement, say, netcat? Launch a jango rest service with kubernetes, which will run in a wm if it's a wrong OS, create a certificate with letsencrypt and access it with a react electron app?
Why would I implement netcat when netcat is already implemented? That's sort of why I'm asking – at least in the circles I move, all the low hanging fruit are already picked, and picking the high fruit seems rare, so I'm interested in which people do and what those fruits are!
Had plenty of use cases where even zmq was overkill.
E.g. interfacing a research machine with an OEM machine or a machine to a standalone embedded target, sometimes it makes sense to just serialize, send over a socket, ethernet crossover cable and be done.
Your description makes it sound sort of like temporary code to perform a one-time (or few-times) job, and not something that's running and maintained for years. Is that a fair characterisation or am I misunderstanding?
I implement protocols to communicate with various hardware (RFID, LiDAR, etc). Most of these communicate over sockets. Some still use serial. I have a library I wrote of socket routines that I use, so it's not like I re-implement that every time, though. And for this, I'd really rather use my library than a 3rd party in case I have to dig in and see what's going on.
Readit News
If you search Algolia for Beej you'll see his material has been on hacker news numerous times.
So, BGP, autonomous systems, peering agreements and other site-to-site routing protocols that keep the entire Internet infrastructure ticking along.
Some bookmarks I have regarding BGP and peering that dives into it a bit:
- https://www.fortinet.com/resources/cyberglossary/bgp-border-...
- https://aws.amazon.com/blogs/architecture/internet-routing-a...
- https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bg...
I found Cantrill’s course pretty solid. It is free and you can just skip to DNS & DNSSEC.
Or, as I just recently learned, you can click on the (beej.us) domain name next to the title at the top of this page and it will take you to a list of all the HN submissions for the site:
https://news.ycombinator.com/from?site=beej.us
https://zguide.zeromq.org/
That's how you do it!
We don't have 64 Kilobytes of limited memory on our email servers anymore.
You should understand when they say "delete" that they mean "remove from their queue", whether it still exists in an "archive" folder, backups, trash, ... is irrelevant.
OK but providing value isn't the same as having power. It's almost the opposite thing.
Providing value is absolutely a way of having power. You change the world, that's power.
Having power isn't the same as yielding benefit (for yourself).
Beej's teaching power only answers "Yes" to the first question here - and that's only if your goals have something to do with teaching or improving the world overall, somewhere, for some people.
Anyway, in an age of information abundance and videos, educators are overvalued if they just give instruction alone. It's the whole package, the checks and balances on the student, that possibly provides some sort of value. Then again, there are irregularities, like the programmer who's never been to college who contributes as much or even more.
Beej's Guide to Network Programming (1994-2020) - https://news.ycombinator.com/item?id=26100075 - Feb 2021 (165 comments)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=13983212 - March 2017 (44 comments)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=13670971 - Feb 2017 (6 comments)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=12402313 - Aug 2016 (4 comments)
Beej's Guide to Network Programming (2012) - https://news.ycombinator.com/item?id=9445692 - April 2015 (31 comments)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=7584974 - April 2014 (1 comment)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=5241220 - Feb 2013 (47 comments)
Beej's Guide to Network Programming - https://news.ycombinator.com/item?id=584557 - April 2009 (22 comments)
Guide to Network Programming - https://news.ycombinator.com/item?id=337371 - Oct 2008 (7 comments)
I will always bring up “but you know who the real beej is, right?” if the other person is a nerd.
So, to me it was beej. But in reality, to the public [you can’t nickname yourself].
So it’s always been b33j0r. My friend who had the first dedicated server I encountered named me that because my name is Brian Jor—-
Finding me has been trivial since I first AWP’d in a CAL-O match. This was before scoutznknives were more than merit badges.
Deleted Comment
I miss WON. Man my friends and I hated Steam when it came out...
https://beej.us/guide/bgnet/html/
1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or
2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it, or for intellectual stimulation.
(Note that in category 2 I count things like "not using protobuf/thrift/etc. for IPC.)
When are these things not true? Genuinely curious! I'm guessing interfacing with existing obscure third parties, or using obscure language environments, but what are examples of that? And what other usages are there?
E.g.: Learn the basics Transport Layer protocols (TCP/UDP) if you work with HTTP
2. Speaking unusual protocols for which (good) libraries don’t exist.
3. I think interacting with the sockets api can also be relevant for tweaking various options to get better performance (though often there are Linux defaults that can be tweaked instead)
i also wrote a similar event-driven nonblocking http client in c for benchmarking and load-testing both that server and a different compatible one being written by another team in c++. this was nonblocking because it was important for the client to be faster than the server. at the time there was no libevent/libev/libuv so i wrote my own
at satellogic i maintained the upper layers of our cubesat space protocol stack, which provided a socket-like interface to our coap stack, which i also maintained part of
i also had to tweak kernel socket buffering settings for rabbitmq there
for wercam i'm experimenting with different approaches to ipc for efficiency; the data being transferred are pixel buffers so i don't think protobuf/thrift/etc. is going to help
for the mail server i share with some friends i wrote a milter in python to smtp-reject outside email to certain recipients unless it matched a whitelist; milter doesn't speak pb either and i couldn't find a python library
i wrote a multicast file transfer program in python (bccpo/bccpi) for copying files across the lan without having to fiddle around with ip addresses; the data being transferred is just (multicast) the ip and port and (unicast) the filename, file size, and file contents, so again i'm not sure pb would simplify anything. the hard part of getting it working was just multicast
i wrote a chat client for icb in python with asyncore, and again i couldn't find a suitable library
even when i'm using a library, it's often pretty useful to be able to read strace output or use select or poll with the file descriptor it's trying to hide from me
i don't think any of these qualify as either your category 1 or 2
i've also done a huge number of things that fall into category 2, of course, like writing an irc client in bash, writing a web server in assembly, and writing a chat server in three 80-column lines of c, writing a mud, and so on
How would you implement, say, netcat? Launch a jango rest service with kubernetes, which will run in a wm if it's a wrong OS, create a certificate with letsencrypt and access it with a react electron app?
E.g. interfacing a research machine with an OEM machine or a machine to a standalone embedded target, sometimes it makes sense to just serialize, send over a socket, ethernet crossover cable and be done.