I've used Usenet quite extensively in its heyday. At one point I even ran a Usenet site with dozens of peers and thousands of users. All on a very beefy (for its time) Sun E450.
In my opinion Usenets greatest strength was also its downfall. It is a distributed system without central authority over who could connect to the network or firm control over groups and their contents. To connect to it as a peer, you just had to find at least one Usenet site willing to exchange messages (peer) with you. Usually that wasn't a problem. ISPs, universities, organizations of all kinds were running their own servers, offering Usenet client access to their customers and members for no additional charge.
In a distributed system without central authority, innovation is exceedingly hard though. The protocols and features were basically set in stone. Taking the concept of offering discussions among users to a centralized or closed system made it possible to innovate very fast and offer superior features to the users.
As the article pointed out, that wasn't the only protocol which got picked up that way and innovated upon until it displaced its origins.
Since Usenet is still frozen feature wise were it was 30 or more years ago, I don't see a revival coming anytime soon.
This is the key problem with decentralised systems. They simply can't innovate as quickly as centralised systems.
We saw this with usenet and we also saw it with IRC. IRC really is what slack was, 30years ahead of it. However, IRC has so many essential missing features it never caught on - push notifications, saving your state when offline, admin controls, etc. I know some of these were solved with extensions and workarounds (having a shell connect to irc an then you connect to the shell for persistence for example) but it's a huge hack. This compares to Slack or Teams where they can push a back and front end update to millions/billions(?) of users in a very short space of time to add additional functionality.
This is another reason why all the decentralised 'blockchain' things would have struggled (on top of many others).
I think this argument is seductive but wrong because it ignores an invisible elephant in the room: funding.
Decentralization is a business problem, not a technical problem. Engineers tend not to see this because we're engineers and so we see technical problems first.
Usenet had no economic model. All the problems you list are solvable if there were funding available to solve them.
Free volunteer developer work tends to stop at the level of polish with which developers are comfortable, which is usually command line interaction and fairly manual processes. Developers generally have to be paid to develop new features and polish those features for the general audience, which is why there are precious few open source systems used by anyone other than developers.
Those that do exist tend to be subsidized by huge companies for the purpose of "commoditizing your compliments" or as tools to herd users into an ecosystem that has upsell opportunities built into it. Examples: Chrome, any open source client for a SaaS service, etc.
Non-profits can fund to some extent, but the truth is that polished feature-rich easy to use software is extraordinarily expensive to produce. A system that a developer can create in their spare time might cost millions to render usable to non-developers. Computers are actually very hard to use. We just don't see this because we're accustomed to it. Making them easy to use is a gigantic undertaking and is often far more difficult and complex than making something work at the algorithmic level.
Centralized systems with built-in economic models like SaaS or commercial software tend to triumph because they can fund the polish necessary to reach a wider audience. A wider audience means exponentially larger network effects. See Metcalfe's Law.
Cryptocurrency could have offered an alternative model but failed for entirely different reasons: perverse incentives that attract scammers. In crypto by far the most profitable thing to do is build a fake project that can appear just credible enough to attract retail buyers onto whom you can dump your tokens. There is no structural incentive to stick with a project and really develop it because all the money is made up front through the initial offering. This also ruins the ecosystem because "the bad chases away the good." Scammers make legitimate people not want to go anywhere near crypto, transforming the whole ecosystem into a "bad neighborhood."
> without central authority over who could connect to the network or firm control over groups and their contents
Moderation. In public forums, everything else is trivial and moderation is the invisible stinking hairy elephant in the room. To the question "why isn't there a decentralized open protocol free software X" the answer is moderation. Think or it as the discreet but efficient bouncers without which a sufficiently popular public place cannot maintain a welcoming atmosphere. Newsgroups became sufficiently popular, death by inadequate moderation ensued.
"Moderation" is a terrible excuse used by the corporate types as a reason to stick in their safe walled gardens.
We had moderation on Usenet. It was simple, but damn, did it work. We had plonka lists (client selected ignored users), keyword scoring, server level scoring, some anti-spam. There was EARLY work in Bayesian filters built in to clients, but was too late for the "big event". (When ISP's everywhere killed Usenet servers, for what is pretty surely assumed as anti-piracy.)
But lets look corporate moderation. It's done in secret, with vague rules that may or may not be stated, and by content moderation farms that use and abuse people. https://www.wired.co.uk/article/facebook-content-moderators-... . Or worse yet, you fool people in faux-ownership (reddit) and get them to do your moderation for you. Well, that is unless you do a bad job and get your subreddit banned for "low/no moderation".
I'd much rather see everything, and craft my own blocklists as I see fit. You know, just like Mastodon. Sure, there's some fediblocks at admin-level over pervasive content from a single server. But aside admin tasks, its pretty damn sweet. And it's loads better than the shit that happens over on Twitter, Reddit, or Facebook.
There were plenty of well moderated newsgroups actually.
A moderated newsgroup had all messages going through the moderator's system, with their programs deciding whether or not the message should be forwarded to the rest of the newsgroup. This doesn't necessarily mean that a human read all messages either.
Some of the oldest flamewars I've participated in are about moderated vs unmoderated USENET newsgroups. It amuses me that we're still debating the issue today.
The old chestnut "The Net interprets censorship as damage and routes around it" referred specifically to USENET if I recall correctly; it is often mis-applied to the Internet itself or to other services. But it referred to the notion that since USENET servers just routed their stash of stories to any node that requested them, censorship was functionally a non-starter; if you (the node owner) wanted content that some other node owner was squelching, you could request it from another node.
But it turns out that model had some major flaws. Nowadays, one could say the 'net interprets lack of censorship as offensive and migrates away from it. If there's one thing the experiments after USENET (from Slashdot to Mastodon) suggest, it's that people on average want some control (and don't on average want the burden of being their own moderator for every damn bitstream that comes down the pipe).
Usenet was - by the standards of the early 00's, which is when it began to decline - slow, and asynchronous. Posts could take hours to days to propagate, which meant conversation threads would go off in multiple directions at once.
Technology that was transformative during the BBS-to-early-internet transition became a liability ten years on.
Slashdot, PHPBB and later on Reddit offered synchronous, low-latency, single-point-of-truth updates to thread/post order.
The promise of self-configured "agents" for content filtering never really caught on - they were too time-consuming for anyone bar a few geeks and academics to manage. The social media "feed" is an evolution of the same idea, but there's not really any self-configuration - instead the system is programmed by the platform's owners to adapt to your behavioural feedback.
I disagree: between 1994-5 and about 2005 I made heavy use of a "private" NNTP server (for the Edge Magazine forums).
Compared to early web forums it was super fast - you could sync all of the messages in about two minutes. Then you could disconnect your modem, read + reply etc. Then go online an hour later to send reply + update the feed.
Broadband changed the game: once the internet was fast and always-on then web forums pretty much took over.
We lost something though: the beauty of NNTP was that you could easily keep your own archive of discussions of the groups you were interested in. Search worked and it as offline. Oh and no adverts or tracking!
I'm talking about server to server message propagation, not client/server - agreed, the Usenet clients were much faster and had better feature-sets than any web forum running on Netscape or the early versions of IE. That part was great, but messages could take hours to even days to propagate across the whole network. The sync-reply-upload work flow was indeed useful - especially so in countries where you had to pay by the minute to be online.
> Then go online an hour later to send reply + update the feed.
Usenet was truly special and an awesome design. Every iteration of something newer has been strictly worse. Mailing lists, forums, and now proprietary platforms controlled by a single corporation. At every step, features and functionality have been lost.
In the early 90s I used to travel coast to coast a lot, so I pulled all the newsgroups I was following to my laptop and spent the flights reading and responding to posts. Then I'd sync at the other coast before flying back.
What really killed it was alt.binaries.*. Vastly bloated bandwidth/storage requirements, so ISPs either killed NNTP altogther, or stopped giving a crap and you'd start, like, randomly dropping 10% of messages, which amounts to the same thing.
I don't think this was true. It was entirely common practice for an ISP to ignore alt.binaries but carry all the rest of Usenet fairly reliably. To the point where users would rebuke each other for posting binary content in a non alt.binaries hierarchy, lest the ISPs detect that as a stealth binary group and drop that too.
What killed Usenet was as the parent said - users are better served by the centralized management and reliability of a web forum, they don't need or care about decentralization and peering. The same thing happened for Discord and Slack replacing IRC.
> Usenet was - by the standards of the early 00's, which is when it began to decline - slow, and asynchronous. Posts could take hours to days to propagate, which meant conversation threads would go off in multiple directions at once.
Maybe that was your particular experience but generally that wasn't the case unless we're talking about a server connected via UUCP to the rest of the world, possibly with a dialup link between the servers and transferring news articles every few hours or once a day.
NNTP employs a flooding algorithm whereby every server offers all messages received to every peer except the ones showing up in the Path Header of the message. Peers were first offered the unique Message-Id each message has and could then decline or accept the transfer. Programs like innfeed implemented this conversation as a bidirectional streaming protocol (RFC4644). We're talking seconds for a text posting to be propagated to every half decently connected news server on the planet.
In addition to been slow, I would also add another factor: lack of notification method, which exasperated the problem of it's "slow"ness.
Another thing is, in effect, Usenet is a tried-to-be globally governed forum service runs on servers that's been configured differently. Some server don't allow large attachments, some don't allow HTML message, some moderator likes to delete while others don't...
The protocol don't really help it too. NNTP for example, is neither a high efficient nor resilient protocol. If your mail got too large, or your link is unstable, you'll have some nightmares trying to send it out. NNTP is also not a well-designed protocol, which makes client-end life-improving optimization very hard, as a result of it, some NNTP clients (Thunderbird and Outlook) will just let the user handle every error encountered, which is very annoying for the user.
So for me, Usenet is just simply outdated. That's all there is to it.
The way usenet clients worked was when new message headers were downloaded, the client would highlight unread messages. It's pretty similar to email where when you receive new messages, you can tell that they haven't been read before you read them. Websites like Hacker News, Reddit, Facebook, etc lack this feature.
Facebook, for instance, does have a notification system, but makes it very difficult to find the specific reply to your post or comment because it frequently filters out comments from being displayed unless "all comments" is selected from the drop down. Once that selection is made, you have to expand every thread to find the reply to your comment.
> Usenet is a tried-to-be globally governed forum service
Globally governed by whom? The only real governance was the creation of new groups in the Big 8 hierarchy. And, unless the group was moderated, anyone could post or reply to a comment.
> Some server don't allow large attachments, some don't allow HTML message
Text groups had conventions disallowing HTML or attachments, so very few people used them (unless their client wasn't configured properly).
> The protocol don't really help it too. NNTP for example, is neither a high efficient nor resilient protocol. If your mail got too large, or your link is unstable, you'll have some nightmares trying to send it out.
Message size isn't an issue for plain text messages and all major usenet services have no issues handling articles up to 750 Kb in size (for binary groups).
> NNTP is also not a well-designed protocol
What specifically is not well designed?
> NNTP clients (Thunderbird and Outlook)
Outlook never supported NNTP
> will just let the user handle every error encountered
What errors are you specifically referring to?
> So for me, Usenet is just simply outdated. That's all there is to it.
Based on what you've posted, it doesn't seem like you have much if any first hand experience using a NNTP client or have read any of the related RFC documents documenting the NNTP application level protocol.
Quite a few people posted under pseudonyms with fake email addresses on usenet.
> notifications
When you downloaded new headers from the server, the client would highlight the new messages in a way that made it very easy to see which messages you haven't seen and whether you got any replies to your message. Try doing that in Hacker News, reddit, Facebook, Twitter, etc.
Forum posts also had embedded images, and didn't require installing/configuring/maintaining a separate application.. all you needed was your web browser.
"I should briefly mention Google buying the Deja News archive, promising to revitalise Usenet, and then promptly abandoning it. Cheers Google. Choogle."
Oh wow, I totally forgot about that. I remember it being a really big deal at the time, and now I barely even remember.
Here, Google pulled one from the Microsoft playbook: embrace, extend, extinguish.
Google bought DejaNews, rebranded it into Google Groups (embrace) and then proceed to Googlify it, making it less and less Usenet-y and more and more Google specific (extend), until only the Google-specific mailing lists remained (extinguish).
Oh wow, that still exists! What a sparse and abysmal end.
I wonder if they even know it's still there? Probably doesn't use enough bandwidth for anyone to notice. Probably just a few people taking a trip down memory lane to re-read things posted 10-20 years ago.
Seems like nobody has mentioned a huge reason Usenet failed:
It needed a client program.
Digg/slashdot didn't succeed because they were flashier or moderated or had less illegal content or privately developed or any of these other reasons given, it was because people could just follow a link to a web page and they're in. Even today reddit or discord do everything they can to convert web users to app users for obvious reasons, but they must have that web version for casual users or they'll die off to whoever does have one.
So ultimately what killed Usenet was Section 230. Without that services could do simple copying/caching like NNTP (protected under telecom rules) but the actual presentation had to be done on the user's computer. And consequently were 230 repealed it wouldn't be the end of the social media world, but rather the resurgence of Usenet-like models where all the presentation is done by a client program.
> it was because people could just follow a link to a web page and they're in
Discord has entered the chat. (wait a minute...)
Seriously though, Discord is precisely an example of this. I was talking to someone about how Skype was degraded, and I wanted to build a competitor, then he showed me Discord. I've been using Discord since (and on my free time thinking of what I would do different to Guilded / Discord). Slack was cute, but Discord was much better. There's lots of room for improvement with Discord, but its just good enough.
This is one of the big reasons, I think. I hear about Usenet all the time, but whenever I try to go looking for it in the past to figure it out, it just makes no sense to me and I cannot figure out how to get to it. To me it's just this nebulous thing that exists somewhere, but I have no idea where.
> Seems like nobody has mentioned a huge reason Usenet failed:
> It needed a client program.
Many services required client programs to use. Email did and still does unless you use webmail. And there are services that allow you to read and post to usenet through one's browser.
Section 230 had nothing to do with it. Offering usenet retention is the same thing as being a common carrier. Section 230 would only come in to play if you actively moderated your usenet retention to prohibit perfectly legal speech.
Usenet served its purpose at a point in time and then faded into obscurity, though it keeps chugging away in a niche, appealing to some.
It's like saying that IRC failed.
What forum community, social network or media platform has been permanent and outlasted everything else? It seems that on every social network, people migrate, people abandon platforms, and losing critical mass can be fatal to the platform, and then it just passes on to the next one, having served its purpose; did MySpace "fail" or did it simply cede the crown to Facebook?
I remember Usenet fondly, though perhaps through rose-colored glasses. It was already full of trolls, flamewars, disruption and discord. Yet I cut my teeth there as a young sysadmin. I listened at the feet of experts, literal prominent experts in the field, who had spare time to just freely share their knowledge, because the Internet was an R&E community! I learned a lot about reputation and trust in the digital age. I gained "Internet street smarts" there.
I believe that it shaped a lot of people's lives, for better or worse, and that's not a failure by any measure.
Yep. Protocol-backed stuff on the Internet "fails" the way civilizations "go into decline" in the board game Small World; they aren't removed from the board, but they no longer serve any purpose other than taking up space that the active civilizations haven't taken over yet.
Exactly, it's protocol-backed, and that's the beauty of free, open source software and standards. As long as anyone care to maintain the client and/or servers, or even if very old code still works, you have a viable service. It's not a monolith that's beholden to the whims and fortunes of a single corporation that can just shut it down. Consider the enormous outcry when a single company has to shut down an entire platform, like a popular MMO or something, and then compare that to what happens when an IRC network has a schism, like the Freenode controversy: yep there was drama, but everyone took their toys and migrated to an identical network and now we have two.
I come from MUDding communities myself, and there is no shortage of old and crusty wizards who cling to their MUD servers and have kept them open, going on 30 years now! Clients come and go, Internet standards change, and social network fads happen, and we're still playing a game that doesn't bother supporting TLS, MFA, Unicode, or graphics. MUDs never failed, they simply achieved posterity and faded from sight.
I've said this a bunch of times before, but I don't see this take in a lot of Usenet postmortems, and I think it's truer than most of what people say about Usenet.
Piracy killed Usenet.
I ran a Freenix-competitive NNTP installation for a very popular ISP in Chicago in the 1990s. Usenet was by far the most demanding infrastructure we ran: we ran striped RAID arrays, read replicas†, transit servers --- all of this on the only expensive Sun hardware we had in our fleet. At one point, we independently reinvented the INN history cache, and there were a couple sites near the top of the Freenix leaderboard in part due to IRC conversations about our dumb patch. All this is just to say I feel like I have some bona fides here.
Keeping up with the Usenet everybody laments today was not this demanding. You could easily have done it with a Pentium FreeBSD server and a fast disk. But customers wouldn't let you. The open secret about Usenet was that it was a binary distribution system first and a discussion forum second.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare. Usenet is the stupidest mechanism for distributing binaries you can possibly imagine: just imagine BitTorrent, but entirely Base64'd, and without the forward error correction --- your file was in 100 pieces, and if you lost one of them, you were just fucked. If users couldn't reliably download pirated games or porn from your Usenet servers, they'd loudly complain and tank your reviews for running a bad Usenet setup. And woe betide any ISP that claimed to do NNTP on anything less than a binary full feed.
The rest of the story is obvious. ISP operators had better things to do with their time and money than cater to the small minority of angry cost-center users who used Usenet this way, and so they outsourced Usenet altogether. Usenet centralized because binary groups forced it to, and once it was centralized there was no reason not to just stick it on a website.
Web forums are better than Usenet was back in the day, so arguably, over the long term, not much was really lost. Reddit today is very much like Usenet was like in the mid-1990s, and they at least have a reliable archive. Apart from the archives of my old comp.security.unix posts, all I really miss about Usenet was the competitive systems engineering aspect of it. I'd love to go back in time and take another whack at Usenet knowing some of what I know now; I didn't realize how fun a problem it was at the time: a real system people actually cared about with a leaderboard.
† These things sound very boring now but were most definitely not boring in 1995.
Some providers (namely ISPs) simply dropped the binary groups and kept the text ones. So piracy alone is clearly not what killed it. In fact, Usenet really only exists today for piracy, so if anything, piracy actually saved Usenet. Just for a different use case than originally intended.
I mentioned this in my comment. As a technical matter, you could simply not take binaries. But as a commercial matter, you could not; if you didn't have binary groups, you weren't offering real Usenet (and if your binary feeds weren't ultra-reliable, you were offering shitty Usenet). The result was a consolidation of the platform down to a small number of providers willing to invest large amounts of resources to placate binaries users.
If lots of people aren't running NNTP servers, there's not much reason to use Usenet. By the time Reddit and Digg rolled in, it was no contest.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare.
The nice thing about NNTP was you didn't have to carry any group you didn't want to. We stopped carrying the binary hierarchy and some of the dedicated pr0n groups because of just this sort of administrative pain.
Some providers simply dropped the binary groups and kept the text ones. So piracy alone is clearly not what killed it. In fact, Usenet really only exists today for piracy, so if anything, piracy actually saved Usenet. Just for a different use case than originally intended.
> Thirdly was the user interface. Usenet looked dull. In a world of animated GIFs and MySpace colour schemes, Usenet didn't even have avatar images! Sure, the spartan nature meant that you could focus on a conversation - but it didn't feel as modern and exciting as the web did
"Modern and exciting" beat "being able to focus on a conversation" :(
Being able to focus is right at the top of my list of priorities. Surely I'm not the only one?
Eg, I don't see the HN UI ever be used for anything graphics related, because there's no image embedding.
This is fine for many subjects, but greatly constrains others.
It also has social effects. HN makes usernames very vague compared to say, Reddit or Slashdot. Which means that after more than 2 years on this site I still don't know any. The site is extremely impersonal to me, compared to Reddit where I can recognize regulars in some of the subreddits. Whether this is a good thing or not varies, but it certainly affects the dynamics.
> It also has social effects. HN makes usernames very vague compared to say, Reddit or Slashdot. Which means that after more than 2 years on this site I still don't know any. The site is extremely impersonal to me, compared to Reddit where I can recognize regulars in some of the subreddits.
I actually like this approach as it reduces the issue of some users dominating the conversation, thereby overshadowing others. I think it's essential to foster a level playing field for everyone when discussing topics of interest.
Claiming you could focus on the conversation with Usenet is putting on some rose tinted glasses.
By the late 90s Usenet was well into the Eternal September and inundated with spam. Unmoderated groups were a mess. There was also just the simple problem of formatting. IIRC Outlook Express defaulted to MIME/multipart posts which would cause all sorts of weirdness if someone with a text-only client quoted them and the OE user replied. Then of course there was the issue of top and bottom quoting. Many Usenet clients had truly awful UIs so a user could inadvertently quote a message's headers in a reply.
Compared to even a simple WWWBoard the experience was often lacking in terms of focus.
> By the late 90s Usenet was well into the Eternal September and inundated with spam. Unmoderated groups were a mess. There was also just the simple problem of formatting. IIRC Outlook Express defaulted to MIME/multipart posts which would cause all sorts of weirdness if someone with a text-only client quoted them and the OE user replied. Then of course there was the issue of top and bottom quoting. Many Usenet clients had truly awful UIs so a user could inadvertently quote a message's headers in a reply.
I didn't really start lurking and posting on usenet until the late '90s and while those problems occasionally cropped up, they really didn't interfere with the main discussion threads in the groups I was a regular in. I only stopped regularly using it by the early/mid 2010s because all the other regulars in the groups I was in ceased posting.
I mean, I used `tin` as my newsreader until somewhere between 2010-2015. It wasn't quite a hellscape of unusability if one was using a text only client.
You're definitely not the only one. I also endorse the idea of function over form, because catering to the masses has historically stood adjacent to 'lowering the bar' (choose any metric for this hypothetical).
Every place that doesn't cater to the masses seems to always be a higher quality community.
Slightly off topic, but dang, if you're reading this, I recommend making HN invite only for a month or two after the incoming reddit fiasco in July 1st. A ton of people on the major subreddits are already parroting things like "Just go to HN, it's basically the new reddit".
I don't think I'm making a hot take when I say reddit has a pretty awful community outside of the tech focused subs. But maybe I'm too concerned - I'm just worried that one of the last comfy parts of the internet will be invaded by plebs and emoji spammers.
There are plenty of good subreddits outside tech. Unfortunately, telling anyone about them risks ruining them due to how easy it is for a subreddit to blow up.
I would have gone with discoverability/UX. Finding and figuring out how to use a Usenet group was a complete pain. Once you’d managed the learning curve and found a group that interested you, it was great. But you can’t beat the discoverability of a Google indexed web forum.
How can I focus on a threaded conversation if I can't visually see the difference between the participants? Quickly looking at an avatar is easier than reading an email address.
Having even basic HTML formatting helps lay out a post - which makes it easier to concentrate on it. But that was something eschewed by most Usenet clients and posters.
Similarly, inline images are great for illustrating a point. I don't have to lose focus by clicking on a link and going to another tab to see what's being discussed.
> How can I focus on a threaded conversation if I can't visually see the difference between the participants? Quickly looking at an avatar is easier than reading an email address.
You post this on a threaded discussion forum without avatars, one you've frequented for a decade
> How can I focus on a threaded conversation if I can't visually see the difference between the participants? Quickly looking at an avatar is easier than reading an email address.
The from column in the thread pane view in my client[1] makes it easy to see the difference between participants, and also gives an indexed view of the entire thread. You don't get that with Hacker News or Reddit.
Readit News
In my opinion Usenets greatest strength was also its downfall. It is a distributed system without central authority over who could connect to the network or firm control over groups and their contents. To connect to it as a peer, you just had to find at least one Usenet site willing to exchange messages (peer) with you. Usually that wasn't a problem. ISPs, universities, organizations of all kinds were running their own servers, offering Usenet client access to their customers and members for no additional charge.
In a distributed system without central authority, innovation is exceedingly hard though. The protocols and features were basically set in stone. Taking the concept of offering discussions among users to a centralized or closed system made it possible to innovate very fast and offer superior features to the users.
As the article pointed out, that wasn't the only protocol which got picked up that way and innovated upon until it displaced its origins.
Since Usenet is still frozen feature wise were it was 30 or more years ago, I don't see a revival coming anytime soon.
We saw this with usenet and we also saw it with IRC. IRC really is what slack was, 30years ahead of it. However, IRC has so many essential missing features it never caught on - push notifications, saving your state when offline, admin controls, etc. I know some of these were solved with extensions and workarounds (having a shell connect to irc an then you connect to the shell for persistence for example) but it's a huge hack. This compares to Slack or Teams where they can push a back and front end update to millions/billions(?) of users in a very short space of time to add additional functionality.
This is another reason why all the decentralised 'blockchain' things would have struggled (on top of many others).
Decentralization is a business problem, not a technical problem. Engineers tend not to see this because we're engineers and so we see technical problems first.
Usenet had no economic model. All the problems you list are solvable if there were funding available to solve them.
Free volunteer developer work tends to stop at the level of polish with which developers are comfortable, which is usually command line interaction and fairly manual processes. Developers generally have to be paid to develop new features and polish those features for the general audience, which is why there are precious few open source systems used by anyone other than developers.
Those that do exist tend to be subsidized by huge companies for the purpose of "commoditizing your compliments" or as tools to herd users into an ecosystem that has upsell opportunities built into it. Examples: Chrome, any open source client for a SaaS service, etc.
Non-profits can fund to some extent, but the truth is that polished feature-rich easy to use software is extraordinarily expensive to produce. A system that a developer can create in their spare time might cost millions to render usable to non-developers. Computers are actually very hard to use. We just don't see this because we're accustomed to it. Making them easy to use is a gigantic undertaking and is often far more difficult and complex than making something work at the algorithmic level.
Centralized systems with built-in economic models like SaaS or commercial software tend to triumph because they can fund the polish necessary to reach a wider audience. A wider audience means exponentially larger network effects. See Metcalfe's Law.
Cryptocurrency could have offered an alternative model but failed for entirely different reasons: perverse incentives that attract scammers. In crypto by far the most profitable thing to do is build a fake project that can appear just credible enough to attract retail buyers onto whom you can dump your tokens. There is no structural incentive to stick with a project and really develop it because all the money is made up front through the initial offering. This also ruins the ecosystem because "the bad chases away the good." Scammers make legitimate people not want to go anywhere near crypto, transforming the whole ecosystem into a "bad neighborhood."
IRC never caught on? IRC was extremely popular throughout the 90's and into the 2000's
Moderation. In public forums, everything else is trivial and moderation is the invisible stinking hairy elephant in the room. To the question "why isn't there a decentralized open protocol free software X" the answer is moderation. Think or it as the discreet but efficient bouncers without which a sufficiently popular public place cannot maintain a welcoming atmosphere. Newsgroups became sufficiently popular, death by inadequate moderation ensued.
We had moderation on Usenet. It was simple, but damn, did it work. We had plonka lists (client selected ignored users), keyword scoring, server level scoring, some anti-spam. There was EARLY work in Bayesian filters built in to clients, but was too late for the "big event". (When ISP's everywhere killed Usenet servers, for what is pretty surely assumed as anti-piracy.)
But lets look corporate moderation. It's done in secret, with vague rules that may or may not be stated, and by content moderation farms that use and abuse people. https://www.wired.co.uk/article/facebook-content-moderators-... . Or worse yet, you fool people in faux-ownership (reddit) and get them to do your moderation for you. Well, that is unless you do a bad job and get your subreddit banned for "low/no moderation".
I'd much rather see everything, and craft my own blocklists as I see fit. You know, just like Mastodon. Sure, there's some fediblocks at admin-level over pervasive content from a single server. But aside admin tasks, its pretty damn sweet. And it's loads better than the shit that happens over on Twitter, Reddit, or Facebook.
A moderated newsgroup had all messages going through the moderator's system, with their programs deciding whether or not the message should be forwarded to the rest of the newsgroup. This doesn't necessarily mean that a human read all messages either.
Some of the oldest flamewars I've participated in are about moderated vs unmoderated USENET newsgroups. It amuses me that we're still debating the issue today.
Dead Comment
But it turns out that model had some major flaws. Nowadays, one could say the 'net interprets lack of censorship as offensive and migrates away from it. If there's one thing the experiments after USENET (from Slashdot to Mastodon) suggest, it's that people on average want some control (and don't on average want the burden of being their own moderator for every damn bitstream that comes down the pipe).
At least, that's the goal.
https://media.ccc.de/v/36c3-11086-the_ecosystem_is_moving
Technology that was transformative during the BBS-to-early-internet transition became a liability ten years on.
Slashdot, PHPBB and later on Reddit offered synchronous, low-latency, single-point-of-truth updates to thread/post order.
The promise of self-configured "agents" for content filtering never really caught on - they were too time-consuming for anyone bar a few geeks and academics to manage. The social media "feed" is an evolution of the same idea, but there's not really any self-configuration - instead the system is programmed by the platform's owners to adapt to your behavioural feedback.
Compared to early web forums it was super fast - you could sync all of the messages in about two minutes. Then you could disconnect your modem, read + reply etc. Then go online an hour later to send reply + update the feed.
Broadband changed the game: once the internet was fast and always-on then web forums pretty much took over.
We lost something though: the beauty of NNTP was that you could easily keep your own archive of discussions of the groups you were interested in. Search worked and it as offline. Oh and no adverts or tracking!
Usenet was truly special and an awesome design. Every iteration of something newer has been strictly worse. Mailing lists, forums, and now proprietary platforms controlled by a single corporation. At every step, features and functionality have been lost.
In the early 90s I used to travel coast to coast a lot, so I pulled all the newsgroups I was following to my laptop and spent the flights reading and responding to posts. Then I'd sync at the other coast before flying back.
What killed Usenet was as the parent said - users are better served by the centralized management and reliability of a web forum, they don't need or care about decentralization and peering. The same thing happened for Discord and Slack replacing IRC.
Maybe that was your particular experience but generally that wasn't the case unless we're talking about a server connected via UUCP to the rest of the world, possibly with a dialup link between the servers and transferring news articles every few hours or once a day.
NNTP employs a flooding algorithm whereby every server offers all messages received to every peer except the ones showing up in the Path Header of the message. Peers were first offered the unique Message-Id each message has and could then decline or accept the transfer. Programs like innfeed implemented this conversation as a bidirectional streaming protocol (RFC4644). We're talking seconds for a text posting to be propagated to every half decently connected news server on the planet.
Another thing is, in effect, Usenet is a tried-to-be globally governed forum service runs on servers that's been configured differently. Some server don't allow large attachments, some don't allow HTML message, some moderator likes to delete while others don't...
The protocol don't really help it too. NNTP for example, is neither a high efficient nor resilient protocol. If your mail got too large, or your link is unstable, you'll have some nightmares trying to send it out. NNTP is also not a well-designed protocol, which makes client-end life-improving optimization very hard, as a result of it, some NNTP clients (Thunderbird and Outlook) will just let the user handle every error encountered, which is very annoying for the user.
So for me, Usenet is just simply outdated. That's all there is to it.
The way usenet clients worked was when new message headers were downloaded, the client would highlight unread messages. It's pretty similar to email where when you receive new messages, you can tell that they haven't been read before you read them. Websites like Hacker News, Reddit, Facebook, etc lack this feature.
Facebook, for instance, does have a notification system, but makes it very difficult to find the specific reply to your post or comment because it frequently filters out comments from being displayed unless "all comments" is selected from the drop down. Once that selection is made, you have to expand every thread to find the reply to your comment.
> Usenet is a tried-to-be globally governed forum service
Globally governed by whom? The only real governance was the creation of new groups in the Big 8 hierarchy. And, unless the group was moderated, anyone could post or reply to a comment.
> Some server don't allow large attachments, some don't allow HTML message
Text groups had conventions disallowing HTML or attachments, so very few people used them (unless their client wasn't configured properly).
> The protocol don't really help it too. NNTP for example, is neither a high efficient nor resilient protocol. If your mail got too large, or your link is unstable, you'll have some nightmares trying to send it out.
Message size isn't an issue for plain text messages and all major usenet services have no issues handling articles up to 750 Kb in size (for binary groups).
> NNTP is also not a well-designed protocol
What specifically is not well designed?
> NNTP clients (Thunderbird and Outlook)
Outlook never supported NNTP
> will just let the user handle every error encountered
What errors are you specifically referring to?
> So for me, Usenet is just simply outdated. That's all there is to it.
Based on what you've posted, it doesn't seem like you have much if any first hand experience using a NNTP client or have read any of the related RFC documents documenting the NNTP application level protocol.
...and reputation/karma, avatars, post counts, moderation, search, categories, emojis, anonymity (email address not exposed), notifications...
Quite a few people posted under pseudonyms with fake email addresses on usenet.
> notifications
When you downloaded new headers from the server, the client would highlight the new messages in a way that made it very easy to see which messages you haven't seen and whether you got any replies to your message. Try doing that in Hacker News, reddit, Facebook, Twitter, etc.
People like the new shiny.
Forum posts also had embedded images, and didn't require installing/configuring/maintaining a separate application.. all you needed was your web browser.
Here, Google pulled one from the Microsoft playbook: embrace, extend, extinguish.
Google bought DejaNews, rebranded it into Google Groups (embrace) and then proceed to Googlify it, making it less and less Usenet-y and more and more Google specific (extend), until only the Google-specific mailing lists remained (extinguish).
Edit: see https://web.archive.org/web/20010226023947/http://groups.goo...
So Google came in and saved it.
https://en.m.wikipedia.org/wiki/Google_Groups#Deja_News
It's like reddit as the source for LLMs today.
So Google neglecting USENET could be about preventing competitors.
I wonder if they even know it's still there? Probably doesn't use enough bandwidth for anyone to notice. Probably just a few people taking a trip down memory lane to re-read things posted 10-20 years ago.
This appears to be inspired by the BBC show 'Look Around You'.
https://en.wikiquote.org/wiki/Look_Around_You
It needed a client program.
Digg/slashdot didn't succeed because they were flashier or moderated or had less illegal content or privately developed or any of these other reasons given, it was because people could just follow a link to a web page and they're in. Even today reddit or discord do everything they can to convert web users to app users for obvious reasons, but they must have that web version for casual users or they'll die off to whoever does have one.
So ultimately what killed Usenet was Section 230. Without that services could do simple copying/caching like NNTP (protected under telecom rules) but the actual presentation had to be done on the user's computer. And consequently were 230 repealed it wouldn't be the end of the social media world, but rather the resurgence of Usenet-like models where all the presentation is done by a client program.
Discord has entered the chat. (wait a minute...)
Seriously though, Discord is precisely an example of this. I was talking to someone about how Skype was degraded, and I wanted to build a competitor, then he showed me Discord. I've been using Discord since (and on my free time thinking of what I would do different to Guilded / Discord). Slack was cute, but Discord was much better. There's lots of room for improvement with Discord, but its just good enough.
https://www.eternal-september.org/
You can access Usenet via Thunderbird (or Pan, mentioned in the article).
Usenet can be accessed via the OSS Thunderbird client, or OSS Usenet clients.
https://www.fastusenet.org/thunderbird-windows-tutorial.html
It's not comprehensive, but will give you a taste of the flavor of what Usenet was like.
> It needed a client program.
Many services required client programs to use. Email did and still does unless you use webmail. And there are services that allow you to read and post to usenet through one's browser.
Deleted Comment
https://www.techdirt.com/2020/06/23/hello-youve-been-referre...
Usenet served its purpose at a point in time and then faded into obscurity, though it keeps chugging away in a niche, appealing to some.
It's like saying that IRC failed.
What forum community, social network or media platform has been permanent and outlasted everything else? It seems that on every social network, people migrate, people abandon platforms, and losing critical mass can be fatal to the platform, and then it just passes on to the next one, having served its purpose; did MySpace "fail" or did it simply cede the crown to Facebook?
I remember Usenet fondly, though perhaps through rose-colored glasses. It was already full of trolls, flamewars, disruption and discord. Yet I cut my teeth there as a young sysadmin. I listened at the feet of experts, literal prominent experts in the field, who had spare time to just freely share their knowledge, because the Internet was an R&E community! I learned a lot about reputation and trust in the digital age. I gained "Internet street smarts" there.
I believe that it shaped a lot of people's lives, for better or worse, and that's not a failure by any measure.
I come from MUDding communities myself, and there is no shortage of old and crusty wizards who cling to their MUD servers and have kept them open, going on 30 years now! Clients come and go, Internet standards change, and social network fads happen, and we're still playing a game that doesn't bother supporting TLS, MFA, Unicode, or graphics. MUDs never failed, they simply achieved posterity and faded from sight.
Piracy killed Usenet.
I ran a Freenix-competitive NNTP installation for a very popular ISP in Chicago in the 1990s. Usenet was by far the most demanding infrastructure we ran: we ran striped RAID arrays, read replicas†, transit servers --- all of this on the only expensive Sun hardware we had in our fleet. At one point, we independently reinvented the INN history cache, and there were a couple sites near the top of the Freenix leaderboard in part due to IRC conversations about our dumb patch. All this is just to say I feel like I have some bona fides here.
Keeping up with the Usenet everybody laments today was not this demanding. You could easily have done it with a Pentium FreeBSD server and a fast disk. But customers wouldn't let you. The open secret about Usenet was that it was a binary distribution system first and a discussion forum second.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare. Usenet is the stupidest mechanism for distributing binaries you can possibly imagine: just imagine BitTorrent, but entirely Base64'd, and without the forward error correction --- your file was in 100 pieces, and if you lost one of them, you were just fucked. If users couldn't reliably download pirated games or porn from your Usenet servers, they'd loudly complain and tank your reviews for running a bad Usenet setup. And woe betide any ISP that claimed to do NNTP on anything less than a binary full feed.
The rest of the story is obvious. ISP operators had better things to do with their time and money than cater to the small minority of angry cost-center users who used Usenet this way, and so they outsourced Usenet altogether. Usenet centralized because binary groups forced it to, and once it was centralized there was no reason not to just stick it on a website.
Web forums are better than Usenet was back in the day, so arguably, over the long term, not much was really lost. Reddit today is very much like Usenet was like in the mid-1990s, and they at least have a reliable archive. Apart from the archives of my old comp.security.unix posts, all I really miss about Usenet was the competitive systems engineering aspect of it. I'd love to go back in time and take another whack at Usenet knowing some of what I know now; I didn't realize how fun a problem it was at the time: a real system people actually cared about with a leaderboard.
† These things sound very boring now but were most definitely not boring in 1995.
That is, until Parchives were invented.
https://en.wikipedia.org/wiki/Parchive
If lots of people aren't running NNTP servers, there's not much reason to use Usenet. By the time Reddit and Digg rolled in, it was no contest.
The nice thing about NNTP was you didn't have to carry any group you didn't want to. We stopped carrying the binary hierarchy and some of the dedicated pr0n groups because of just this sort of administrative pain.
"Modern and exciting" beat "being able to focus on a conversation" :(
Being able to focus is right at the top of my list of priorities. Surely I'm not the only one?
Eg, I don't see the HN UI ever be used for anything graphics related, because there's no image embedding.
This is fine for many subjects, but greatly constrains others.
It also has social effects. HN makes usernames very vague compared to say, Reddit or Slashdot. Which means that after more than 2 years on this site I still don't know any. The site is extremely impersonal to me, compared to Reddit where I can recognize regulars in some of the subreddits. Whether this is a good thing or not varies, but it certainly affects the dynamics.
Also if you use old reddit usernames in reddit are as impersonal as they are here because of no discernible avatars.
I actually like this approach as it reduces the issue of some users dominating the conversation, thereby overshadowing others. I think it's essential to foster a level playing field for everyone when discussing topics of interest.
By the late 90s Usenet was well into the Eternal September and inundated with spam. Unmoderated groups were a mess. There was also just the simple problem of formatting. IIRC Outlook Express defaulted to MIME/multipart posts which would cause all sorts of weirdness if someone with a text-only client quoted them and the OE user replied. Then of course there was the issue of top and bottom quoting. Many Usenet clients had truly awful UIs so a user could inadvertently quote a message's headers in a reply.
Compared to even a simple WWWBoard the experience was often lacking in terms of focus.
I didn't really start lurking and posting on usenet until the late '90s and while those problems occasionally cropped up, they really didn't interfere with the main discussion threads in the groups I was a regular in. I only stopped regularly using it by the early/mid 2010s because all the other regulars in the groups I was in ceased posting.
Every place that doesn't cater to the masses seems to always be a higher quality community.
Slightly off topic, but dang, if you're reading this, I recommend making HN invite only for a month or two after the incoming reddit fiasco in July 1st. A ton of people on the major subreddits are already parroting things like "Just go to HN, it's basically the new reddit".
I don't think I'm making a hot take when I say reddit has a pretty awful community outside of the tech focused subs. But maybe I'm too concerned - I'm just worried that one of the last comfy parts of the internet will be invaded by plebs and emoji spammers.
Having even basic HTML formatting helps lay out a post - which makes it easier to concentrate on it. But that was something eschewed by most Usenet clients and posters.
Similarly, inline images are great for illustrating a point. I don't have to lose focus by clicking on a link and going to another tab to see what's being discussed.
You post this on a threaded discussion forum without avatars, one you've frequented for a decade
https://www.emacswiki.org/emacs/XfacesSupport
I remember them being used by some Usenet posters in the 1990s.
The from column in the thread pane view in my client[1] makes it easy to see the difference between participants, and also gives an indexed view of the entire thread. You don't get that with Hacker News or Reddit.
[1] https://i.imgur.com/i226q2c.png