SSB is such a wonderful idea. There’s no global timeline — just archipelagos. It assumes that network heterogeny is the default, and is transmission layer agnostic. Breakages will occur. Maybe you’re living on a catamaran in the South Pacific and you only have connectivity once a month — SSB will work even then.
Your own timeline is a sigchain — a sequenced list of signed messages. You replicate the content in your network (2 hops away). Bridges between communities can be built or burned. Many islands can exist without needing to erase the others from even existing — mutual separation is possible. Consensus is not necessary.
Is global network culture still possible? If it is, in the midst of the national internets we now live inside of, I suspect it will look something like this. A little different from what we were promised, but maybe a little better too.
I'm thinking the next constellation of LEO satellites will cover the planet with high-latency (e.g.: 90 minutes) and high-bandwidth connections. If someone can fulfill your request in your footprint, you'll have low-latency bandwidth for a few minutes. Otherwise, someone else in the shell will send it up later once the satellite is in your footprint and get it down to you later once you're back in the footprint.
All you need to do is send a request to someone else in your shell and they'll upload it the next time the satellite passes.
Probably some centralized or decentralized crypto payment to make it all happen. Still need cheaper phased-beam arrays though.
For a look at SSB in particular, I would strong recommend the protocol guide — it explains the core ideas quite well, which I think are broadly applicable to future networks of this pattern: https://ssbc.github.io/scuttlebutt-protocol-guide/
I absolutely love the simplicity of Scuttlebutt, but the APIs and (dev)user extensibility are anything but, imo. I hope for continued work there.
As an example, i wanted to write an application on top of SSB. I found it difficult to determine _how_ to achieve this though. Conceptually the appendlog is stupid simple, but SSB still has meaningful complexity in the secure handshake implementation. While they do have libs, it mostly seemed JavaScript (NodeJS) was the only meaningfully complete lib to use (where as i am on Rust). I also found it difficult to determine what the network would accept. If i write an app and start pushing blobs onto the network, will they reject it because they're not of the same type? What if my data is too big? Am i abusing the net?
I also had difficulty conceptualizing how to write app data in SSB. It felt like i needed to abandon SQL or common application interfaces for data. Having to reinvent wheels in the application code because SSB-data is mostly (i think?) a JSON blob store felt bad.
Shared identities also proved to be a bit of a blocker. Since the app i was writing wanted multiple devices to author data the notion of a device identity came up, and at the time that didn't seem well (or at all) supported. I didn't want to invent my own mechanism that would later seem invalid to the community.
For now i decided to hold off and implement my own SSB friendly data storage mechanism. Since i know the data layer of SSB is simple, and SSB folks have proven it can handle Git fine (they distribute their source in SSB iirc). I imagine my own data later will plug into SSB without issue.
These aren't critiques of SSB exactly, just highlighting some areas (for anyone interested) where SSB is currently weak in my mind. I love SSB. Even if i end up using an entirely different p2p tech stack than SSB, it has changed my way of thinking on this. What SSB does right is everything that i dislike about IPFS. SSB is small, simple tooling. It feels like Git. Not some huge complex network of IPFS nodes, but 2 people, 3 people. It can work over P2P, email, floppy disk. It can be more, but it also can be less. And that's something more projects could adopt.
I've wanted to write a longer blog post about this, but after a few years of working on SSB I appreciate you highlighting these pain points. I think everyone in the SSB maintainer community feels the same way.
My hot take: Scuttlebutt should be something that you can implement yourself in an afternoon. I've done this a few times, using Python, Node.js, and Deno, but I've only implemented the message type. (Not Multiserver, MuxRPC, Secret-Handshake, Box-Strean, Private-Box, or any of the other protocols that are associated with SSB.)
> It feels like Git.
You got it. You could express an SSB chain with Git and a pre-commit hook (to verify messages), but using Git as your database is even harder than learning the toy database that SSB uses.
I agree that the nodejs implementation makes contributing and using the libraries difficult. There are rust libraries being worked on and incorporated into the project as we speak, so if you feel like contributing, now might be a good time to look back into the project.
Alternatively, if you're interested in a replicated datatype that was written specifically for Rust, which is more compact than JSON, I'd suggest looking at RON:
Thanks for the analysis. Have you checked out Matrix? I'm looking for a p2p network layer and am most attracted to it so far but haven't used any of them yet
I've been debating Matrix too! Though, i've not looked into it. My hope was to make (as i mentioned) a Git-like layer for my data that was in the way i wanted (wheels reinvented hah), which would work over SSH/Email/etc, and then plug it into a mature P2P setup. Matrix is definitely on the list of P2P impls i want to look into.
Though so far i've never heard of anyone actually using Matrix for a generic P2P backend. I'm curious what you've found on Matrix for this use case?
One of the cool features of SSB is making actual P2P a first-class feature. It works just as well on an isolated private adhoc network as it does over the web.
I wish more dencentralized/federated services operated this way.
Matrix is making p2p a first-class feature, and how they’re doing it is pretty simple. They’re just embedding a “homeserver” right onto the user device alongside the client.
With the new fast homeserver efforts, like Dendrite and Conduit, this suddenly becomes feasible.
"Tending and pruning are not a stranger’s duty, it is through near moderation and free listening that we improve our surroundings" https://scuttlebutt.nz/docs/principles/
The idea is that moderation will be based on who you trust.
First and foremost, that is yourself, i.e. you can moderate your own timeline by choosing who to follow or block. Since this happens on your device, there's nobody who can interfere with this.
Further, it also means that you can trust your peers when you want. One default behaviour is that if you do NOT follow a feed A, and feeds B and C, who you do follow, each block A, your client will not by default replicate A's messages and thus you won't see them. Of course, if you unfollow B and C, or if you choose to explicitly follow A, then you'd see their content again. That's what I would understand by "close moderation" anyhow.
I believe it means handling moderation yourself rather than relying on platform providers to do it for you. You have to choose your peers that you receive content from carefully rather than rely on platform moderators to filter content.
About the only thing considered universally intolerable and subject to administrative removal, at least on USENET's alt.* hierarchy, was spam. But spammers could spam faster than cancellers could cancel, so it eventually overwhelmed the platform.
I can see the appeal. I imagine this means that if none of my friends follow conspiracy theory content, I won't see any of that, and if one of them did, I could presumably block the conspiracy theory content feeds while still following my friends.
The downside may be that, presumably, this puts everyone into a bubble even more than Facebook did? At least there is no algorithmic meddling or centralized content control.
Last time I looked it was half baked and not easy to use, has it changed in recent years or months and can it be considered practical in a way that let's say Matrix is?
I don't see how these apps dont eventually get taken down like the Fediverse apps earlier this year. Eventually (probably once it's polished enough to be usable) the deplatformed communities will discover SSB and then Google/Apple will take the apps down.
Manyverse’s work on room servers version 2 will be huge.
The rooms will function a lot more like real life buildings. Some will be private like homes, some will be public like libraries, others will be restricted like a club.
It also spun up my laptops fans for minutes when syncing, I don’t know if thats still an issue. Cool project though and there are some interesting people involved and contributing.
Yes, the initial syncing and indexing can take a moment. I would recommend joining a single Pub or SSB room [1] to find people and grow your network organically. The large Pubs [2] have a lot of content to download and also follow bots (which you would have to block).
Yes[0]. AFAIK fully transferring an identity from one device to the next should not be a problem. The problem is more that updating the feed from multiple devices would create diverging feeds.
It's kind of funny to think that decentralised apps, built by idealistic Silicon Valley hackers, now have a potential core audience among extreme MAGA types.
Scuttlebutt wasn't built by "Silicon Valley hackers", but people from New Zealand, Sweden etc.
Unrelated to who builds it, decentralized or self-hosted platforms have been the go-to place for lots of extremist groups, even before MAGA, so not sure what this comment is supposed to bring to light?
I think a lot of the dweb people first got interested in this area because they were on BBSes and so on sharing stuff like the anarchist's cookbook at risk of getting a visit by the FBI and used to think information wanted to be free.
Now they're adults, and have kept their interest in dweb technologies, but have come to see "dangerous speech" as a huge problem, and are waking up to the realization that their technical interests and goals would enable these kinds of things. And they're not happy about it, or at the very least are experiencing cognitive dissonance. But that was always the case, and for many, they were the unsavory people in their youth that the older folks were saying were participating in dangerous rhetoric and ought to be silenced or investigated, just like they are saying now about people who are saying things adjacent to things which are illegal.
At the very least we should seek consistency. If you support decentralized technology for speech, you should be happy with the net benefits and net harms. The alternative is the status quo. Arguably the spirit of the internet was not to have what we have now.
I agree that the BBS tradition of G-files plays into this, but those were very different times. The people downloading this sort of stuff were almost exclusively kids who got a thrill out of possessing forbidden knowledge, but aside from setting off a smoke bomb at an abandoned quarry or something, nothing ever came of it. The near mainstream culture of prepping and guns and conspiracy is far different than it was back then.
I still think that there is a positive place for DWeb technologies, but that place is specifically to enable more truly free speech. A lot of what got us into this mess is that speech on social media is not truly free. The only speech that is allowed/promoted on those platforms is that which appeals to base instincts, because it is that speech that drives revenue for the platforms. What we're allowed to say, and have heard, should not depend on what is most profitable for a few companies.
You will find that the people who were on the unsavoury bbs are the people who are against censorship today.
Out of the half dozen people I know from that period in my life (2000-2006) not one has had a kind thing to say about facebook, google or twitter since 2010. Those firms are the lame dinosaur stuck in a tar pit that we made fun of Microsoft for being in the 00s.
It's the people who called us nerds and made high school hell for us that somehow ended up in hr at tech co and are now setting policy there.
You're making a false equivalence.[1] The "dweb" people were indeed merely anarchic and irresponsible. The deeper, conspiracy-minded parts of the MAGA community are verifiably dangerous. They said for weeks that a storm is coming, that they were going to take washington, that congress had to be stopped, that the vice president should be executed, etc...
And then on Wednesday they made the attempt.
Comments like yours are predicated on this idea that speech itself can only ever be merely "irresponsible". But... the world isn't like that. If you spend years making "jokes" about insurrection, you eventually find someone attacking congress.
[1] Edit to answer the question below: you are falsely equating harmless anarchic geeks of the 80's and 90's with violent terrorists. It's not appropriate to manage these communities using the same tools.
In Germany the government used census data tabulated on IBM punchcard equipment to find who to round up. Later, during the liquidation phase of the effort each camp had punch card equipment to help with maintaining the inventory of slave labor skills. That's what the number on the arm was for - the primary key. Copies of punched cards were sent to the central administrative HQ of the camp organization for additional processing and were used to identify specific skilled people who needed to be moved to other camps. Authoritarians rely on the stupid for the grunt work but they're not stupid themselves and can exploit technology to achieve their goals.
The goal of decentralization isn't to give a voice to people inciting violence, however that is a consequence of such systems when you share technology with everyone - though we shouldn't fear this, we do need to actively and proactively counter it at a higher layer, educating people and helping them heal their closed heart and mind.
The goal of decentralization is to take power away from companies who govern poorly or in a way we don't agree with, so our data and network is mobile, so we can "vote" by leaving their platform - and not buy their products or services or give them our attention to monetize with ads. And then once this fluid system exists with a hierarchy of good, good hearted, well meaning governors - we must then focus on educating and healing the population - where we won't reach them easily in the digital world due to self-perpetuating filter bubbles, anyone who tries to reason with them are ignored or simply ban/blocked to reenforce their echo chamber.
Inciting violence is illegal in most countries, you can still prosecute people who participate in such actions with a decentralised system.
You might not be able to remove their speech but it would still have consequences (again in most countries).
This could even be worse for people who engage in such activities since you can't remove your words in hindsight! I've seen a lot of back-pedalling and removing of tweets from people.
You mention educating and healing twice. Could you elaborate on who "we" is, and also, how we can go from where we are to a place where education and healing is taking place? What are the steps we should take?
I think it's actually rather sad. I think there is a good chance that these in my opinion very important technologies will get a bad reputation because of this.
I know very little about Scuttlebutt, but supposing, say an ex-president decided to start using it. Would it be able to handle the level of traffic this might generate?
I realise it's decentralised, but would that, for example, at some point require every one who were to get on Scuttlebutt to download on 80gb file in order to get started?
his followers can barely figure out how to get on parler, many of them thought parler groups on facebook was parler itself. They will NEVER, ever, figure out scuttlebutt, DJT couldn't even figure out scuttlebutt.
People want to sign up and login easily. This doesn't seem to do any of that. Convenience is king and necessity it's history maker.
In some ways we’ve come full circle: SV was enabled by defence types, I bet a lot of the early work in advanced electronics and decentralized communications was carried out by “square” types who used “socialist” as an insult. There are still plenty of traditionalist right-wingers in STEM.
I really like the idea of Scuttlebutt. I think there are a couple of things it needs on top of what it already has;
Right now, your public key is who you are, which I think is a mistake. It's easy to imagine a world where we have a "web of trust" solution here, where you can scan someone's public key from a QR code straight off their phone, and then sign said key so other people who trust you know that the key is legit. Then you could use that key to sign future keys, allowing easy key rotation. It would also be easy to post a message signed with your private key that revokes a key and all messages it has authored/all keys it has signed.
I also think there needs to be multiple transports for posting content - if I want to send a message to my buddy Dan, I should be able to send it directly to him if he's online. If he isn't, though, I should be able to let other users whom we both know grab a copy to pass on to him later, or maybe have a "deaddrop" server or a "pub" where I can stash messages and have them get picked up by someone else later.
It needs a top-notch client, too. Something to compete with Facebook and Twitter.
But it's a very promising start, and it's the way this needs to go if we're ever going to escape the surveillance players in this industry - we need a social network that's truly distributed, based on open standards that anyone can implement.
>I also think there needs to be multiple transports for posting content - if I want to send a message to my buddy Dan, I should be able to send it directly to him if he's online. If he isn't, though, I should be able to let other users whom we both know grab a copy to pass on to him later, or maybe have a "deaddrop" server or a "pub" where I can stash messages and have them get picked up by someone else later.
Scuttlebutt is built around enabling this as a (the?) primary feature. It's not just "supported", the entire ecosystem does this by design. It even calls that kind of server "pubs" (they're normal peers, they just auto-follow anyone who asks).
Readit News
Your own timeline is a sigchain — a sequenced list of signed messages. You replicate the content in your network (2 hops away). Bridges between communities can be built or burned. Many islands can exist without needing to erase the others from even existing — mutual separation is possible. Consensus is not necessary.
Is global network culture still possible? If it is, in the midst of the national internets we now live inside of, I suspect it will look something like this. A little different from what we were promised, but maybe a little better too.
I'm thinking the next constellation of LEO satellites will cover the planet with high-latency (e.g.: 90 minutes) and high-bandwidth connections. If someone can fulfill your request in your footprint, you'll have low-latency bandwidth for a few minutes. Otherwise, someone else in the shell will send it up later once the satellite is in your footprint and get it down to you later once you're back in the footprint.
All you need to do is send a request to someone else in your shell and they'll upload it the next time the satellite passes.
Probably some centralized or decentralized crypto payment to make it all happen. Still need cheaper phased-beam arrays though.
https://news.ycombinator.com/item?id=22915460 (2020)
https://news.ycombinator.com/item?id=22909984 (2020)
https://news.ycombinator.com/item?id=20828356 (2019)
https://news.ycombinator.com/item?id=16877603 (2018)
https://news.ycombinator.com/item?id=16273096 (2018)
https://news.ycombinator.com/item?id=14050049 (2017)
And other smaller ones over the years...
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
As an example, i wanted to write an application on top of SSB. I found it difficult to determine _how_ to achieve this though. Conceptually the appendlog is stupid simple, but SSB still has meaningful complexity in the secure handshake implementation. While they do have libs, it mostly seemed JavaScript (NodeJS) was the only meaningfully complete lib to use (where as i am on Rust). I also found it difficult to determine what the network would accept. If i write an app and start pushing blobs onto the network, will they reject it because they're not of the same type? What if my data is too big? Am i abusing the net?
I also had difficulty conceptualizing how to write app data in SSB. It felt like i needed to abandon SQL or common application interfaces for data. Having to reinvent wheels in the application code because SSB-data is mostly (i think?) a JSON blob store felt bad.
Shared identities also proved to be a bit of a blocker. Since the app i was writing wanted multiple devices to author data the notion of a device identity came up, and at the time that didn't seem well (or at all) supported. I didn't want to invent my own mechanism that would later seem invalid to the community.
For now i decided to hold off and implement my own SSB friendly data storage mechanism. Since i know the data layer of SSB is simple, and SSB folks have proven it can handle Git fine (they distribute their source in SSB iirc). I imagine my own data later will plug into SSB without issue.
These aren't critiques of SSB exactly, just highlighting some areas (for anyone interested) where SSB is currently weak in my mind. I love SSB. Even if i end up using an entirely different p2p tech stack than SSB, it has changed my way of thinking on this. What SSB does right is everything that i dislike about IPFS. SSB is small, simple tooling. It feels like Git. Not some huge complex network of IPFS nodes, but 2 people, 3 people. It can work over P2P, email, floppy disk. It can be more, but it also can be less. And that's something more projects could adopt.
My hot take: Scuttlebutt should be something that you can implement yourself in an afternoon. I've done this a few times, using Python, Node.js, and Deno, but I've only implemented the message type. (Not Multiserver, MuxRPC, Secret-Handshake, Box-Strean, Private-Box, or any of the other protocols that are associated with SSB.)
> It feels like Git.
You got it. You could express an SSB chain with Git and a pre-commit hook (to verify messages), but using Git as your database is even harder than learning the toy database that SSB uses.
Various links:
- My SSB data expressed as a Git repository (outdated): https://github.com/christianbundy/ssb-git-data
- My speedrun SSB implementation with SQLite, using HTTP for replication: https://github.com/christianbundy/http-ssb
Alternatively, if you're interested in a replicated datatype that was written specifically for Rust, which is more compact than JSON, I'd suggest looking at RON:
http://replicated.cc/https://github.com/gritzko/ron
Though so far i've never heard of anyone actually using Matrix for a generic P2P backend. I'm curious what you've found on Matrix for this use case?
I wish more dencentralized/federated services operated this way.
With the new fast homeserver efforts, like Dendrite and Conduit, this suddenly becomes feasible.
What does "near moderation" mean?
First and foremost, that is yourself, i.e. you can moderate your own timeline by choosing who to follow or block. Since this happens on your device, there's nobody who can interfere with this.
Further, it also means that you can trust your peers when you want. One default behaviour is that if you do NOT follow a feed A, and feeds B and C, who you do follow, each block A, your client will not by default replicate A's messages and thus you won't see them. Of course, if you unfollow B and C, or if you choose to explicitly follow A, then you'd see their content again. That's what I would understand by "close moderation" anyhow.
About the only thing considered universally intolerable and subject to administrative removal, at least on USENET's alt.* hierarchy, was spam. But spammers could spam faster than cancellers could cancel, so it eventually overwhelmed the platform.
The downside may be that, presumably, this puts everyone into a bubble even more than Facebook did? At least there is no algorithmic meddling or centralized content control.
Oasis is a new frontend (alternative to the Patchwork desktop app): https://github.com/fraction/oasis/
Manyverse continues development, including working on low-level database improvements, and gossip "rooms": https://www.manyver.se/blog/2021-01-update
Planetary (iOS app) approaches public launch: https://planetary.social/https://viewer.scuttlebot.io/%25kXEXGmyNyo%2BuKkcySMwghVKpvR...
The rooms will function a lot more like real life buildings. Some will be private like homes, some will be public like libraries, others will be restricted like a club.
Really cool development!
[1] https://github.com/staltz/ssb-room/blob/master/FAQ.md [2] https://github.com/ssbc/ssb-server/wiki/Pub-Servers
[0]: https://github.com/ssbc/handbook.scuttlebutt.nz/blob/master/...
Unrelated to who builds it, decentralized or self-hosted platforms have been the go-to place for lots of extremist groups, even before MAGA, so not sure what this comment is supposed to bring to light?
Now they're adults, and have kept their interest in dweb technologies, but have come to see "dangerous speech" as a huge problem, and are waking up to the realization that their technical interests and goals would enable these kinds of things. And they're not happy about it, or at the very least are experiencing cognitive dissonance. But that was always the case, and for many, they were the unsavory people in their youth that the older folks were saying were participating in dangerous rhetoric and ought to be silenced or investigated, just like they are saying now about people who are saying things adjacent to things which are illegal.
At the very least we should seek consistency. If you support decentralized technology for speech, you should be happy with the net benefits and net harms. The alternative is the status quo. Arguably the spirit of the internet was not to have what we have now.
I still think that there is a positive place for DWeb technologies, but that place is specifically to enable more truly free speech. A lot of what got us into this mess is that speech on social media is not truly free. The only speech that is allowed/promoted on those platforms is that which appeals to base instincts, because it is that speech that drives revenue for the platforms. What we're allowed to say, and have heard, should not depend on what is most profitable for a few companies.
Out of the half dozen people I know from that period in my life (2000-2006) not one has had a kind thing to say about facebook, google or twitter since 2010. Those firms are the lame dinosaur stuck in a tar pit that we made fun of Microsoft for being in the 00s.
It's the people who called us nerds and made high school hell for us that somehow ended up in hr at tech co and are now setting policy there.
And then on Wednesday they made the attempt.
Comments like yours are predicated on this idea that speech itself can only ever be merely "irresponsible". But... the world isn't like that. If you spend years making "jokes" about insurrection, you eventually find someone attacking congress.
[1] Edit to answer the question below: you are falsely equating harmless anarchic geeks of the 80's and 90's with violent terrorists. It's not appropriate to manage these communities using the same tools.
[0] https://twitter.com/ArvindKrishna/status/1346932293991079936
The goal of decentralization is to take power away from companies who govern poorly or in a way we don't agree with, so our data and network is mobile, so we can "vote" by leaving their platform - and not buy their products or services or give them our attention to monetize with ads. And then once this fluid system exists with a hierarchy of good, good hearted, well meaning governors - we must then focus on educating and healing the population - where we won't reach them easily in the digital world due to self-perpetuating filter bubbles, anyone who tries to reason with them are ignored or simply ban/blocked to reenforce their echo chamber.
You might not be able to remove their speech but it would still have consequences (again in most countries).
This could even be worse for people who engage in such activities since you can't remove your words in hindsight! I've seen a lot of back-pedalling and removing of tweets from people.
> we must then focus on educating and healing the population
Do you not see how wrong this sounds? Who gets to create this “healing” education curriculum which will “open hearts and minds”?
Deleted Comment
I realise it's decentralised, but would that, for example, at some point require every one who were to get on Scuttlebutt to download on 80gb file in order to get started?
People want to sign up and login easily. This doesn't seem to do any of that. Convenience is king and necessity it's history maker.
Firstly, only the followers (and maybe their friends) would need to download 80GB of videos posted by that supposed ex-prez.
Then, it's peer-to-peer. I don't know the details, though. It could be torrent (very effective, scalable), or direct connections (poorly scalable).
Some pubs (servers) can retain your data, IIRC, though they could pick who they accept.
Syncing would be pretty easy on local networks and sneakernets, provided you have the storage capacity.
Deleted Comment
Right now, your public key is who you are, which I think is a mistake. It's easy to imagine a world where we have a "web of trust" solution here, where you can scan someone's public key from a QR code straight off their phone, and then sign said key so other people who trust you know that the key is legit. Then you could use that key to sign future keys, allowing easy key rotation. It would also be easy to post a message signed with your private key that revokes a key and all messages it has authored/all keys it has signed.
I also think there needs to be multiple transports for posting content - if I want to send a message to my buddy Dan, I should be able to send it directly to him if he's online. If he isn't, though, I should be able to let other users whom we both know grab a copy to pass on to him later, or maybe have a "deaddrop" server or a "pub" where I can stash messages and have them get picked up by someone else later.
It needs a top-notch client, too. Something to compete with Facebook and Twitter.
But it's a very promising start, and it's the way this needs to go if we're ever going to escape the surveillance players in this industry - we need a social network that's truly distributed, based on open standards that anyone can implement.
Scuttlebutt is built around enabling this as a (the?) primary feature. It's not just "supported", the entire ecosystem does this by design. It even calls that kind of server "pubs" (they're normal peers, they just auto-follow anyone who asks).