We tried our hardest back in the early 2000's to make this stick. I cofounded a startup building servers, clients, and SDK's. Maybe the world is finally ready for it now. The issue we ran into was customers/consumers just didn't care about federated systems. And various implementations had all sorts of quirks making interoperability between domains and clients difficult, at best. There are just SO MANY THINGS you have to do to make good messenger software that lining up all the pieces with open standards proved impossible in the ~8 years I was working on it. Again, maybe it's better now, but I'm skeptical, probably jaded. :)
It's not about the protocol. Most people don't care about XMPP, RSS, or any of those things.
They have things they want done, like chatting with a friend. They want a solution that delights them in doing that. What delights people has generally changed (like more people care about others not listening in these days).
Whose the user and what's their experience? This is where XMPP always failed me. The clients had a poor experience.
If someone wants that to change now... focus on a great experience for the people you want to be users.
1000% agree, so much so that I started such a project and wrote a blog post about this very problem[1] and launched what has become a full-time project to solve it (Snikket).
These days I believe very much in identifying and serving a specific set of users. XMPP is just a protocol, it has lots of potential applications, but it's nothing unless you channel that into a tangible usable product for real people.
This epiphany came to me after more than 10 years of working on XMPP as a server developer and spending time promoting open communication protocols to people. It was soul-crushing that at the end of a day I would still see my own family communicating with each other via WhatsApp.
Six months after the initial Snikket prototypes, and after a little user testing, I migrated my family over to XMPP by sending a simple invitation link (the start of the Snikket onboarding journey). They're now using it daily and totally happy with it. I'm beginning to hear similar stories from others who have repeated this with their own families/groups and their own Snikket setups.
All it took was to look at the issue through the eyes of "normal people" for a moment, and it changed my approach (and success rate) significantly.
I miss the "golden age" of chat where I could talk to anyone (google/fb/msn messenger/aol?) with my own jabber server. Pidgin was a great client at the time.
I've been looking for a modern self-hosted chat solution. I want it to be XMPP so badly (running XMPP with something like prosody is so convenient). The clients are really letting me down though. Desktop clients are old-fashioned and awful (I could get behind that, but I don't think my kids could). The mobile situation is even worse.
The new-fangled chat solutions (Matrix, Zulip, etc.) are interesting, but the operational requirements are insane (in the case of Matrix, I'm also very nervous about reliability).
I can't say I'm super hopeful, but I'm rooting for you, XMPP.
On Android I use Conversations[1] (from F-Droid), on Linux I use gajim[2]. For me it works, for others, I have no clue. Gajim is available on macOS and Windows, too. I dunno if by default it has the OMEMO[3], and PGP plugin though. I think they are a must-have. Make sure that "XEP-0384: OMEMO Encryption" is supported by the server you pick.
I think the writing was on the wall for XMPP once people started chatting on Facebook and Facebook got past their brief XMPP compatible phase. Then Google also deprecated it.
It's a nice protocol, but if you're missing all the conversations it doesn't help. When it was popular, it was the system that let you connect Yahoo to Google to MSN to Facebook to Zephyr to... it's network effects, people have to be on it for anyone to want to use it.
Plenty of people willing and able to build that experience, but not for free.
If we got a nickel from everyone that complained about XMPP client fragmentation, we would have funding to pay enough man-hours to get high-quality clients for every major platform and even the minor ones.
This is why I think the Matrix approach is superior. Matrix seems to prioritize features first and then makes changes to the spec based on the features to support. There's a minority that likes to criticize Matrix for this approach claiming it makes it hard for third-party implementers, but the point is that only people that write software would think about a spec first and the usecase second. No non-technical user cares.
Yes, someone should fix those iOS clients... I have Siskin, Monal and ChatSecure all running in parallel on my iPhone and none works as it is supposed to do.
Siskin seems to be the best of these three, but is neither as reliable nor as feature-complete as Conversations on Android or Gajim on the Desktop.
I think Android is the only platform that currently has a good user experience with Conversations or Quicksy for the less technical people.
There is a lot of parallels with Matrix here. I'm sad to say that the Element client just isn't polished enough to win me over Discord.
Most people, including myself, don't really care about the proprietary nature of Discord. I can fairly trivially move off Discord if it fails me so I don't really see the point in working harder to use Matrix.
Back in the day I love the trillio or something like that client that connected a bunch of different networks together in a single app for the very reason you talk about. I wanted to chat with my friends, and others but some where on AOL, some on MSN, some on gchat, etc etc
So I could connect to all the networks from one app
Today the would connect Facebook, Teams, Slack, Whatsapp, signal, etc etc...
Yes, this is generally the issue with open networks. Very, very hard to get complex and disparate client/servers to play nicely with each other. Perhaps with the capital that is being put behind web3 someone will figure this out.
I tried to run an open source community on XMPP at around the same time and found it equally crap.
The XML schema felt bloated (baring in mind a lot of people were still on dial up, and later the data caps of early feature/smart phones), chat client support was rubbish, linking to other servers was non-trivial (they'd often be running extensions you weren't. In fact the whole extension system was broken). And IIRC the leading XMPP server was written in Haskell and the only way to configure it was to edit the source. It was just a horrible experience all round.
I had Skype, MSN, AOL IM, IRC, and another (possibly IRQ?) all linked together -- most of which are proprietary -- under one server and the only protocol I couldn't get working properly was XMPP. Which really says a lot about the state of XMPP at that time.
Maybe things have improved. But now that Google Talk, Facebook Messenger and others don't use XMPP any more, there is even less reason to bother trying.
Suffice to say; go and download a modern client like conversations.im and install ejabberd.im on a machine somewhere (or just use a throwaway account on a public server). The ecosystem has significantly improved since you last tried it... 15-20 years ago?
XMMP was and is very commonly used in finance for various trading systems. Pretty much any exchange or a large trading firm with a white label product has a chat function in the desktop client and they all almost always used XMMP because amongst other things you needed federation.
It’s not falling out of fashion a bit with many of the trading clients moving to web based interfaces but it have a sense it will still stick around for a long time.
Any names you can name using federated xmpp? In my little corner of the trading world everybody seems to use either ICE Chat and/or Bloomberg. My guess would be ICE Chat is xmpp under the hood, but I don't think it's federated.
The thing I struggled with was how the XMPP chat based app I was trying to use was very much tied to the domain. If you weren't the domain admin you were SOL trying to set it up. Setting up two on the same domain--we were attempting to make it robust against disruption, think multiple servers on the same network where the interconnects may go down unexpectedly, but all local users need to remain connected and messages need to be relayed once the connections are re-established. They had previously used IRC which worked ok, but was "insecure" and had a mandate to switch to XMPP for security. Also had to be a COTS solution.
In the end I left the project before they got it working. I have no idea if it ever got deployed.
> The issue we ran into was customers/consumers just didn't care about federated systems.
Quite the opposite: users actively do not want federated systems, as federated systems allow anyone on the internet to spam them. This is why Google Talk stopped supporting federation: 99.9%+ of users never used it to talk outside of Google instances, and 99.9%+ of the inbound messages from the outside world were spam.
The spam problem was exaggerated and used as an excuse to avoid FTC scrutiny. Even if 99.9% of federated inbound messages were spam, that doesn't mean it all reached the client. Most e-mail is spam but very little makes it to the user. (AFAIU, the primary motivations were 1) lack of MSN reciprocity, and 2) Google's endless dithering and floundering attempts to capture users the same way Facebook, WhatsApp, etc had and/or were going to.)
Large walled garden IM platforms have the spam problem potential. It's solved by using whitelists (via invites) instead of blacklists. The same mechanism is trivial to accomplish using XMPP, and I'm sure there are already XEPs for that. (Multiple, even!) You can end up with invite spam, but that's exactly the problem you see on Facebook, WhatsApp, etc.
As for me, I ran my own XMPP server for years along side an SMTP server and never received XMPP spam. Both my XMPP and e-mail addresses were the same, and my e-mail address had been in use since circa 2000 and exists on countless spam lists. I doubt the spam problem could have been very significant. I never used a GMail or GTalk address, but of the people I know who did, some of whom I chatted with over federation, I never once heard of GTalk spam.
Anybody who cares about privacy should care about federation and should advocate for federation. Being cynical about the potential is counterproductive. If you think federation is impractical because centralization is required to prevent abuse, then you must admit that privacy beyond the current status quo is also impractical; just give up. If you don't want to give up on privacy, don't give up on federation. The technical problems to federation will never go away, but the will to solve them is entirely voluntary. (Though, if we legally mandate federation then that's at least a substantial, long-term commitment.)
I have used XMPP since forever, and actively used it for GTalk federation. I don’t think I remember any spam messages, but there might have been one or two.
Facebook/Messenger? Not a week goes by without some sexy lady wanting to have intercourse with me.
It's a little odd. Facebook and Google talk both allowed use of standard xmpp clients when they were using xmpp underneath - so one could get away with a single app to talk to people over xmpp, Facebook chat and gtalk. And eg OTR worked fine.
But both Facebook and Google refused federation, and then phased out xmpp.
I assume all the big players are equally eager to kill email, but haven't quite figured out how yet.
First, I can't remember who said it but there's a quote that goes something like (paraphrased) "open standards are for losers" (side note: if anyone has the original quote, please let me know). It's a pithy way of pointing out that the winners in any market don't care. Others want open standards to level the playing field.
Second, and I can't stress this enough, literally nobody wants federated systems apart from a handful techno-enthusiasts. It doesn't solve any problem users care about. In fact, it tends to create problems users do care about.
Look no further than the POTS system. Part of the reason you have spam and robocalls is one provider is pretty much free to send or forward traffic to anyone and to essentially spoof the sender such that things like caller ID do very little. Plus there's all the routing issues. Users tend to want to keep their numbers so you can't use a phone number as an identifier anymore. You have to use something lower level (think of this as the difference between a domain name and an IP address) and then deal with all the porting issues and the security issues that creates (eg SIM jacking).
Yes -- if you want a good open data system, you're in a bad place.
As long as people continue to use the Internet for their own benefit, that's fine. The internet that will be the ultimate "open standard" for the masses of people.
I reverse-engineered the comms for my cheap Ecovacs robot vacuum and was surprised to discover that, like some angsty teen, it spent all day hanging out in an XMPP chatroom waiting for somebody to talk to it: https://github.com/wpietri/sucks/blob/master/developing.md
It's little known that XMPP is a big hit with military and police organizations. (e.g. the Baltimore police department has XMPP-based chat and feeds the messages into Lotus Notes)
Vendors in that market heavily use extension mechanisms. For instance commanders of infantry units might fill out forms every day about what kind of contacts they had with the enemy and get them sent to a central location.
> Vendors in that market heavily use extension mechanisms. For instance commanders of infantry units might fill out forms every day about what kind of contacts they had with the enemy and get them sent to a central location.
On the battlefield you don't have good internet so the cycle of GET the HTML form and POST the fields is not reliable.
XMPP Forms are deeply asynchronous. I'd assume that the "blank form" already exists on the soldier's laptop and the message gets stored-and-forwarded until it gets to the destination.
heh, back in my day that role was filled by IRC over a low bandwidth SIPRNet link - with each room being a different command or area of operation. You can see evidence of this from Manning's tactical data dump.
Also, the bridge system in matrix is solid. Most people do not want to ban all other messengers, but they want all their communication in one place. Matrix-based messengers like Element or Bleeper seem like the way to go. And if people then start using matrix based chatrooms, all the better.
But I doubt you can reach critical mass with a system that is not open to other chat systems, even if those are closed.
XMPP was developed as an open protocol to bridge to proprietary communication platforms, just as Matrix is today.
There are still such bridges (more commonly called "gateways" or "transports" in XMPP), such as for WhatsApp, Signal, and others. However my perception is that over the long period of time XMPP has been around, focus shifted to the "native" XMPP-XMPP use-case, because it's easier to build a better experience that way.
My personal prediction is that Matrix will certainly go the same way - using a focus on bridging to bootstrap the network in the initial years, and then gradually shifting to focus on "native Matrix" for most use-cases after reaching a certain critical mass.
Yup, Matrix as a network has no central authority at all, although it's a common untruth which gets passed around, rather depressingly.
Things which could be at the root of this misunderstanding include:
* The fact that we operate a *strictly optional* logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves - we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead.
* Element runs a centralised integration manager (aka app store) to make it easier to add bots/bridges/widgets to rooms. Again, this is completely optional - although back in 2019 we had a stupid bug in Element Web which meant it checked the integration manager even if you weren't using it. This was fixed in June 2019 (https://github.com/vector-im/element-web/issues/6802).
* Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this.
* Servers need to be able to validate events from servers even if those servers are offline. To do this you need to grab the public key for the offline server from somewhere; by default Synapse uses matrix.org as a fallback, but loudly nags the admin to pick a better default when you install the server.
None of these actually allow a "central authority to have influence on another server" though; I've just listed the only places I can think of where a typical self-hosted Matrix implementation might interact with centralised services run by Matrix.org or Element.
Unfortunately I think the real root of the misinformation is that it's just a bit too tempting and easy to throw around FUD. Some day we may get to the point where alternative FOSS projects will realise that the real enemy are centralised proprietary user-exploiting services rather than one another...
> The fact that we operate a strictly optional logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves
(Not a Matrix user so maybe help me understand here.) I think the reason why I have about 30 contacts (instead of like 3) on Signal is due to phone number discovery. It's not decentralized, which is of course subject to criticism, but exchanging & verifying public keys between O(N^2) people hasn't "taken off" in the X decades since it was invented.
> we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead.
Do you have an example of any mainstream product that has ever achieved that, without either being partly centralized? Decentralized identity is really, really, really hard, and mostly the layer-8 implementation (discovery, name-squatting, impersonation, abuse, spam, key-management and exchange, etc).
Every Matrix protocol client I've ever tried, including yours, seems to connect to your "strictly optional" centralized directory service. I've never once wanted to use it, and, despite your claims of it being optional, 100% of the times I have logged in to a Matrix instance, via the web or via an app, my computer has talked to your computer.
Amazing work. Bravo. I'm very much hoping to see P2P Matrix arise. That is no small task, I realize (and I'm grateful to those who have contributed to this). What are you thoughts on that aspect of the ecosystem?
I just tried logging in to Element.io with an account on my own homeserver and I saw several requests to vector.im before the privacy policy popup, including what seems to be some kind of token exchange.
I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked.
From a protocol standpoint, no identity server is necessary. From a practical endpoint, all feature-rich clients (the ones I know support e2ee) seem to send requests to vector.im. Matrix is decentralized, but a lot of data still seems to get leaked to a centralized place by accident from my short experiments. It's not a great scenario, even if I'm fine with a few requests to the 3PID server every now and then.
Client issues are ecosystem issues if all usable clients suffer from them.
Matrix is a lot better than every proprietary service, but it's poor XMPP compliance currently makes it impossible to have essential features like encrypted communications or adding Matrix users to multi-user chats.
I worked on a large XMPP system for a while and realized it's actually not a very good protocol, for one reason: presence. "Presence" is the feature that shows a little indicator next to each one of your contacts indicating whether they're online or away.
The problem with presence is that it generates a LOT of traffic. It's N^2 with the average number of contacts each user has - every time a user changes presence, you have to send a message to every one of their contacts to update them about the presence. This happens whether they're paying attention to your client or not. So on a large system, the big majority of the traffic is just presence messages flying back and forth saying who is at their keyboard and who isn't. This is really wasteful in general. Worse, it's not a super important feature for a lot of people, so it's definitely not worth the effort. And ever since mobile computing became ubiquitous, it's not even clear what this is supposed to represent.
So, I say let XMPP die. It was a fine idea for its time. But the protocol has no place in the modern world.
Actually, it is an important feature for a lot of people, with virtually every popular messenger implementing it. Telegram and Whatsapp both go one step further and not only show when a user is at his keyboard, but also if he's currently typing. In the grand scheme of things, a few strings being routed regarding presence information don't pose a big problem for server providers (I know two of those and they didn't complain nor think of it as an issue whatsoever).
You shouldn't impose your use case to "a lot of people", and more importantly also shouldn't conclude that it's "definitely not worth it" from that. The industry disagrees with you, and so does my mom.
When somebody is typing or not does not get broadcast to every contact. It’s only sent to the individual who is about to receive the typed message. O(1) communication vs O(N). That is a different feature which is not XMPP presence and I agree way more relevant in the modern world.
Even if the server load from presence is acceptable (we clearly disagree here) it’s not okay to bombard somebody’s mobile device with these messages all the time. If your phone’s CPU and radio need to wake from sleep every time any one of your contacts picks up or puts down their phone, your battery life is ruined. You may think these messages important, but they are not urgent.
I wonder why people jump fast to conclusion that to "let XMPP die", when the protocol can also be iteratively improved. Presence is not required in XMPP, its an optional feature. Everything you said has been considered in newer XMPP extension protocols (like XEP-0396: MIX).
Why improve when you can start over? Modern tools for pub sub and websockets are so much better. We don’t need to use XML any more. (For the love of god please let XML die.) The old code and implementations are more of a liability than an asset, especially if you want to take out major functionality. Unless interop with AOL instant messenger’s install base is a key feature or something. (Don’t know if AIM uses XMPP but the timing is about right and representative of what you’d get.)
That's afaik a problem with pretty much any system that supports presence, even though it can be somewhat optimized potentially (where I don't know how many options exist in XMPP, but I don't think would be fundamentally impossible to add)?
I remember using XMPP for a site I built that connected tenants and landlords, a reverse MLS of sorts. Back then, I choose XMPP because I had heard this is what Facebook was using at the time for their new messenger and it provided services like online status, typing status, and live messaging out-of-the-box. I don't recall it being terribly difficult to implement and it was awesome to see it working for the first time!
This was almost 10-years ago now, it's incredible to see this technology still moving forward.
This was my experience. It did end up doing what we needed and being able to use off the shelf libraries/servers at the time was convenient, but holy heck is the protocol chatty and annoying to debug at the trace level.
Instead of all the web3 nonsense, which still requires implementations working with interoperable protocols like this, that energy should be put into advocacy for standard protocols, or building apps that use them.
Yup, I second this. The whole web3 idea is completely pointless if the ecosystem is designed to be heavily centralized. Everyone is and will be fighting for a pie in the “platform” market. We need decentralization in the application side first, and that is only possible with standard protocols.
What an earnest and well-meaning BS post.... I mean seriously, XMPP is fundamentally flawed, and it's not coming back. What's more, we have matrix which is the protocol re-written from the ground up with the knowledge of XMPP and better design.
I use both XMPP and Matrix regularly, and run servers for both. I wouldn't say Matrix is better than XMPP across the board, but that they have different trade-offs.
The main architectural difference between them is that Matrix provides "eventually-consistent cryptographically secure synchronisation of room state across a global open network of federated servers and services", whereas XMPP just passes messages from point to point (where one of the endpoints may be a multiuser room).
The main practical benefit of Matrix is that multi-user rooms don't have a "home" server; they're distributed across the homeservers of every member of the room. This means that though everyone may have signed up for a room identified as #shitposting:matrix.org, if the matrix.org homeserver goes down, the chat still works. People can't join it until a new name is published for it, but it works for everyone already in it. In XMPP, chats are hosted on a particular server, and if the server a chat is on goes down, the chat goes down.
The main practical benefit of XMPP is that it is much more lightweight than Matrix. Being based on synchronization of room state means that Matrix stores a lot of data, generally all messages and attachments back to when the room was created (or possibly the first time someone on your homeserver joined the room). Apparently it's possible to prune room history, but it's not done by default, and as far as I know, it's not officially supported. It's much easier to control how much data your XMPP server stores.
XMPP is also a simpler protocol, which means there is a wider variety of clients; while there are several vaguely viable Matrix clients, if you're not using Element, you are much more likely to have problems, especially with encrypted rooms. Of course, the flip side of this is that a lot of XMPP clients don't support all the extensions which make modern XMPP useful, either.
Both of them have trouble with multi-device E2EE key management, though Matrix has the edge. But again, if you're not using Element, you are likely to have problems.
> whereas XMPP just passes messages from point to point
So does Matrix. All the other qualities are build on top of that. XMPP also enables e2ee, message synchronisation and an "global open network of federated servers and services".
Matrix is not even a protocol. It is by far inferior to xmpp, especially regarding extendability, and the only flaw in XMPP is a rather lethargic council of elders that are not really interested in its development. This council is too be ignored, the base of the protocol is healthy.
You say Matrix is not a protocol, and then you go on and compare it to another protocol. By your own logic, you are comparing apples and oranges here. Make up your mind, either it is a protocol or it isn't.
The TL;DR seems to be the concern that Matrix was created by an existing team of folks who worked together (we built VoIP stacks for telcos), and because 50% of the people in the Spec Core Team who define Matrix's direction are from that original team, they have a disproportionate influence over how the protocol develops.
The reality, as far as I can tell, is that we are incorporating the best proposals from across the whole community, wherever they come from, and the protocol is progressing steadily under this open governance model, and folks are constantly experimenting with new features under prefixes, which they propose MSCs (Matrix Spec Changes) for, and then get incorporated into the official spec once they're approved by the Spec Core Team (which requires 75% consensus).
We spent ages setting up https://matrix.org/foundation and trying to enshrine a fair and neutral governance process for the spec, and as far as I can tell it's overall working. The thread linked above gives examples of ways in which the process is working as intended.
It'd take the flawed XMPP protocol any day over the ad-hoc implementation of Matrix, which hardly even deserves to be called a protocol. Matrix feels like something made by making things up as the wrote the code, there is no coherency at all.
Jabber was written by making things up as we wrote the code, and then pitching a prototype to a core group for revisions.
XMPP didn't really pitch any client or server protocol breaking changes.
Since the messages themselves were just addressed XML elements, we were able to do revisions over time, such as the first group chat being replaced by MUC or adding in the ability for XHTML-formatted rich text.
I'm not going claim that it's fundamentally flawed, but here's an anecdote. Many years ago, when XML was having its day in the sun, long before it was sidelined by the simplicity of REST and JSON, I was at a Java One convention listening to a speaker present on some new XML parsing API. After the talk, I approached the presenter for some post-talk Q&A to ask how one might use the API to parse the Jabber protocol, which may or may not be relevant to what XMPP is today (I haven't been keeping up.)
The presenter was unfamiliar with the protocol, so I had to describe how the xml document was opened when you establish a connection, and how elements keep getting appended to it, and how the "xml document" isn't really completed until you're all done and the connection is terminated.
They looked at me like I had two heads.
To them, XML didn't make any sense at all unless you have the entire document available all at once. After all, how on earth could one ever apply an XSLT transform to it, right!?
I really wanted to like XMPP back in the days, but honestly I always ended up feeling that the protocol is just bad. This idea of opening an XML document at the beginning of the stream, then only allowing a subset of XML and all the mess with the xmlns, all that bringing really nothing to the table except complexity.
I think ideally in a good protocol, the server should not have to parse the content for the messages that are not targetted to itself (only the metadata useful for routing). the XML mess makes it impossible to do that since you have to validate the full document.
At the time I think this page was a good summary of the issues https://about.psyc.eu/XMPP
No idea if this is still relevant though.
Readit News
They have things they want done, like chatting with a friend. They want a solution that delights them in doing that. What delights people has generally changed (like more people care about others not listening in these days).
Whose the user and what's their experience? This is where XMPP always failed me. The clients had a poor experience.
If someone wants that to change now... focus on a great experience for the people you want to be users.
These days I believe very much in identifying and serving a specific set of users. XMPP is just a protocol, it has lots of potential applications, but it's nothing unless you channel that into a tangible usable product for real people.
This epiphany came to me after more than 10 years of working on XMPP as a server developer and spending time promoting open communication protocols to people. It was soul-crushing that at the end of a day I would still see my own family communicating with each other via WhatsApp.
Six months after the initial Snikket prototypes, and after a little user testing, I migrated my family over to XMPP by sending a simple invitation link (the start of the Snikket onboarding journey). They're now using it daily and totally happy with it. I'm beginning to hear similar stories from others who have repeated this with their own families/groups and their own Snikket setups.
All it took was to look at the issue through the eyes of "normal people" for a moment, and it changed my approach (and success rate) significantly.
[1]: https://snikket.org/blog/products-vs-protocols/
I've been looking for a modern self-hosted chat solution. I want it to be XMPP so badly (running XMPP with something like prosody is so convenient). The clients are really letting me down though. Desktop clients are old-fashioned and awful (I could get behind that, but I don't think my kids could). The mobile situation is even worse.
The new-fangled chat solutions (Matrix, Zulip, etc.) are interesting, but the operational requirements are insane (in the case of Matrix, I'm also very nervous about reliability).
I can't say I'm super hopeful, but I'm rooting for you, XMPP.
[1] https://f-droid.org/en/packages/eu.siacs.conversations/
[2] https://gajim.org/download/
[3] OMEMO is an XMPP Extension Protocol (XEP) for secure multi-client end-to-end encryption based on Axolotl and PEP. Might want to go through https://dev.gajim.org/gajim/gajim-plugins/-/wikis/OmemoGajim....
It's a nice protocol, but if you're missing all the conversations it doesn't help. When it was popular, it was the system that let you connect Yahoo to Google to MSN to Facebook to Zephyr to... it's network effects, people have to be on it for anyone to want to use it.
If we got a nickel from everyone that complained about XMPP client fragmentation, we would have funding to pay enough man-hours to get high-quality clients for every major platform and even the minor ones.
Siskin seems to be the best of these three, but is neither as reliable nor as feature-complete as Conversations on Android or Gajim on the Desktop.
I think Android is the only platform that currently has a good user experience with Conversations or Quicksy for the less technical people.
Most people, including myself, don't really care about the proprietary nature of Discord. I can fairly trivially move off Discord if it fails me so I don't really see the point in working harder to use Matrix.
So I could connect to all the networks from one app
Today the would connect Facebook, Teams, Slack, Whatsapp, signal, etc etc...
That would be the dream....
The XML schema felt bloated (baring in mind a lot of people were still on dial up, and later the data caps of early feature/smart phones), chat client support was rubbish, linking to other servers was non-trivial (they'd often be running extensions you weren't. In fact the whole extension system was broken). And IIRC the leading XMPP server was written in Haskell and the only way to configure it was to edit the source. It was just a horrible experience all round.
I had Skype, MSN, AOL IM, IRC, and another (possibly IRQ?) all linked together -- most of which are proprietary -- under one server and the only protocol I couldn't get working properly was XMPP. Which really says a lot about the state of XMPP at that time.
Maybe things have improved. But now that Google Talk, Facebook Messenger and others don't use XMPP any more, there is even less reason to bother trying.
Suffice to say; go and download a modern client like conversations.im and install ejabberd.im on a machine somewhere (or just use a throwaway account on a public server). The ecosystem has significantly improved since you last tried it... 15-20 years ago?
It’s not falling out of fashion a bit with many of the trading clients moving to web based interfaces but it have a sense it will still stick around for a long time.
In the end I left the project before they got it working. I have no idea if it ever got deployed.
Quite the opposite: users actively do not want federated systems, as federated systems allow anyone on the internet to spam them. This is why Google Talk stopped supporting federation: 99.9%+ of users never used it to talk outside of Google instances, and 99.9%+ of the inbound messages from the outside world were spam.
Large walled garden IM platforms have the spam problem potential. It's solved by using whitelists (via invites) instead of blacklists. The same mechanism is trivial to accomplish using XMPP, and I'm sure there are already XEPs for that. (Multiple, even!) You can end up with invite spam, but that's exactly the problem you see on Facebook, WhatsApp, etc.
As for me, I ran my own XMPP server for years along side an SMTP server and never received XMPP spam. Both my XMPP and e-mail addresses were the same, and my e-mail address had been in use since circa 2000 and exists on countless spam lists. I doubt the spam problem could have been very significant. I never used a GMail or GTalk address, but of the people I know who did, some of whom I chatted with over federation, I never once heard of GTalk spam.
Anybody who cares about privacy should care about federation and should advocate for federation. Being cynical about the potential is counterproductive. If you think federation is impractical because centralization is required to prevent abuse, then you must admit that privacy beyond the current status quo is also impractical; just give up. If you don't want to give up on privacy, don't give up on federation. The technical problems to federation will never go away, but the will to solve them is entirely voluntary. (Though, if we legally mandate federation then that's at least a substantial, long-term commitment.)
Facebook/Messenger? Not a week goes by without some sexy lady wanting to have intercourse with me.
This does not seem like it’s a federation issue.
But both Facebook and Google refused federation, and then phased out xmpp.
I assume all the big players are equally eager to kill email, but haven't quite figured out how yet.
First, I can't remember who said it but there's a quote that goes something like (paraphrased) "open standards are for losers" (side note: if anyone has the original quote, please let me know). It's a pithy way of pointing out that the winners in any market don't care. Others want open standards to level the playing field.
Second, and I can't stress this enough, literally nobody wants federated systems apart from a handful techno-enthusiasts. It doesn't solve any problem users care about. In fact, it tends to create problems users do care about.
Look no further than the POTS system. Part of the reason you have spam and robocalls is one provider is pretty much free to send or forward traffic to anyone and to essentially spoof the sender such that things like caller ID do very little. Plus there's all the routing issues. Users tend to want to keep their numbers so you can't use a phone number as an identifier anymore. You have to use something lower level (think of this as the difference between a domain name and an IP address) and then deal with all the porting issues and the security issues that creates (eg SIM jacking).
As long as people continue to use the Internet for their own benefit, that's fine. The internet that will be the ultimate "open standard" for the masses of people.
https://www.qacafe.com/resources/2015-03-26-using-xmpp-for-t...
Vendors in that market heavily use extension mechanisms. For instance commanders of infantry units might fill out forms every day about what kind of contacts they had with the enemy and get them sent to a central location.
How does XMPP help with this over a HTTP form?
XMPP Forms are deeply asynchronous. I'd assume that the "blank form" already exists on the soldier's laptop and the message gets stored-and-forwarded until it gets to the destination.
But: - HTTP forms might be less flexible when it is about structured data. - HTML needs a complex renderer software
Remember that this stuff might need to work on low resource devices and over low bandwidth connections.
Elaborate please. Matrix doesn't have a central authority server which can influence other servers.
But I doubt you can reach critical mass with a system that is not open to other chat systems, even if those are closed.
There are still such bridges (more commonly called "gateways" or "transports" in XMPP), such as for WhatsApp, Signal, and others. However my perception is that over the long period of time XMPP has been around, focus shifted to the "native" XMPP-XMPP use-case, because it's easier to build a better experience that way.
My personal prediction is that Matrix will certainly go the same way - using a focus on bridging to bootstrap the network in the initial years, and then gradually shifting to focus on "native Matrix" for most use-cases after reaching a certain critical mass.
Solid? They crash or malfunction all the time.
Things which could be at the root of this misunderstanding include:
* The fact that we operate a *strictly optional* logically centralised directory service to look up matrix IDs by email address or phone number. I really wish we'd never added this to Matrix; it causes way more grief than it solves - we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead.
* Element runs a centralised integration manager (aka app store) to make it easier to add bots/bridges/widgets to rooms. Again, this is completely optional - although back in 2019 we had a stupid bug in Element Web which meant it checked the integration manager even if you weren't using it. This was fixed in June 2019 (https://github.com/vector-im/element-web/issues/6802).
* Element has to query your server to find out what authentication mechanisms are available before you log in. Therefore if its config points to the default matrix.org homeserver, you end up pinging that every time you log in, even if your homeserver is elsewhere. The solution for now is to change your default homeserver in the config to avoid this.
* Servers need to be able to validate events from servers even if those servers are offline. To do this you need to grab the public key for the offline server from somewhere; by default Synapse uses matrix.org as a fallback, but loudly nags the admin to pick a better default when you install the server.
None of these actually allow a "central authority to have influence on another server" though; I've just listed the only places I can think of where a typical self-hosted Matrix implementation might interact with centralised services run by Matrix.org or Element.
Unfortunately I think the real root of the misinformation is that it's just a bit too tempting and easy to throw around FUD. Some day we may get to the point where alternative FOSS projects will realise that the real enemy are centralised proprietary user-exploiting services rather than one another...
(Not a Matrix user so maybe help me understand here.) I think the reason why I have about 30 contacts (instead of like 3) on Signal is due to phone number discovery. It's not decentralized, which is of course subject to criticism, but exchanging & verifying public keys between O(N^2) people hasn't "taken off" in the X decades since it was invented.
> we'd have been better off waiting off for someone to build a decentralised keybase service and use that instead.
Do you have an example of any mainstream product that has ever achieved that, without either being partly centralized? Decentralized identity is really, really, really hard, and mostly the layer-8 implementation (discovery, name-squatting, impersonation, abuse, spam, key-management and exchange, etc).
Something's broken here.
I don't think the intent is malicious, but the system is definitely leaky. My homeserver config doesn't mention vector.im (perhaps it's a hidden default somewhere?), I checked.
From a protocol standpoint, no identity server is necessary. From a practical endpoint, all feature-rich clients (the ones I know support e2ee) seem to send requests to vector.im. Matrix is decentralized, but a lot of data still seems to get leaked to a centralized place by accident from my short experiments. It's not a great scenario, even if I'm fine with a few requests to the 3PID server every now and then.
Client issues are ecosystem issues if all usable clients suffer from them.
The problem with presence is that it generates a LOT of traffic. It's N^2 with the average number of contacts each user has - every time a user changes presence, you have to send a message to every one of their contacts to update them about the presence. This happens whether they're paying attention to your client or not. So on a large system, the big majority of the traffic is just presence messages flying back and forth saying who is at their keyboard and who isn't. This is really wasteful in general. Worse, it's not a super important feature for a lot of people, so it's definitely not worth the effort. And ever since mobile computing became ubiquitous, it's not even clear what this is supposed to represent.
So, I say let XMPP die. It was a fine idea for its time. But the protocol has no place in the modern world.
You shouldn't impose your use case to "a lot of people", and more importantly also shouldn't conclude that it's "definitely not worth it" from that. The industry disagrees with you, and so does my mom.
Even if the server load from presence is acceptable (we clearly disagree here) it’s not okay to bombard somebody’s mobile device with these messages all the time. If your phone’s CPU and radio need to wake from sleep every time any one of your contacts picks up or puts down their phone, your battery life is ruined. You may think these messages important, but they are not urgent.
This was almost 10-years ago now, it's incredible to see this technology still moving forward.
I had the impression the general opinion about XMPP is that it's resented as the SOAP of messaging.
Matrix IS XMPP 2.0.
The main architectural difference between them is that Matrix provides "eventually-consistent cryptographically secure synchronisation of room state across a global open network of federated servers and services", whereas XMPP just passes messages from point to point (where one of the endpoints may be a multiuser room).
The main practical benefit of Matrix is that multi-user rooms don't have a "home" server; they're distributed across the homeservers of every member of the room. This means that though everyone may have signed up for a room identified as #shitposting:matrix.org, if the matrix.org homeserver goes down, the chat still works. People can't join it until a new name is published for it, but it works for everyone already in it. In XMPP, chats are hosted on a particular server, and if the server a chat is on goes down, the chat goes down.
The main practical benefit of XMPP is that it is much more lightweight than Matrix. Being based on synchronization of room state means that Matrix stores a lot of data, generally all messages and attachments back to when the room was created (or possibly the first time someone on your homeserver joined the room). Apparently it's possible to prune room history, but it's not done by default, and as far as I know, it's not officially supported. It's much easier to control how much data your XMPP server stores.
XMPP is also a simpler protocol, which means there is a wider variety of clients; while there are several vaguely viable Matrix clients, if you're not using Element, you are much more likely to have problems, especially with encrypted rooms. Of course, the flip side of this is that a lot of XMPP clients don't support all the extensions which make modern XMPP useful, either.
Both of them have trouble with multi-device E2EE key management, though Matrix has the edge. But again, if you're not using Element, you are likely to have problems.
So does Matrix. All the other qualities are build on top of that. XMPP also enables e2ee, message synchronisation and an "global open network of federated servers and services".
Oh, and Matrix is plenty extensible, for example via custom event types: https://spec.matrix.org/v1.1/client-server-api/#types-of-roo...
The TL;DR seems to be the concern that Matrix was created by an existing team of folks who worked together (we built VoIP stacks for telcos), and because 50% of the people in the Spec Core Team who define Matrix's direction are from that original team, they have a disproportionate influence over how the protocol develops.
The reality, as far as I can tell, is that we are incorporating the best proposals from across the whole community, wherever they come from, and the protocol is progressing steadily under this open governance model, and folks are constantly experimenting with new features under prefixes, which they propose MSCs (Matrix Spec Changes) for, and then get incorporated into the official spec once they're approved by the Spec Core Team (which requires 75% consensus).
We spent ages setting up https://matrix.org/foundation and trying to enshrine a fair and neutral governance process for the spec, and as far as I can tell it's overall working. The thread linked above gives examples of ways in which the process is working as intended.
Please explain why you believe this.
This is the spec: https://spec.matrix.org/latest/
These are the spec change proposals: https://spec.matrix.org/unstable/proposals/
XMPP didn't really pitch any client or server protocol breaking changes.
Since the messages themselves were just addressed XML elements, we were able to do revisions over time, such as the first group chat being replaced by MUC or adding in the ability for XHTML-formatted rich text.
how is it fundamentally flawed, can you elaborate?
The presenter was unfamiliar with the protocol, so I had to describe how the xml document was opened when you establish a connection, and how elements keep getting appended to it, and how the "xml document" isn't really completed until you're all done and the connection is terminated.
They looked at me like I had two heads.
To them, XML didn't make any sense at all unless you have the entire document available all at once. After all, how on earth could one ever apply an XSLT transform to it, right!?
Good times.
I think ideally in a good protocol, the server should not have to parse the content for the messages that are not targetted to itself (only the metadata useful for routing). the XML mess makes it impossible to do that since you have to validate the full document.
At the time I think this page was a good summary of the issues https://about.psyc.eu/XMPP No idea if this is still relevant though.