I threw in the towel after about 10 years but not because of deliverability problems. When I first started running an email server for myself and my friends and family, it was a great learning experience. And it kept being a good learning experience for a few years as I rolled out things like Bayesian spam filters and SPF.
But eventually it got stable enough and did its job well enough that I stopped messing with it to learn things. And then it became a job instead of a hobby: friends texting me because my ISP was having an outage and they couldn't read their email, rolling out critical security patches ASAP, coping with the occasional DOS attack, and so on.
At some point, I realized that I'd started to resent running it. It was eating little bits of my time for no additional benefit to me.
I also, as one does with age, started thinking about what happens if I die or become incapacitated: my friends and family all lose their email service the next time there's an outage that requires manual recovery.
So I moved it to a commercial email service and have zero regrets about the move. I make sure I've always prepaid for at least 6 months of service so that if something happens to both me and my wife, my friends and family will at least have half a year to make other arrangements.
I'm likewise curious - a family member runs a mail service on a box in his basement for everyone with a @lastname.com with about 40 or 50 inboxes. He's indicated he'd like to pass that responsibility on as he's nearing 75 years old and the everchanging spam and delivery rules make it tedious to keep up, but I have very little interest in running a mail server myself. Would love to find somewhere for super low volume mail under $100 or $150/month that could keep that domain intact..
"But my emails are just not delivered anymore. I might as well not have an email server."
I will never understand people who see it as an all or nothing proposition. You can perfectly self-host while using an outgoing SMTP provider. This would solve all the problems the OP is having while still preserving the main benefit of self-hosting, meaning: managing your own MX records, you own IMAP server, so that government authorities don't have a single organization they can target to get access to your emails. Heck, if you are paranoid you can even have multiple outgoing SMTP providers and round-robin between them.
Edit: the author does mention not wanting to pay for an outgoing SMTP provider, but there are many free option: for example Gmail lets you use their SMTP server as long as you authenticate with a (free) Google account. That's possible with Yahoo, or many other providers.
> You can perfectly self-host while using an outgoing SMTP provider.
That's mostly selfhosting, not fully selfhosting. I full selfhost my email, I 100% control as much of it as is possible to do. When I send an email to someone, it goes from my client to my server and out to the recipient's server.
Many people who self-host email do so to avoid having years of all their personal email communications logs centralized at one third-party company with ether lax security or lax privacy.
It seems to me that you are proposing exactly this: routing all emails through one of these companies. How can this be a solution to "self-hosting"?
I empathize with the author's viewpoint - maybe it's just my personal experience of running email for large corporations and ISPs but there's a reason why these systems have ended up with zero tolerance policies for spam sources.
It's just not worth the overhead in people and hardware to try to be surgical in your approach - you don't win anything. It's the closest thing to working at the DMV for tech.
Notes, cc:Mail, Groupwise, Exchange, Sendmail, Exim, qmail, postfix, etc... I have no interest in running my own email ever again.
> there's a reason why these systems have ended up with zero tolerance policies for spam sources
The article is not about blackholing spam sources. It's about blackholing mail from sources that have never sent a single spam message since the dawn of the internet.
It's not, though - it's about blackholing unknown sources. Yes, this makes it incredibly difficult to self-host or to start up a new provider in the space. But from the perspective of anyone trying to protect their users, it makes sense.
By definition, every new source has never sent spam -- but it's reasonable to assume that an unknown source is likely spam, however unfortunate that may be.
This author claims to be sending alumni newsletters from his server. I hey you anything that someone has (at least once) marked one of those newsletters as spam.
> "You just cannot create another first-class node of this network.
Uh, yeah, you can, I did it again recently. I've done it at Oracle Cloud, AWS Lightsail, Linode, GoDaddy, and several ISP's before that. I could start right this minute with Digital Ocean, Vultr, etc. pick one and be up and running, delivering email to Gmail, Microsoft, etc. in a day.
Each time one of these articles is posted, I feel I must be some kind of email savant.
Here's the recipe: set up your own email server. Tweak configuration until eventually a test email from yourself to yourself lands in the inbox, then call it a day. Never actually measure your deliverability. Never investigate why you sometimes don't get replies to emails where you were expecting to get a reply to. In fact, just close your eyes and stick fingers in your ears. Then go on HN and talk about how easy it is to do this thing which you, totally, for real, really did do, like, for reals for reals.
Gmail, Hotmail and Office365 are the largest email providers and also the most strict ones.
I have accounts on all of these providers so testing deliverability is trivial. You could argue that testing deliverability with one account is unreliable but in my experience it simply is not. Usually you send a couple emails across a week and if they all go through you are good.
If you are paranoid, you can ask if any of your friends have Office365/Hotmail and email them. They probably have company or university accounts on there.
My server has only been blocked once after all this, and ironically it was by another company that self-hosts their email...
We read from different cookbooks and my reading comprehension is high. My sent email stats shows over 20K successfully sent emails just for my personal account, since 1999, when I started tracking it.
I went down an internet rathole on coffee grinders. (I realize you were probably referring to health benefits.)
After many hours of this over many months, I finally looked for a comparison of the actual taste of coffee from cheap grinders versus 10-20x more expensive burr grinders. (As opposed to comparisons of grinder technologies, which are everywhere.)
There are almost no published side-by-side taste comparisons; when I did find one, the cheap grinders had won!
Sure, but how long would your new server keep working (i.e., delivering mail to GMail, etc)? Do you know some magic incantation for staying off the naughty lists?
Setting up a server is trivial, yes. Keeping it going is a never-ending treadmill of not really technical problems.
Do you know some magic incantation for staying off the naughty lists?
Not magic, but when I managed outgoing Postfix servers for a few companies I had to set rate limits for yahoo.com an a couple other domains to reduce concurrency or they would block one of the SNAT's for a while. It probably sounds tedious but it really wasn't. There were not many MX that were as strict as Yahoo. I never ran into issues with Gmail but I think they cut some slack for corporate IP addresses and domain names.
For my own personal email servers I never had issues because I never sent at a rate that anyone cared about. The closest I got to that was running a forum that would email when threads would get updated and people subscribed to them but my solution there was to suggest to the people on the forum not to do that.
> Sure, but how long would your new server keep working (i.e., delivering mail to GMail, etc)?
I can't see the future, if the big email providers who likely have some of their trolls posting in the comments ever decide to start choking out us personal email server runners, then it'd be game over. If things remain for the next 20 years assuming I live that long then deliverability would be 100% for the next 20 years.
> Do you know some magic incantation for staying off the naughty lists?
I don't spam, that and don't make a finger fumble edit like I did the one time in over 20 years and didn't check to make sure it was working correctly first.
I wish there was a good nonprofit 'infrastructure cooperative' that could provide some of these core services but have a corporate governance that could be trusted. The place I most want it for is a domain name registrar but DNS and mail servers would be good additional services.
You know, if you started one, I bet there would be a bunch of people here who would use it. I would be one of them (but I don't have the time to start such an endeavor).
Man, these articles come out every once in a while. I'm in this camp. My own email server going on 25 years. I'm also responsible for the email of other businesses as well. And maybe that's the difference? I'm getting paid to understand the nuances in mail delivery? But it's a chore no doubt. I never plan on stopping. EMail is TOO IMPORTANT. I don't have any secret formula other than be reachable. Read your email to postmaster@domain, sign up for all the Postmaster Feedback loops. I have two major issues (which have lessened over time) 1) you have to slow-deliver mail to gmail/comcast/yahoo/hotmail or you will get deferred very quickly. 2) users getting their passwords hacked leading to spam delivery. But I catch that automatically these days with a script that checks the IP of the connection. Too many successful connections from Pakistan within a minute gets the account disabled. It's kept me off blacklists for years.
I self host my own email using mailinabox. Its working like a charm for my personal use case.
If anyone is interested in setting this up by themselves, here is the opentofu (formally terraform) code I am using: https://github.com/JonasTaulien/opentofu-mailinabox
The odds that this works is basically zero. Your post didn't refer to deliverability at all and the repo's README didn't address deliverability either. I can basically guarantee a portion of your email is not being delivered, you just aren't measuring it.
Its true, that I did not refer to deliverability.
But I can do this now :)
Mailinabox has a dahboard that shows you if you are on any spam list. It will also just stop with the setup, if it detects, that the IP of your machine is blacklisted.
The cases where my email got blocked, I always received an email back from the box, so I think there are no cases where my email just disappeared into the dark.
Reasons for blocking where:
- Some strange spamlist blocked my IP. I was able to resolve this by filling out a i-am-not-a-spammer-form on the spamlist provider
- Some email server required from me that I add my personal address to my website, so that they know who maintains the server
Never have I send an email but it was not delivered without notice
>In many countries politicians are forced to deploy their own email servers for security and confidentiality reasons. We only need one politician's emails not delivered due to poorly implemented or arbitrary hellbans and this will be a hot button issue.
The GOP sued Google for blocking their campaign e-mails and lost.
They weren't blocked. They were received and filtered into the spam folder. Which is damned well what it should be; politicians shouldn't be immune to spam blocking.
These cries of protest were just more ridiculous outrage fuel, from a party that loves portraying themselves as some kind of victim.
Readit News
But eventually it got stable enough and did its job well enough that I stopped messing with it to learn things. And then it became a job instead of a hobby: friends texting me because my ISP was having an outage and they couldn't read their email, rolling out critical security patches ASAP, coping with the occasional DOS attack, and so on.
At some point, I realized that I'd started to resent running it. It was eating little bits of my time for no additional benefit to me.
I also, as one does with age, started thinking about what happens if I die or become incapacitated: my friends and family all lose their email service the next time there's an outage that requires manual recovery.
So I moved it to a commercial email service and have zero regrets about the move. I make sure I've always prepaid for at least 6 months of service so that if something happens to both me and my wife, my friends and family will at least have half a year to make other arrangements.
I will never understand people who see it as an all or nothing proposition. You can perfectly self-host while using an outgoing SMTP provider. This would solve all the problems the OP is having while still preserving the main benefit of self-hosting, meaning: managing your own MX records, you own IMAP server, so that government authorities don't have a single organization they can target to get access to your emails. Heck, if you are paranoid you can even have multiple outgoing SMTP providers and round-robin between them.
Edit: the author does mention not wanting to pay for an outgoing SMTP provider, but there are many free option: for example Gmail lets you use their SMTP server as long as you authenticate with a (free) Google account. That's possible with Yahoo, or many other providers.
That's mostly selfhosting, not fully selfhosting. I full selfhost my email, I 100% control as much of it as is possible to do. When I send an email to someone, it goes from my client to my server and out to the recipient's server.
I genuinely think some people are not aware you can have free, reliable, outgoing SMTP providers.
And you can have an additional route just for some recipients, that doesn't make it less (or more) selfhosting.
It seems to me that you are proposing exactly this: routing all emails through one of these companies. How can this be a solution to "self-hosting"?
Sure, it's a never ending battle with the behemoths but there are options and the most beautiful thing is what most of built-in in the SMTP itself.
It's just not worth the overhead in people and hardware to try to be surgical in your approach - you don't win anything. It's the closest thing to working at the DMV for tech.
Notes, cc:Mail, Groupwise, Exchange, Sendmail, Exim, qmail, postfix, etc... I have no interest in running my own email ever again.
The article is not about blackholing spam sources. It's about blackholing mail from sources that have never sent a single spam message since the dawn of the internet.
By definition, every new source has never sent spam -- but it's reasonable to assume that an unknown source is likely spam, however unfortunate that may be.
Uh, yeah, you can, I did it again recently. I've done it at Oracle Cloud, AWS Lightsail, Linode, GoDaddy, and several ISP's before that. I could start right this minute with Digital Ocean, Vultr, etc. pick one and be up and running, delivering email to Gmail, Microsoft, etc. in a day.
Each time one of these articles is posted, I feel I must be some kind of email savant.
- Setup blacklist monitoring (e.g. HetrixTools/MXToolbox)
- Check if you can email Gmail
- Check if you can email Hotmail
- Check if you can email Office365
- Check on Microsoft SNDS that you are not blocked: https://sendersupport.olc.protection.outlook.com/snds/
Gmail, Hotmail and Office365 are the largest email providers and also the most strict ones.
I have accounts on all of these providers so testing deliverability is trivial. You could argue that testing deliverability with one account is unreliable but in my experience it simply is not. Usually you send a couple emails across a week and if they all go through you are good.
If you are paranoid, you can ask if any of your friends have Office365/Hotmail and email them. They probably have company or university accounts on there.
My server has only been blocked once after all this, and ironically it was by another company that self-hosts their email...
We read from different cookbooks and my reading comprehension is high. My sent email stats shows over 20K successfully sent emails just for my personal account, since 1999, when I started tracking it.
I feel like for every pro host-your-own-mail on HN there is one that contradicts it.
After many hours of this over many months, I finally looked for a comparison of the actual taste of coffee from cheap grinders versus 10-20x more expensive burr grinders. (As opposed to comparisons of grinder technologies, which are everywhere.)
There are almost no published side-by-side taste comparisons; when I did find one, the cheap grinders had won!
Setting up a server is trivial, yes. Keeping it going is a never-ending treadmill of not really technical problems.
Not magic, but when I managed outgoing Postfix servers for a few companies I had to set rate limits for yahoo.com an a couple other domains to reduce concurrency or they would block one of the SNAT's for a while. It probably sounds tedious but it really wasn't. There were not many MX that were as strict as Yahoo. I never ran into issues with Gmail but I think they cut some slack for corporate IP addresses and domain names.
For my own personal email servers I never had issues because I never sent at a rate that anyone cared about. The closest I got to that was running a forum that would email when threads would get updated and people subscribed to them but my solution there was to suggest to the people on the forum not to do that.
I can't see the future, if the big email providers who likely have some of their trolls posting in the comments ever decide to start choking out us personal email server runners, then it'd be game over. If things remain for the next 20 years assuming I live that long then deliverability would be 100% for the next 20 years.
> Do you know some magic incantation for staying off the naughty lists?
I don't spam, that and don't make a finger fumble edit like I did the one time in over 20 years and didn't check to make sure it was working correctly first.
Also, I use http://www.mxtoolbox.com/ to keep an eye on my server.
Put up an ASK HN and see what happens.
The cases where my email got blocked, I always received an email back from the box, so I think there are no cases where my email just disappeared into the dark. Reasons for blocking where: - Some strange spamlist blocked my IP. I was able to resolve this by filling out a i-am-not-a-spammer-form on the spamlist provider - Some email server required from me that I add my personal address to my website, so that they know who maintains the server
Never have I send an email but it was not delivered without notice
It's times like this that I wish HN had better moderation, you are insultingly rude to the parent and to one of my comments.
The GOP sued Google for blocking their campaign e-mails and lost.
These cries of protest were just more ridiculous outrage fuel, from a party that loves portraying themselves as some kind of victim.