Ask HN: What do you self-host?

loki + cadvisor + node-exporter + grafana + prometheus syncthing tinc vpn server dokuwiki firefox-sync firefox-send vscode server bitwarden freshrss znc bouncer + lounge irc client + bitlbee an httptunnel server (like ngrok) firefly iii monicahq kanboard radicale syncthing wallabag tmate-server

“Self-host” is such a weird word. Having your own stuff yourself should be the default, should it not? I mean, you don’t “self-drive” your car, nor “self-work” your job. The corresponding words instead exists for the opposites: You can have a chauffeur and you can outsource your job.

I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then have your own server at home, you are forced to have it elsewhere with sensible internet connections.

It’s as if, in an alternate reality, no private residences had parking space for cars; no garages, no street parking. Everyone would be forced to either use public transport, taxis and chauffeur services to get anywhere. Having a private vehicle would be an expensive hobby for the rich and/or enthusiasts, just like having a personal server is in our world.

IanCal · 6 years ago

Hmm. I get other people to build my car, grow my food, generate my electricity, extract and refine my petrol, clean my water, I've rented cars, I get others to fly planes for me that I don't own, I use trains others drive and own.

I do everything for little to nothing in my life, and there's no reasonable default as to where the line is other than a cost/benefit comparison.

For my for many years owning a car was far more expensive than renting or getting taxis when needed. Owning a car absolutely would have been an expensive hobby, and the same is true for many in cities.

Having a personal server is exceptionally cheap. I had a VPS unnoticed recently I'd forgotten to cancel which cost about 10 dollars per year. That's about one minimum wage hour where I live. If you mean literally a personal server a raspberry pi can easily run a bunch of things and can cost about the same as a one off.

It's time, and upfront costs of software. If I want updates, and I do (security at least) I need some ongoing payments for those, and then I need to manage a machine. That management is better done by people other than me (as even if they earned the same as me they'd be faster and better) and they can manage more machines without a linear increase in their time.

So why self host? Sometimes it'll make sense, but the idea it should be the default to me doesn't hold. Little needs to be 100% in house, and sharing things can often be far more efficient. Software just happens to be incredibly easy to share.

teddyh · 6 years ago

> So why self host? Sometimes it'll make sense, but the idea it should be the default to me doesn't hold.

You can’t outsource your privacy. Once you’ve given your information to a third party, that third party can and will probably use it as much as they can get away with. And legal protection from unreasonable search and seizure is also much weaker once you’ve already given out your information to a third party.

To generalize, and to also answer your other comments in a more general sense, you can’t outsource your freedom or civic responsibility. If you do, you turn yourself into a serf; someone with no recourse when those whom you place your trust in ultimately betray you.

(Also, just like “owning” a timeshare is not like owning your house, having a VPS is not self-hosting.)

rovr138 · 6 years ago

> I had a VPS unnoticed recently I’d forgotten to cancel

I found on my Linode account one last weekend. It’s been up since 2010 running Debian 5, no updates cause the repos are archived. Couple of PHP sites on there which I don’t control the domains of (but the sites where active).

Last email I have from the people there is 2012, a backup. The company apparently is not in business anymore (I know the domains registrar was on the personal account of the owner. He might have have auto renew on).

Backed up everything there and shut it down.

bhauer · 6 years ago

> I think the problem is entirely caused by the US having absolutely abysmal private internet speeds and capacity. Since you can’t then have your own server at home, you are forced to have it elsewhere with sensible internet connections.

The trend definitely traces to the advent and eventual domination of asymmetric Internet connectivity. My first DSL connection was symmetric, so peer-to-peer networking and running servers ("self-hosting") were just natural. Since then, asymmetric bandwidth has ruled the US.

It's not so much that connectivity technology in the US is strictly poor—many cities have options providing hundreds of megabits or a gigabit or more of aggregate bandwidth. It's that the capacity allocation of some shared delivery platforms (e.g., cable) is dramatically biased toward download/consumption, and against upload/share/host. And there's no way for consumers to opt for a different balance. I'd gladly take 500/500 versus 1000/50. Even business accounts, which for their greatly increased costs are a refuge of symmetric connectivity and static IPs, are more commonly asymmetric today.

I think that this capacity imbalance and bias toward consumption snowballs and reinforces the broader assumptions of consumption at the edge (why make a product you self-host when most people don't have the proper connectivity?). This in turn means more centralization of services, applications, and data.

Nevertheless, even with mediocre upload speeds (measured in mere tens of megabits), I insist on self-hosting data and applications as much as I can muster. All of my devices are on my VPN (using the original notion of "VPN," meaning quite literally a virtual private network; not the more modern use of VPN to mean "encrypted tunnel to an Internet browsing egress node located in a data center"). For example, why would I use Dropbox when I can just access my network file system from anywhere? To me, it's a matter of simplicity. Everything I use understands a simple file system.

maxerickson · 6 years ago

If you take a broader lens, having a private vehicle is an expensive hobby for the rich.

And most people actually do outsource their jobs. They are employees rather than working for themselves…

avl999 · 6 years ago

> If you take a broader lens, having a private vehicle is an expensive hobby for the rich.

That might be true if you are in SF, NY, Toronto, London or some other major metropolitan with a good public transportation network. However for a large number of places in North America including metropolitans like LA, San Diego, Minneapolis, Dallas, having a car is almost as necessary as anything as that is the only way to get around the city without spending half a day in public transit.

ekianjo · 6 years ago

> having a private vehicle is an expensive hobby for the rich.

Having a car is not a hobby when you live outside of a very dense city center. That's just the tool that enables you to live.

tbrownaw · 6 years ago

I tend to think of "expensive hobby" as meaning you do it for fun rather than for practical reasons.

While I know that some car owners do just have it for fun, I think a lot more are because it's useful.

dillonmckay · 6 years ago

So, that would be interesting to note who here is in the EU self-hosting w/ their symmetric, low-cost, high-speed ISPs, versus the US, paying $600/mo for a 5 year contract for 10/10 Mbit DIA setup (anecdote).

stiray · 6 years ago

I am paying 76 euros/month for 500/100 (This is max achievable, with latency at around 6ms, after replacing their crappy router (12ms+) with mikrotik, throughput can be lower but mostly it is throttled by source) fiber connection + 1 phone (50gb download, LTE, with 80% of country coverage) + max iptv scheme with HBO + static ipv4 ip and reverse resolve. I would love to hear what the prices are around the world.

(edit: forgot to state country, Slovenia)

kraftman · 6 years ago

Reminds me of "wild" camping. Used to just be called camping...

kleer001 · 6 years ago

>> Everyone would be forced to either use public transport, taxis and chauffeur services to get anywhere.

Saudi is like this, I hear, Jakarta too. I assume there's more.

ianthiel · 6 years ago

"self-driving" ones car may become the common parlance before we die

teddyh · 6 years ago

“Where can I get a hire car? Self-drive.”

“No self-drive. Only taxis.”

— The Prisoner, 1967

* nginx to reverse-proxy each of the services. * NextCloud. * Matrix Homeserver (synapse). * My website (dumb Flask webapp). * Tor (non-exit) relay. * Tor onion service for my website. * Wireguard VPN (not running in a container, obviously).

nginx vCenter urbackup UniFi SDN, Protect Portainer / unms / Bitwarden Wordpress (isolated) Guacamole PiHole InfluxDB / grafana Active Directory Windows 10 VM for Java things L2TP on my router

On my home server (refurbished ThinkPad X201 with a Core i5-520M, 8GB of memory, 1TB internal SSD sync'd nightly to an external 1TB HDD) I run a single-node Kubernetes cluster with the following stuff:

* MinIO: for access to my storage over the S3 API, I use it with restic for device backups and to share files with friends and family

* CoreDNS: DNS cache with blacklisted domains (like Pihole), gives DNS-over-TLS to the home network and to my phone when I'm outside

* A backup of my S3-hosted sites, just in case (bejarano.io, blog.bejarano.io, mta-sts.bejarano.io and prefers-color-scheme.bejarano.io)

* https://ideas.bejarano.io, a simple "pick-one-at-random" site for 20,000 startup ideas (https://news.ycombinator.com/item?id=21112345)

* MediaWiki instance for systems administration stuff

* An internal (only accessible from my home network) picture gallery for family pictures

* TeamSpeak server

* Cron jobs: dynamic DNS, updating the domain blacklist nightly, recursively checking my websites for broken links, keeping an eye on any new release of a bunch of software packages I use

* Prometheus stack + a bunch of exporters for all the stuff above

* IPsec/L2TP VPN for remote access to internal services (picture gallery and Prometheus)

* And a bunch of internal Kubernetes stuff for monitoring and such

I still have to figure out log aggregation (probably going to use fluentd), I want to add some web-based automation framework like NodeRED or n8n.io for random stuff. I'd also like to host some password manager but I still have to study that.

I also plan on rewriting wormhol.org into supporting any S3 backend, so that I can bind it's storage with MinIO.

And finally, I'd like to move off single-disk storage and get a decent RAID solution to provide NFS for my cluster, as well as a couple more nodes to add redundancy and more compute.

Edit: formatting.

whycombagator · 6 years ago

> * CoreDNS: DNS cache with blacklisted domains (like Pihole), gives DNS-over-TLS to the home network and to my phone when I'm outside

I would be _very_ interested in a write up/explanation of this set up

ricardbejarano · 6 years ago

There you go!

Essentially, this setup achieves 5 features I wanted my DNS to have:

- Confidentiality: from my ISP; and from anyone listening to the air for plain-text DNS questions when I'm on public WiFi. Solution: DNS-over-TLS[1]

- Integrity: of the answers I get. Solution: DNS-over-TLS authenticates the server

- Privacy: from web trackers, ads, etc. Solution: domain name blacklist

- Speed: as in, fast resolution times. Solution: caching and cache prefetching[2]

- Observability: my previous DNS was Dnsmasq[3], AFAIK Dnsmasq doesn't log requests, only gives a couple stats[4], etc. Solution: a Prometheus endpoint

CoreDNS ticks all of the above, and a couple others I found interesting to have.

To set it up, I wrote my own (better) CoreDNS Docker image[7] to run on my Kubernetes cluster; mounted my Corefile[8] and my certificates as volumes, and exposed it via a Kubernetes Service.

The Corefile[8] essentially sets up CoreDNS to:

- Log all requests and errors

- Forward DNS questions to Cloudflare's DNS-over-TLS servers

- Cache questions for min(TTL, 24h), prefetching any domains requested more than 5 times over the last 10 minutes before they expire

- If a domain resolves to more than one address, it automatically round-robins between them to distribute load

- Serve Prometheus-style metrics on 9153/TCP, and provide readiness and liveness checks for Kubernetes

- Load the /etc/hosts.blacklist hosts file (which has just short of 1M domains resolved to 0.0.0.0), reloads it every hour, and does not provide reverse lookups for performance reasons

- Listens on 53/UDP for regular plain-text DNS questions (LAN only), and on 853/TCP for DNS-over-TLS questions, which I have NAT'd so that I can use it when I'm outside

The domain blacklist I generate nightly with a Kubernetes CronJob that runs a Bash script[9]. It essentially pulls and deduplicates the domains in the "safe to use" domain blacklists compiled by https://firebog.net/, as well as removing (whitelisting) a couple hosts at the end.

That's pretty much it. The only downside to this set up is that CoreDNS takes just short of 400MiB of memory (I guess it keeps the resolve table on memory, but 400MiB!?) and lately I'm seeing some OOM restarts by Kubernetes, as it surpasses the 500MiB hard memory limit I have on it. A possible solution might be to keep the resolve table on Redis, which might take up less memory space, but I'm still to try that out.

[1] Which I find MUCH superior to DNS-over-HTTPS. The latter is simply a L7 hack to speed up adoption, but the correct technical solution is DoT, and operating systems should already support it by now (AFAIK, the only OS that supports DoT natively is Android 9+).

[2] It was when I discovered CoreDNS' cache prefetching that I convinced myself to switch to CoreDNS.

[3] http://www.thekelleys.org.uk/dnsmasq/doc.html

[4] It gives you very few stats. I also had to write my own Prometheus expoter[5] because Google's[6] had a fatal flaw and no one answered to the issue. In fact, they closed the Issues tab on GitHub a couple months after my request, so fuck you, Google!

[5] https://github.com/ricardbejarano/dnsmasq_exporter

[6] https://github.com/google/dnsmasq_exporter (as you can see the Issues tab is no longer present)

[7] https://github.com/ricardbejarano/coredns, less bloat than the official image, runs as non-root user, auditable build pipeline, compiled from source during build time. These are all nice to have and to comply with my non-root PodSecurityPolicy. I also like to run my own images just so that I know what's under the hood.

[8]

  local:65535 {
    ready
    health
  }

  (global) {
    log
    errors

    cache 86400 {
      prefetch 5 10m 10%
    }
    dnssec
    loadbalance

    prometheus :9153
  }

  (cloudflare) {
    forward . tls://1.1.1.1 tls://1.0.0.1 {
      tls_servername cloudflare-dns.com
    }
  }

  (blacklist) {
    hosts /etc/hosts.blacklist {
      reload 3600s
      no_reverse
      fallthrough
    }
  }

  .:53 {
    import global
    import blacklist
    import cloudflare
  }

  tls://.:853 {
    import global
    import blacklist
    import cloudflare
    tls /etc/tls/fullchain.pem /etc/tls/privkey.pem
  }

[9]

  #!/bin/bash

  HOSTS_FILE="/tmp/hosts.blacklist"
  HOSTS_FILES="$HOSTS_FILE.d"

  mkdir -p "$HOSTS_FILES"
  download() {
    echo "download($1)"
    curl \
      --location --max-redirs 3 \
      --max-time 20 --retry 3 --retry-delay 0 --retry-max-time 60 \
      "$1" > "$(mktemp "$HOSTS_FILES"/XXXXXX)"
  }

  # https://firebog.net/
  ## suspicious domains
  download "https://hosts-file.net/grm.txt"
  download "https://reddestdream.github.io/Projects/MinimalHosts/etc/MinimalHostsBlocker/minimalhosts"
  download "https://raw.githubusercontent.com/StevenBlack/hosts/master/data/KADhosts/hosts"
  download "https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Spam/hosts"
  download "https://v.firebog.net/hosts/static/w3kbl.txt"
  ## advertising domains
  download "https://adaway.org/hosts.txt"
  download "https://v.firebog.net/hosts/AdguardDNS.txt"
  download "https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt"
  download "https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt"
  download "https://hosts-file.net/ad_servers.txt"
  download "https://v.firebog.net/hosts/Easylist.txt"
  download "https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts;showintro=0"
  download "https://raw.githubusercontent.com/StevenBlack/hosts/master/data/UncheckyAds/hosts"
  download "https://www.squidblacklist.org/downloads/dg-ads.acl"
  ## tracking & telemetry domains
  download "https://v.firebog.net/hosts/Easyprivacy.txt"
  download "https://v.firebog.net/hosts/Prigent-Ads.txt"
  download "https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt"
  download "https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.2o7Net/hosts"
  download "https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt"
  ## malicious domains
  download "https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt"
  download "https://mirror1.malwaredomains.com/files/justdomains"
  download "https://hosts-file.net/exp.txt"
  download "https://hosts-file.net/emd.txt"
  download "https://hosts-file.net/psh.txt"
  download "https://mirror.cedia.org.ec/malwaredomains/immortal_domains.txt"
  download "https://www.malwaredomainlist.com/hostslist/hosts.txt"
  download "https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt"
  download "https://v.firebog.net/hosts/Prigent-Malware.txt"
  download "https://v.firebog.net/hosts/Prigent-Phishing.txt"
  download "https://phishing.army/download/phishing_army_blocklist_extended.txt"
  download "https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt"
  download "https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt"
  download "https://ransomwaretracker.abuse.ch/downloads/CW_C2_DOMBL.txt"
  download "https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt"
  download "https://ransomwaretracker.abuse.ch/downloads/TC_C2_DOMBL.txt"
  download "https://ransomwaretracker.abuse.ch/downloads/TL_C2_DOMBL.txt"
  download "https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist"
  download "https://v.firebog.net/hosts/Shalla-mal.txt"
  download "https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Risk/hosts"
  download "https://www.squidblacklist.org/downloads/dg-malicious.acl"

  cat "$HOSTS_FILES"/* | \
  sed \
    -e 's/0.0.0.0//g' \
    -e 's/127.0.0.1//g' \
    -e '/255.255.255.255/d' \
    -e '/::/d' \
    -e '/#/d' \
    -e 's/ //g' \
    -e 's/  //g' \
    -e '/^$/d' \
    -e 's/^/0.0.0.0 /g' | \
  awk '!a[$0]++' | \
  sed \
    -e '/gamovideo.com/d' \
    -e '/openload.co/d' > "$HOSTS_FILE"

  rm -rf "$HOSTS_FILES"

devthane · 6 years ago

You might take a look at https://github.com/grafana/loki if you haven’t seen it yet for logs. It’s still really new but it’s been working for me great.

ricardbejarano · 6 years ago

Thanks for the heads up, I'll check it out!

bluegreyred · 6 years ago

nice to see somebody using a thinkpad as a homeserver.

I remember comparing low power homeservers, consumer NAS and a refurb Thinkpad and the latter won when considering the price/performance and idle power consumption (<5W). You also get a built screen & keyboard for debugging and a efficient DC-UPS if you're brave enough to leave the batteries in. That's of course assuming you don't need multiple terabytes of storage or run programs that load the CPU 24/7, which I don't. These days a rPi 4 would probably suffice for my needs but I still think the refurb thinkpad is a smart idea.

ricardbejarano · 6 years ago

I don't overload the CPU and my storage requirements are low. 95% of my used storage is stuff I wouldn't care if it got lost, but just nice to have around. I only have around 2GB of data I don't want to lose.

I do leave the batteries in. Is it dangerous? I read some time ago that it is not dangerous, but the capacity of the battery drops significantly, I don't care about capacity, and safe shutdowns are important to me.

In the past I used an HP DL380 Gen. 7 (which I still own, and wouldn't mind selling as I don't use it), but I had to find a solution for the noise. And power consumption was at around 18EUR for my EUR/kWh.

Cramming down what ran on 12 cores and 48GiB of RAM on a 2-core, 4GiB (I only upgraded the memory 2 months ago) machine was a real challenge.

The ThinkPad cost me 90EUR (IBM refurbished), we bought two of them, the other one burnt. The recent upgrades (8GiB kit + Samsung Evo 1TB) cost me around 150EUR. Overall a really nice value both in compute per EUR spent and in compute per Wh spent. Really happy with it, I just feel it is not very reliable as it is old.

Havoc · 6 years ago

Using your previous generation gear as server when upgrading works very well too. Even a 5 year old i5 or whatever has plenty left for server duty

bpye · 6 years ago

Hey - I'm not the only one using CoreDNS like this! I'm just abusing the hosts plugin - do you have something more elegant?

ricardbejarano · 6 years ago

I'm throwing it the hosts blacklist as a file. For performance reasons I turn off reverse lookups and limit reloading to once every hour:

    (blacklist) {
      hosts /etc/hosts.blacklist {
        reload 3600s
        no_reverse
        fallthrough
      }
    }

    .:53 {
      import blacklist

      ... (more config)
    }

jamieweb · 6 years ago

Good job for using MTA-STS. :)

mpnordland · 6 years ago

I've been wanting to do a kubernetes setup with my home server, but most tutorials are aimed at multi node clusters. Do you have any links on how to setup such a system?

gravypod · 6 years ago

If you follow a multi node cluster tutorial all you should need to do is remove the master taint from the node and normal pods will spawn on it

lbotos · 6 years ago

I'm trialling k3s right now. https://k3s.io/

magicfractal · 6 years ago

Could you share a bit about your picture gallery?

ricardbejarano · 6 years ago

Sure! Here's the source code: https://github.com/ricardbejarano/pyctures

I could set up a demo if you want to.

It's a cheap Flask app that scans a given "library" directory for "album" subdirectories, which contain the pictures you want to display.

It has a big issue with image size (16 images per page, my phone takes 5MB pictures, 80MB per page is HUUUGE). Thumbnailing would be great. I'm open for PRs ;)!

If anyone knows about a better alternative... I set this up when we got back from one vacation for my relatives to easily see the pictures (without social media).

thequailman · 6 years ago

How are you doing multi tenant MinIO?

ricardbejarano · 6 years ago

I don't :)

Right now I have public (read-only) and private buckets only, and I'm the only who writes into any of them.

Public buckets contain files I didn't even create myself and that friends might find useful (Windows ISO, movies, VirtualBox VMs...). Privates have, well, private data, and can only be accessed using my admin account's credentials.

IIRC MinIO has access control through users, but I'm still very new to MinIO to the point where I discover new features every time I use it.

If I were to give someone else their own buckets I'd probably run a second instance to keep things separate, though. I'm even considering running another one myself to keep private buckets only accessible from my home network... (right now the entire instance is reachable from WAN, regardless of whether they are public or not).

mavidser · 6 years ago

I reworked my servers a while ago to host literally everything through docker, managed via terraform.

All web-services are reverse-proxied through traefik

At home:

    loki + cadvisor + node-exporter + grafana + prometheus
    syncthing
    tinc vpn server
    jackett + radarr + sonarr + transmission
    jellyfin
    samba server
    calibre server

On a remote server:

tnsittpsif · 6 years ago

How much do you spend on the remote server on a monthly basis? Also, what's the hardware you use for the home server?

Remote server's a 20USD/month DigitalOcean droplet with 4GB memory. Though even half of that would also have specified for these services.

Home server's a Raspberry Pi 4.

nerdponx · 6 years ago

Was it hard to set up Firefox-Sync/Send? Last I checked, self-hosting these was undocumented and difficult.

Sync's docs were a bit outdated, yes. Had to spend some time to get it up and running.

The config I ended up using - https://0bin.net/paste/gnWY4+Tn-jZ2UMZm#RgQfZ3uD7MIlK7nWKLLX...

It's deployed on docker, proxied through traefik.

sdan · 6 years ago

It's not hard when you use Docker. Just spin it up and then use Traefik to route.

dmos62 · 6 years ago

I see you're using Bitwarden.

Does anyone have recommendations for password+sensitive-data management?

I'm currently using Keepass and git, but I have one big qualm. You cannot choose to not version-control that one big encrypted (un-diff-able) file.

johntash · 6 years ago

You might like Pass [0] or GoPass [1] which had more features the last I looked at it.

They both store passwords/data in gpg-encrypted files in a git repo. I'm not sure what the state of GUIs/browser plugins are for it, but I'm pretty sure there are some out there.

You can also set up your git config to be able to diff encrypted .gpg files so that the files are diff-able even though they're encrypted.

[0]: https://www.passwordstore.org/

[1]: https://github.com/gopasspw/gopass

monotux · 6 years ago

Bitwarden can be self-hosted and it's server is open source (and security audited, for what it's worth). I've used it for a few years or so and I've had no issues this far.

One other alternative to keepass is pass[1].

[1]: https://www.passwordstore.org/

erulabs · 6 years ago

Vault or Bitwarden are great for projects once they get serious - Unfortunately there isn't a one-size-fits all solution that doesn't suck in one way or another. Setting up vault is fairly non-trivial.

captn3m0 · 6 years ago

Or stacks look so similar, it’s creepy. Thankfully, not running sync thing now.

Yeah, I too have noticed that. Haven't seen a lot of terraform usage for personal services.

What are the issues with syncthing?

drakenot · 6 years ago

What do you use instead?

emit_time · 6 years ago

Have you considered moving from tinc to Wireguard?

Yes, I've been meaning to give it a go for a while now. Couldn't use it initially because of (then) lack of availability on BSD.

masterfooo · 6 years ago

I use both, and one thing I found that is sucky about WG is that it does not work well with the Windows firewall. I need to give full permission to an app to be able to access ip addresses routed by WG. Tinc does not have this problem.

How do you like bitlbee?

cyphar · 6 years ago

I self-host the following at home. Everything is running under LXD (and I have all of the scripts to set it up here[1]):

All running on an openSUSE Leap box, with ZFS as the filesystem for my drives (simple stripe over 2-way mirrors of 4TB drives).

It also acts as an NFS server for my media center (Kodi -- though I really am not a huge fan of LibreELEC) to pull videos, music, and audiobooks from. Backups are done using restic (and ZFS snapshots to ensure they're atomic) and are pushed to BackBlaze B2.

I used to run an IRC bouncer but Matrix fills that need these days. I might end up running my own Gitea (or gitweb) server one day though -- I don't really like that I host everything on GitHub. I have considered hosting my own email server, but since this is all done from a home ISP connection that probably isn't such a brilliant idea. I just use Mailbox.org.

[1]: https://github.com/cyphar/cyphar.com/tree/master/srv

douglascoding · 6 years ago

> * Wireguard VPN (not running in a container, obviously).

I plan to use Wireguard too, so I shouldn't run on containers? Can you elaborate on that?

BrandoElFollito · 6 years ago

From the small research I did, I think you need a customized kernel on the host to do that.

I run it on the host.

mwcampbell · 6 years ago

> It also acts as an NFS server for my media center [...] to pull videos, music, and audiobooks from.

This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi? Do you also have a way of syncing them to a more portable device, like your phone?

> This is a bit tangential, but to clarify, do you mean that you listen to audiobooks on your TV using Kodi?

Sometimes, though not very often -- I work from home and so sometimes I'll play an audiobook in my living room and work at the dinner table rather than working from my home office.

> Do you also have a way of syncing them to a more portable device, like your phone?

Unfortunately not in an automated way (luckily I don't buy audiobooks very regularly -- I like to finish one before I get another one). I really wish that VLC on Android supported NFS, but it doesn't AFAIK (I think it requires kernel support).

big_chungus · 6 years ago

Why SUSE over another OS? I've used it and like it, though I see more ubuntu, debian, centos among servers. Any particular distinguishing factor/advantage, or just preference?

I've worked for SUSE for quite a few years now, and so I've gotten fairly used to running openSUSE on all my machines (and I do quite like things like the Open Build Service and other openSUSE projects). I'm am a package maintainer for a bunch of openSUSE packages (most of the container-related ones and a few others) -- so I might as well use them myself to make sure they work properly.

I'm curious about why you're using lxd. Is it just that you wanted to try something different from Docker and its rivals? Or is there a reason you think lxd is better for your setup? For a service per container, I figured minimal, immutable containers, rather than containers running full distros, would be better.

The primary reason is that LXD has an indisputably better overall security policy than Docker. They support isolated user namespaces (containers running with different userns mappings), user namespaces are the default, they make use of far more new kernel hardening features than Docker, and so on. If I'm going to self-host something at home and expose it to the internet, I'm simply not going to use Docker.

I used to run Docker containers several years ago, but I found them far more frustrating to manage. --restart policies were fairly hairy to make sure they actually worked properly, the whole "link" system in Docker is pretty frustrating to use, docker-compose has a laundry-list of problems, and so on. With LXD I have a fairly resilient setup that just requires a few proxy devices to link services together, and boot.autostart always works.

Personally, I also find it much simpler to manage a couple of services as full-distro containers. Having to maintain your own Dockerfiles to work around bugs (and missteps) in the "official library" Docker images also added a bunch of senseless headaches. I just have a few scripts that will auto-set up a new LXD container using my configuration -- so I can throw away and recreate any one of my LXD containers.

[Note: I do actually maintain runc -- which is the runtime underneath Docker -- and I've contributed to Docker a fair bit in the past. So all of the above is a bit more than just uneducated conjecture.]

I host a bunch of docker containers plus Traefik to route everything. It runs on a cheap GCP instance (more on this here: https://sdan.xyz/sd2)

Overleaf: https://sdan.xyz/latex

A URL Shortener: https://sdan.xyz

All my websites (https://sdan.xyz/drf, https://sdan.xyz/surya, etc.)

My blog(s) (https://sdan.xyz/blog, https://sdan.xyz/essays)

Commento commenting server (I don't like disqus)

Monitoring (https://sdan.xyz/monitoring, etc.)

Analytics (using Fathom Analytics) and some more stuff!

djsumdog · 6 years ago

I run netdata too, but I keep that behind my VPN. I'd suggest the same for you. No reason to have that exposed to the entire world.

I wrote this to setup my web server, mail server and VPN server, and auto-generate all my VPN keys.

https://github.com/sumdog/bee2

You're 100% right. Actually was a bit concerned myself when I realized hundreds of people were peering into how my server is doing.

But at the same time, I understand the security risks and if I have to I can just stop netdata's container and add some more security on it before turning it on again (I'm not running some SaaS startup, so security isn't a huge concern and I don't think you can do anything with my netdata that can affect or show anything else that can make me prone to attack)

Any reason to have it behind a VPN?

RulerOf · 6 years ago

It’s a relatively popular choice but I’ll ask you about it...

I see a lot of people putting their home stuff behind CloudFlare, but when I reviewed their free tier, I didn’t actually see any security benefit to outweigh the privacy loss, and I didn’t see that covered on your blog post.

tbyehl · 6 years ago

> I didn’t actually see any security benefit to outweigh the privacy loss

The main thing is being able to hide your origin IP address. That turns many types of DDoS attacks into CloudFlare's problem, not yours, and it doesn't matter that you're on the free tier[0]. If you firewall to only allow traffic from CF[1], then you can make your services invisible to IP-based port scans / Shodan.

CloudFlare isn't a magic-bullet for security, but, used correctly, they greatly reduce the attack surface.

Whether any of that is worth the privacy / security risk of letting CloudFlare MITM your traffic is up to you.

[0] https://news.ycombinator.com/item?id=21170847

[1] https://www.cloudflare.com/ips/

Thanks for the read!

1. This is hosted on GCP. Actually was thinking of using Cloudflare Argo once my GCP credits expire so that I can truly self host all this (although all I have is an old machine).

2. For me, Cloudflare makes my websites load faster on pages. Security wise, I have pretty much everything enabled... like always on HTTPS, etc. and I some strict restrictions on SSHing into my instance (also note that none of my ip addresses are exposed thanks to Cloudflare), so really I'm not sure what security risk there may be.

3. How am I losing privacy loss? Just curious, not really understanding what you're saying there.

pm7 · 6 years ago

You may want to consider adding netdata user to docker group. It will allow checking Docker names of containers instead of numeric id.

Of course, it would simplify privilege escalation if someone successfully attack netdata service. If you want public dashboard, streaming is supposed to be quite safe (no way to send instruction to streaming instance of netdata).

_emacsomancer_ · 6 years ago

how difficult is overleaf to self host?

Super easy, here's the docker compose file I used:

https://github.com/dantuluri/sd2/blob/master/docker-compose....

You'll need Mongo and Redis (last I remember) as well (which I believe are the two images that follow the sharelatex image.

whalesalad · 6 years ago

Bums me out when I see people putting so many resources into running/building elaborate piracy machines. Plex, radarr, sonarr, etc... (you note some of these services but /r/homelab is notorious for this)

Here’s my home lab: https://imgur.com/a/aOAmGq8

I don’t self host anything of value. It’s not cost effective and network performance isn’t the best. Google handles my mail. GitHub can’t be beat. I use Trello and Notion for tracking knowledge and work, whether personal or professional. Anything else is on AWS. I do have a VPN though so I can access all of this when I’m not home.

The NAS is for backing up critical data. R720 was bought to experiment with Amazon Firecracker. It’s usually off at this point. Was running ESXI, now running Windows Server evaluation.

The desktop on the left is the new toy. I’m learning AD and immersing myself 100% in the Microsoft stack. Currently getting an idiomatic hybrid local/azure/o365 setup going. The worst part about planning a MS deployment is having to account for software licensing that is done on a per-cpu-core basis.

Marsymars · 6 years ago

It bums me out when I see corporations putting so many resources into monopolizing copyright and preventing media from entering the public domain, which leads to consumers putting resources into purchasing media that would otherwise be in the public domain.

The status quo is radically anti-consumer, IMO, as radical as abolition of all copyright would be.

zanny · 6 years ago

It more generally burns me out that we as a society still feel it is necessary to construct and reinforce so arbitrary an apparatus as copyright to substantially stymie the tremendous potential information exchange of computer networks.

Of all the ways to try to promote creativity in the 21st century, making information distribution illegal by default and then using force of law to restrict said distribution unless authorized is pretty wack.

LeoPanthera · 6 years ago

I use Plex. It virtually exclusively contains rips of blu-rays and DVDs that I have bought. I do not consider this piracy. I do not think format shifting is unethical. I do think DRM is unethical.

cannonedhamster · 6 years ago

Listen here, just because you bought a piece of plastic doesn't mean you own what's on the piece of plastic. It's like a car, just because you bought a car doesn't mean you own the steering wheel it the seats.... Oh wait.../s

esotericn · 6 years ago

> The worst part about planning a MS deployment is having to account for software licensing that is done on a per-cpu-core basis.

> Bums me out when I see people putting so many resources into running/building elaborate piracy machines.

These two comments are rather at odds to me.

That said, IME generally the type of person who's big into self hosting isn't a Microsoft guy. I work with MS stuff at work at the moment. The entire thing is set up for Enterprise and Regulations. It's hugely overcomplicated for that specific goal only.

At home I don't care about Regulations(tm). The only reason I can see for someone to bother with it is if they want to train out of hours for a job at an MS shop.

input_sh · 6 years ago

As soon as I find a DRM-free way to purchase my shows/movies/books, I'll be glad to do so. Until then, yarr.

ptman · 6 years ago

There are DRM-free ebooks available. https://www.defectivebydesign.org/guide/ebooks (and there seems to be other stuff as well: https://www.defectivebydesign.org/guide/audio ). But I mostly agree with you. I hope watermarks would be used instead of DRM.

saagarjha · 6 years ago

Plex can and is often used for hosting content that you own the rights to.

DominoTree · 6 years ago

I have well over 2,000 publicly-available cybersecurity talks on my Plex and I'm currently watching one right now.

I also have piracy.

Sure, in the same way that BitTorrent can be used to download Linux ISOs :)

aklemm · 6 years ago

Not to be rude, but you are very much cozied up to “the man” ya know?

shakna · 6 years ago

How would _you_ suggest I handle the 2TB of public domain media I have, then?

gcj · 6 years ago

I'm sorry to bum you out, but I recently built a Raspberry Pi piracy box and it's amazing :D

Tell me about the wood shelving. Did you build that?

It seems to hold rack-mounted gear quite well.

e15ctr0n · 6 years ago

> Tell me about the wood shelving.

Looks like the IKEA IVAR storage system. https://www.ikea.com/kr/en/catalog/categories/departments/li...

futhey · 6 years ago

Oh man, I have the same shelf from Ikea, never thought of using it for a couple rack-mountables. I like the look!

I was wondering about that shelf. It is the perfect width.

Deleted Comment

In colo:

  nginx
  Plex
  Radarr / Sonarr / SABnzbd / qBittorrent / ZeroTier -> online.net server
  FreeNAS x2
  Active Directory

Everything I expose to the world goes through CloudFlare and nginx with Authenticated Origin Pulls [0], firewalled to CF's IPs [1], and forced SSL using CF's self-signed certs. I'm invisible to Shodan / port scans.

Have been meaning to move more to colo, especially my Wordpress install and some Wordpress.com-hosted sites, but inertia.

[0] https://support.cloudflare.com/hc/en-us/articles/204899617-A...

NewDimension · 6 years ago

Do you have a static IP at home? How does your cloudflare setup work?

I've done similar. You firewall your home network to all IP's other than Cloudflare's. You can use a Cloudflare provided certificate for HTTPS - they will MITM and use a trusted cert for outward connections. You can update Cloudflare DNS records via their API - the typical dynamic DNS tools work fine. It works well.

I've always been unable to pull this off completely as I always want a way to SSH into my home network - but maybe there is a better way I can pull off this sort of 'break glass' functionality.

Pmop · 6 years ago

Wait. What? Windows VM for Java?

throwaway8941 · 6 years ago

It's most likely client-side stuff. Probably some crappy banking client, or an authentication client for some government websites, or something like that.

I use one for the sites below. It is written in Java/Kotlin, but barely works anywhere except Windows.

https://egov.kz/cms/en

https://cabinet.salyk.kz/

...

Mostly for old shitty IPMI.

snagglegaggle · 6 years ago

vCenter but no hosts? Why VMware stuff?

Colo: Three Hyper-V hosts on R620s. Goofball Quanta and Foxconn hardware for FreeNAS bare metal. All 2xE5v2 w/ 160-256GB RAM.

Home: Two VMware hosts on Hyve Zeus (Supermicro, 2xE5 64GB), one on an HP Microserver Gen8 (E3-1240v2 16GB). PiHole bare metal on a recycled Datto Alto w/ SSD (some old AMD APU, boots faster than a Pi and like 4w). Cloud Key G2 Plus for UniFi / Protect.

VMware because it's what I'm used to. Hyper-V because it's not. Used to have some stuff on KVM but :shrug:

zelly · 6 years ago

SSH: for git and tunneling literally everything: VNC, sftp, Emacs server, tmux, ....

Docker running random stuff

Used to run Pihole until I got an Android and rooted it. Used to mess with WebDAV and CalDAV. Nextcloud is a mess; plain SFTP fuse mounts work better for me. My approach has gone from trying to replicate cloud services to straight up remoting over SSH (VNC or terminal/mosh depending connectivity) to my home computer when I want to do something. It's simple and near unexploitable.

This is the way it should always have been done from the start of the internet. When you want to edit your calendar, for example, you should be able to do it on your phone/laptop/whatever as a proxy to your home computer, actually locking the file on your home computer. Instead we got the prolifetation of cloud SaaSes to compensate for this. For every program on your computer, you now need >1 analogous but incompatible program for every other device you use. Your watch needs a different calendar program than your gaming PC than your smart fridge, but you want a calendar on all of them. M×N programs where you could have just N, those on your home computer, if you could remote easily. (Really it's one dimension more than M×N when you consider all the backend services behind every SaaS app. What a waste of human effort and compute.)

I sympathize. My meditations about this lead me to thinking about waste as well.

Why computer at home though? For someone who moves around a lot and doesn't invest into "a home", this would be bothersome. Not to mention it's more expensive, in terms of energy and money. I think third-party data centers are fine for self-hosting.

There's really no difference. Mainly I use a machine at home instead of a data center VM because that's just one less bill to pay. I have two GPUs in there which would be very expensive on public cloud.

I guess one reason people might gravitate to home hosting is owning your own disks, the tinfoil hat perspective. You can encrypt volumes on public cloud as well, but it's still on someone else's machine. They could take a snapshot of the heap memory and know everything you are doing.

oarsinsync · 6 years ago

I like to be able to view and edit my calendar when I’m offline. This is remarkably often, regardless of whether I’m in London (UK), New York (USA), or some other country entirely.