I'm afraid I don't have a lot to add to this conversation but I have to say I just love Tailscale. I don't often run across software that feels so right and when I do it's a great surprise. Every time I see a new feature they're releasing I'm always impressed at how adept they are at targeting modern pain points.
I grew up and got into software by messing around with self-hosting web servers and game communities as a kid. As time has gone on I felt like we had lost some of the magic of easily sharing your machines and your creations with other people. We have a ton of services where you can now deploy and share your creations, but we've moved further and further away from direct sharing. There were plenty of good reasons why this has happened, with security being the most obvious factor, but it still makes me a little sad. I want my things to be able to talk to each other no matter where I am. I want to be able to invite my friends in and have access to my stuff.
Tailscale makes all of that quick, easy and awesome. I think it's really neat, makes me feel like a little nerdy kid again.
> I'm afraid I don't have a lot to add to this conversation but I have to say I just love Tailscale.
Strongly seconded. In my last company we used TailScale in some medium-advanced configurations, and from the dead-simple basic stuff up though some of the trickier stuff it's just a joy to use.. It's making much better networking practices highly-accessible and I'd bet ends up making the Internet a more secure, better organized system as a whole.
They run an amazingly transparent engineering process, for example their issue page (https://github.com/tailscale/tailscale/issues) is a model of transparent, responsive, involved open development. They embrace cool, modern, quirky stuff like NixOS (https://tailscale.com/blog/nixos-minecraft/). It's just generally really high-quality software developed with a very cool "hacker" philosophy.
TailScale is IMHO the coolest place to work right now, and something that almost any software company should look at if they do any networking.
If there's anything not to love, I can't see it. :)
Tailscale is cool, but if we focus on the product that this post discusses, Funnel won't give you the ability to use your own domain name. Cloudflare Tunnels will do that though. I will continue to use Tunnels.
> As time has gone on I felt like we had lost some of the magic of easily sharing your machines and your creations with other people.
> I want my things to be able to talk to each other no matter where I am.
What isn't easy about forwarding packets destined for port 80/443 of your public IP to the local service in question and being a part of the public Internet like things were from the start?
Using Tailscale is the opposite of self-hosting, you're bringing someone else's third party service in, and adding more complexity and another point of failure.
> What isn't easy about forwarding packets destined for port 80/443 of your public IP to the local service in question and being a part of the public Internet like things were from the start?
- Not every home internet service gets a publicly routable IPv4 address anymore (e.g. CGNAT)
- Not every home internet service gets a static IPv4 address so folks have to handle DynDNS
- Not everyone is comfortable exposing their home network IP address in DNS (Tailscale only shares the endpoint IP once the endpoint is auth'd onto the network)
- Not everyone is comfortable configuring heavy auth/fail2ban/app layer safeties (Tailscale makes the services uncontactable unless you are auth'd into the Tailscale network)
- Not everyone is comfortable/can be bothered configuring Wireguard in highly dynamic environments
> Using Tailscale is the opposite of self-hosting, you're bringing someone else's third party service in, and adding more complexity and another point of failure.
Self-hosting need not be a zealot position - rather one can pick and choose what makes sense for them. Tailscale allows you to build your own network where all the nodes are auth'd (and tailscale lock means you don't even need to trust their keys by default) and non-public internet routable but still globally reachable from known safe devices. This can actually make folks more comfortable with self-hosting their own stuff since it removes so many other considerations. There is also headscale if folks want to self-host the coordination server.
Some argue that a third party service adds complexity and a point of failure. I'll point out that configuring a self-hosted publicly exposed thing from scratch for the first time has a rabbit hole of unknown complexity to the uninitiated. A tool like Tailscale can remove some of those complexities allowing focus on others.
> What isn't easy about forwarding packets destined for port 80/443 of your public IP to the local service in question and being a part of the public Internet like things were from the start?
Most of the evil in the world currently can be traced back to NATs and dynamic IPs.
In a more general sense, I think these compromises were made available because of a consumerist attitude towards the internet. Yes, we had a real issue with ipv4 exhaustion, but it if it affected businesses who couldn’t even host a website anymore, would this really have been an issue still? More likely people said that these things were an ok workaround because consumers don’t need X anyway. Sometimes these smart hacks engineers are so good at coming up with invalidate crucial invariants about the systems we love.
> As time has gone on I felt like we had lost some of the magic of easily sharing your machines and your creations with other people. We have a ton of services where you can now deploy and share your creations, but we've moved further and further away from direct sharing.
This is interesting, as it hasn't been my experience on the hobbyist side (game servers, personal projects, etc). ngrok, localtunnel, tunnelmole, rathole, tunnelto, zrok, et al. If the use case is just sharing something you built thats behind NAT / on a private subnet, there is no shortage of solutions.
I constantly read good things about Tailscale, as well as to a lesser degree Cloudflare, that I think I'm missing out.
But I've experienced so many times that companies change things and this can mess up the workflow or infrastructure really bad, adding days of work to implement an alternative.
With your hype, how much do you trust that you can rely on Tailscale? Should I feel safe when giving them control?
Any company can take a turn for the worse, and any time you've got SaaS deep in your stack there's risk there.
I can only say that I worry about TailScale growing up to be evil the least of basically every SaaS company I've ever used. They seem extremely serious about making the interaction a "win/win" and keeping it that way as they grow.
This feature is a delight to use. I've tested a few web applications, APIs, and webhooks using it over the last month or two and only experienced a handful of glitches even before it was in beta.
I like the idea of consolidating all my network ACLs with a single configuration file with Tailscale, but I don't like being wedded to a proprietary platform for my personal use. Hopefully headscale gets a similar feature, perhaps minus Tailscale's DNS management.
The is like DynDNS on steroids. Awesome job. It should be noted that for high bandwidth applications, you'll incur a bit of a penalty due to hops but other than that it's pretty solid.
Every time I see tailscale do something really neat I'm always a little disappointed to find out they still offer only the three auth schemes- and I really don't want to tie my networking to google/github/ms. On top of the various tinfoil hat reasons, I know a variety of people who have had these accounts terminated out of the blue, and it throwing out my networking stack would be insanely aggravating.
If you're reading tailscale, I will pay you actual real dollars per month to offer a different not-tied-to-a-megacorp authentication scheme. Till then, guess I've got headscale.
Well god damn there it is! Three days fresh, even! Thanks!
Looks like a fair lot of work to get it configured, but few good things come entirely free. Wonder if there's enough people that could get together for a communal one...?
Got to the end of that post and thought: definitely don't want to self host that!
Are there good options for an IdP that has good data policies that are easy to wire in with tailscale? I'm not opposed to paying for it. I wonder if Zoho can do this for me, I'm very happy paying them $12/yr for email.
Question about the docs, it mentions that "The WebFinger endpoint must be hosted at the domain of the email address provided during setup". Would it be possible to support a subdomain?
Also, a small ask: could the webfinger request that's sent include the `rel` and a well-known user resource params, for the situations where there's already a webfinger implementation there that isn't 100% under dev control which requires these params
like
GET /.well-known/webfinger?
resource=tailscale-webfinger%3A%40mydomain.com&
rel=http%3A%2F%2Fopenid.net%2Fspecs%2Fconnect%2F1.0%2Fissuer
HTTP/1.1
Host: mydomain.com
lastly, is this request resent at every auth event?
Are there really Microsoft accounts that were terminated out of the blue? I always had the feeling they acted a bit more responsibly around that than Google.
How is it with high bandwidth application? E.g would it be okay to put my media server behind it? Currently tunneling it through a VPS so cloudflare doesn't get mad.
Tailscalar here: there is a bandwidth limit, it's a funnel, not a hose. We don't announce what the bandwidth limit is, but please keep in mind that it does exist. I would suggest setting up your media server inside your tailnet for the best experiences, but it depends on who you are sharing it with and why.
I might be missing something; isn’t a Tailnet a bunch of user devices with wireguard tunnels connecting to each other directly? Where does the limit happen?
(And thanks for your work!)
Edit after 1 minute: of course, limit on Tailscale Funnel itself. (Too deep into thinking about Tailscale and forgot about the actual topic of the post. )
Hola, how would the bandwidth limit work within the tailnet if I am accessing it from outside my home network? Wouldn't it incur some bandwidth on Tailscale's end?
I wonder if the DERPy-stuff helps remove most of the bandwidth concerns - thinking out loud...
Since tailscaled uses the tun/tap driver and thus copies all traffic to userspace (and back), it is extremely inefficient. On my Haswell i5 (plus multiple servers with comparable hardware) the process consumes 40% of CPU time at just 4 MiB/s, and close to 100% at 10-11 MiB/s (with recent sendmmsg/recvmmsg patches¹).
This is about ~2-3x worse than similar applications written in highly optimized C, so don't expect any miracles from further optimizations unless they switch to kernel Wireguard (which doesn't seem likely in the nearby future).
They claim it's very difficult if not impossible, but this sounds like an issue with their architecture — a similar application from their competitors² has had kernel WireGuard support from the start (no relation, I don't even use it and cannot recommend for or against it).
Tailscalar here, for what it's worth, I run my plex server on Tailscale (i5 10600) and I haven't noticed any observable lag due to the TUN/TAP driver. Even with 4k bluray rips at several tens of megabits per second of video quality. I also regularly get near the limit of gigabit ethernet when transferring big files like machine learning models (the 1280 byte MTU plus WireGuard overhead adds up over time and can make the application observed rate be less than what the NIC is actually doing).
Kernel WireGuard for Tailscale is hard because of DERP (HTTPS/TCP fallback relay, all connections start over DERP so that they can Just Work if hole punching fails), but I'm sure it could happen with the right combination of eBPF and Rust in the kernel. It'd be a bit easier if there was a high level abstraction for using the kernel TLS stack to do outgoing TLS connections.
Hi! Tailscaler here, one of the folks who worked on the recent throughput improvements. One of the machines I was testing with during our work on segment offloading was a Haswell. I absolutely understand your concern if we're using 40% of CPU at 4MiB/s, we should be doing substantially better than that on efficiency. In our various testbeds which include CPUs like yours, we see higher performance. If you'd like us to look into the issue, do email support@tailscale.com - we'd be really happy to dig in and find the cause.
We have continued our work on performance improvements, and along that path, as an example, we recently diagnosed an issue with a change in the kernel frequency scaling governor that has a regression that Tailscale can tickle and we have an ongoing discussion with the kernel maintainers about that problem. I'm not at all assuming this particular thing is the key source of the performance you're observing, it is more to provide an anecdote that we're still digging deep into areas where we aren't performing well and finding the root cause, and working both inside and outside to address those and where appropriate to add workarounds as well.
I observe there's about 37% overhead when using TS connection on a local gigabit network.
Copying large file from Synology DS1821+ NAS (Amd Ryzen V1500B) to Windows PC (i7-6700K)
is about 111-113 MB/s when accessing NAS directly and 70-73 MB/s when traffic goes through TS
(different large files, so no caching here).
• Ngrok pulled a pricing bait-and-switch a year ago increasing prices to $240/year/user if you wanted a stable subdomain, even for bandwidth-trivial users.
-
Edit: Looks like they now have an $8/month/user tier for a single stable subdomain and now offer some edge hosting as well.
$8/user/mo is still far too much for a stable domain without the spam-guard intermediary page, and I'm glad there's some free competition in this space now.
This is my first time using tailscale, and I set up and figured out funnel within fifteen minutes.
from what I can gather it provides the same functionality as ngrok without reaching for another tool. If Tailscale already exists in your networking tool belt this functionality comes really handy.
Cloudflare gets a lot of criticism on HN (I can fundamentally understand why) but it turns to irrational blind absolutist hatred very quickly.
Cloudflare tunnels have been around for a while. They have a variety of features (IMO) well beyond what Tailscale has in beta here.
In terms of the other comments, Cloudflare has many millions of satisfied customers and roughly 80% of the CDN market so people hosting internet facing properties obviously see value in what they provide.
Cloudflare tunnels are a more mature, more capable, more performant, and cheaper version of Funnel backed by one of the largest networks in the world with hundreds of other features from CloudFlare tailscale doesn't have (and factoring in network, never will).
If you have some grudge against Cloudflare for MITM, ToS, etc now you have an alternative (of sorts) to Cloudflare tunnels.
From the article: When you turn on Funnel, we create public DNS records for your node.tailnet.ts.net name that points to a set of ingress servers we operate around the world, and then we give those servers very limited access to your tailnet.
Readit News
I grew up and got into software by messing around with self-hosting web servers and game communities as a kid. As time has gone on I felt like we had lost some of the magic of easily sharing your machines and your creations with other people. We have a ton of services where you can now deploy and share your creations, but we've moved further and further away from direct sharing. There were plenty of good reasons why this has happened, with security being the most obvious factor, but it still makes me a little sad. I want my things to be able to talk to each other no matter where I am. I want to be able to invite my friends in and have access to my stuff.
Tailscale makes all of that quick, easy and awesome. I think it's really neat, makes me feel like a little nerdy kid again.
Strongly seconded. In my last company we used TailScale in some medium-advanced configurations, and from the dead-simple basic stuff up though some of the trickier stuff it's just a joy to use.. It's making much better networking practices highly-accessible and I'd bet ends up making the Internet a more secure, better organized system as a whole.
They run an amazingly transparent engineering process, for example their issue page (https://github.com/tailscale/tailscale/issues) is a model of transparent, responsive, involved open development. They embrace cool, modern, quirky stuff like NixOS (https://tailscale.com/blog/nixos-minecraft/). It's just generally really high-quality software developed with a very cool "hacker" philosophy.
TailScale is IMHO the coolest place to work right now, and something that almost any software company should look at if they do any networking.
If there's anything not to love, I can't see it. :)
Only thing atm I don’t like it the battery use on my iPhone. But it’s well worth it.
FWIW, that's a very high priority currently by a number of people at Tailscale. We're working on it.
> I want my things to be able to talk to each other no matter where I am.
What isn't easy about forwarding packets destined for port 80/443 of your public IP to the local service in question and being a part of the public Internet like things were from the start?
Using Tailscale is the opposite of self-hosting, you're bringing someone else's third party service in, and adding more complexity and another point of failure.
- Not every home internet service gets a publicly routable IPv4 address anymore (e.g. CGNAT)
- Not every home internet service gets a static IPv4 address so folks have to handle DynDNS
- Not everyone is comfortable exposing their home network IP address in DNS (Tailscale only shares the endpoint IP once the endpoint is auth'd onto the network)
- Not everyone is comfortable configuring heavy auth/fail2ban/app layer safeties (Tailscale makes the services uncontactable unless you are auth'd into the Tailscale network)
- Not everyone is comfortable/can be bothered configuring Wireguard in highly dynamic environments
> Using Tailscale is the opposite of self-hosting, you're bringing someone else's third party service in, and adding more complexity and another point of failure.
Self-hosting need not be a zealot position - rather one can pick and choose what makes sense for them. Tailscale allows you to build your own network where all the nodes are auth'd (and tailscale lock means you don't even need to trust their keys by default) and non-public internet routable but still globally reachable from known safe devices. This can actually make folks more comfortable with self-hosting their own stuff since it removes so many other considerations. There is also headscale if folks want to self-host the coordination server.
Some argue that a third party service adds complexity and a point of failure. I'll point out that configuring a self-hosted publicly exposed thing from scratch for the first time has a rabbit hole of unknown complexity to the uninitiated. A tool like Tailscale can remove some of those complexities allowing focus on others.
Most of the evil in the world currently can be traced back to NATs and dynamic IPs.
In a more general sense, I think these compromises were made available because of a consumerist attitude towards the internet. Yes, we had a real issue with ipv4 exhaustion, but it if it affected businesses who couldn’t even host a website anymore, would this really have been an issue still? More likely people said that these things were an ok workaround because consumers don’t need X anyway. Sometimes these smart hacks engineers are so good at coming up with invalidate crucial invariants about the systems we love.
Now we have "IPv4 scarcity" and CGNAT bullshit :/
This is interesting, as it hasn't been my experience on the hobbyist side (game servers, personal projects, etc). ngrok, localtunnel, tunnelmole, rathole, tunnelto, zrok, et al. If the use case is just sharing something you built thats behind NAT / on a private subnet, there is no shortage of solutions.
But I've experienced so many times that companies change things and this can mess up the workflow or infrastructure really bad, adding days of work to implement an alternative.
With your hype, how much do you trust that you can rely on Tailscale? Should I feel safe when giving them control?
I can only say that I worry about TailScale growing up to be evil the least of basically every SaaS company I've ever used. They seem extremely serious about making the interaction a "win/win" and keeping it that way as they grow.
I like the idea of consolidating all my network ACLs with a single configuration file with Tailscale, but I don't like being wedded to a proprietary platform for my personal use. Hopefully headscale gets a similar feature, perhaps minus Tailscale's DNS management.
If you're reading tailscale, I will pay you actual real dollars per month to offer a different not-tied-to-a-megacorp authentication scheme. Till then, guess I've got headscale.
You also don't need to pay Tailscale to use it.
Looks like a fair lot of work to get it configured, but few good things come entirely free. Wonder if there's enough people that could get together for a communal one...?
Are there good options for an IdP that has good data policies that are easy to wire in with tailscale? I'm not opposed to paying for it. I wonder if Zoho can do this for me, I'm very happy paying them $12/yr for email.
Question about the docs, it mentions that "The WebFinger endpoint must be hosted at the domain of the email address provided during setup". Would it be possible to support a subdomain?
Also, a small ask: could the webfinger request that's sent include the `rel` and a well-known user resource params, for the situations where there's already a webfinger implementation there that isn't 100% under dev control which requires these params like
lastly, is this request resent at every auth event?Thanks!@!
(Also see the various comments in the discussion.)
(And thanks for your work!)
Edit after 1 minute: of course, limit on Tailscale Funnel itself. (Too deep into thinking about Tailscale and forgot about the actual topic of the post. )
I wonder if the DERPy-stuff helps remove most of the bandwidth concerns - thinking out loud...
This is about ~2-3x worse than similar applications written in highly optimized C, so don't expect any miracles from further optimizations unless they switch to kernel Wireguard (which doesn't seem likely in the nearby future).
They claim it's very difficult if not impossible, but this sounds like an issue with their architecture — a similar application from their competitors² has had kernel WireGuard support from the start (no relation, I don't even use it and cannot recommend for or against it).
1: https://tailscale.com/blog/throughput-improvements
2: https://github.com/netbirdio/netbird
Kernel WireGuard for Tailscale is hard because of DERP (HTTPS/TCP fallback relay, all connections start over DERP so that they can Just Work if hole punching fails), but I'm sure it could happen with the right combination of eBPF and Rust in the kernel. It'd be a bit easier if there was a high level abstraction for using the kernel TLS stack to do outgoing TLS connections.
We have continued our work on performance improvements, and along that path, as an example, we recently diagnosed an issue with a change in the kernel frequency scaling governor that has a regression that Tailscale can tickle and we have an ongoing discussion with the kernel maintainers about that problem. I'm not at all assuming this particular thing is the key source of the performance you're observing, it is more to provide an anecdote that we're still digging deep into areas where we aren't performing well and finding the root cause, and working both inside and outside to address those and where appropriate to add workarounds as well.
Copying large file from Synology DS1821+ NAS (Amd Ryzen V1500B) to Windows PC (i7-6700K) is about 111-113 MB/s when accessing NAS directly and 70-73 MB/s when traffic goes through TS (different large files, so no caching here).
• Ngrok pulled a pricing bait-and-switch a year ago increasing prices to $240/year/user if you wanted a stable subdomain, even for bandwidth-trivial users.
-
Edit: Looks like they now have an $8/month/user tier for a single stable subdomain and now offer some edge hosting as well.
This is my first time using tailscale, and I set up and figured out funnel within fifteen minutes.
https://ngrok.com/docs/secure-tunnels/tunnels/tls-tunnels/
https://ngrok.com/docs/secure-tunnels/
Cloudflare gets a lot of criticism on HN (I can fundamentally understand why) but it turns to irrational blind absolutist hatred very quickly.
Cloudflare tunnels have been around for a while. They have a variety of features (IMO) well beyond what Tailscale has in beta here.
In terms of the other comments, Cloudflare has many millions of satisfied customers and roughly 80% of the CDN market so people hosting internet facing properties obviously see value in what they provide.
Cloudflare tunnels are a more mature, more capable, more performant, and cheaper version of Funnel backed by one of the largest networks in the world with hundreds of other features from CloudFlare tailscale doesn't have (and factoring in network, never will).
If you have some grudge against Cloudflare for MITM, ToS, etc now you have an alternative (of sorts) to Cloudflare tunnels.
Competition and choice is a good thing!
Deleted Comment
I've made some simple BASH functions to easily open and close ports to the public internet with Funnel: https://github.com/jeremyckahn/dotfiles/commit/2f02c594d8f2b...