Sandstorm was (is?) way ahead of its time. Fortunately a lot of its ideas live on (a la capnproto). My understanding is the achilles heel is the amount of effort required to integrate and update apps, since each app needs to be rather invasively modified to work with the centralized authorization framework. Also the general difficulty of managing a VPS or port forwarding. But once integrated it's really slick and secure.
The overall UX is very similar to my vision of the future of selfhosting. It should be as easy as installing an app on your phone, then going through a quick OAuth2 flow to set up a tunnel on a domain, and you're up and running.
I don't think app packaging could be said to be the achilles heel -- how much work it involves depends a lot on the app (some things are very easy, others can be painful), but it is certainly not the case that Sandstorm didn't take off because packaging was too hard.
“Was”, what’s left of sandstorm anymore besides a couple weekender coders whose $dayjob is at cloudflare these days? Doubt CF has anything like 10%/20% time like GOOG.
This codebase is basically abandonware at this point and can’t be trusted for anything serious these days. If CF brought this on as an official product (and renamed it), it would go a long ways to build back trust
Packaging existing apps tends to be extra work, since you have to retrofit things (like user accounts). But creating brand new apps can actually be less work, because it has things built in (like user accounts).
I think the answer is that packaging apps for Sandstorm is a bad hack, as ripping out the auth system and design granularity is all hard for an existing application. The best Sandstorm apps are written for Sandstorm, and in those cases, the granularity and built-in authentication make it way easier to write an app. I wrote a Sandstorm-native app in less than an hour.
But if you built your entire application assuming you're shipping it via Docker, untangling the sheer amount of redundant garbage you need to pare down to a single-document app that accepts outside authentication and security, ends up being a ton of work to package and then maintain it long-term.
Ideally, I think having apps not made for Sandstorm originally is mostly just a strategy to get enough people operating on Sandstorm's model for people to build Sandstorm-specific apps in earnest.
I think making apps that have a very clean separation between external security code (e.g. "the claims in the JWT") and internal security code (e.g. "the app's own internal understanding of security, regardless of provider") is always a fantastic idea.
The issue for me with Sandstorm is that it's great until some service/app you need is missing or broken, and then you're back to manual self hosting. I always find that self hosting even one app is the pain, self-hosting more isn't that bad (once you have your box setup).
I have tried for a client who wanted to install open source app in server with sandstorm about 3 or 4 years ago, it is just very slow at the time. Hope they improved that since.
I've done plenty of manual self-hosting. I explored sandstorm for a year or two in the mid 2010s. It worked as promised, but the complexity of integrations, fraying, showed.
I switched to yunohost 6 years ago. I almost entirely don't think about it. I use a few services, wallabag and Uptime Kuma. Tried Nextcloud, HedgeDoc /HackMD/CodiMD, and Monica PRM because it was so painless to give them a go.
My initial motivation for exploring it, came from the hope it could install NextCloud. My attempts at installing OpenCloud and NextCloud, with scripts that would make non-idempotent changes that fail on a second run after first pass fixes, left me sensing an unfriendly forced upsell environment.
I have had to do some manual fixing to complete an upgrade or two, but it has workflows for backups, and some stringent, but fully automated configuration, sanity checking. I don't worry about the complexity.
I tried out Yunohost and I do like it in principle, however what really started to annoy me was that they decided to roll their own sso, and my impression is that it's not very good (there are some threads on the forum criticising it for being quite a mess IIRC). The issue is that because of this it doesn't always play ball with apps which allow for SSO functionality, e.g. the seafile desktop app does not work with yunohosts SSO.
I wish they would just switch to an established system like authentik, authelia or keycloak. There were some issues about this in the tracker but I think the suggestions were shut down pretty quickly.
My understanding is YunoHost makes absolutely no attempt to isolate apps from one another, it just helps you install stuff on a Linux box. Unfortunately that means a single flaw in a single app compromises your entire server.
It’s was designed predocker and is a pretty old school setup. I remember they had issues updating Debian because the new Debian had a new php version which caused issues for some apps.
Yes, Yunohost makes installs very easy. But if something isn't working right then you got extra layers to deal with. I don't know how Sandstorm deals with this, but Yunohost taking care of the install and upgrade procedures means that you are bound to the Yunohost integration's version of the software. For instance when installing Discourse via Yunohost this version lags behind significantly. And also Yunohost disables the Discourse upgrade process, while it offers a superior experience.
I feel like we are always getting into the same trap with those self-hosting platforms.
It is the same problem we faced with the Linux Desktop & Mobile. A lot of cool visions, but fragmented efforts, limited app ecosystem & low adoption. We already have Sandstorm, Umbrel, Start9, Yuno, etc. but none of them had the success they deserved.
I'm wondering if taking something like Fedora CoreOS/IoT, Cockpit, linuxserver.io images & develop a nice GUI & documentation on top wouldn't have much greater chance to succeed.
People who will try to self host & will most likely stick to it are already familiar with tech & willing to tinker anyway.
Engineers love engineering, they love their own engineering when it comes to open source/free time projects, and they tend to not like the legwork of publishing, polishing and making it ready for public consumption. (Which is a lot of work on its own)
Engineers and QA/useability/marketing people tend not to mingle in these projects. I am broken record at this point, but Blender and Godot tackled this problem.
Take sandstorm, it looks like a informative website but just looking at it I see a ton of friction installing it. Whether or not that is true, but the optics are against it.
Sandstorm was actually a startup with product and marketing and we did spend a lot of time trying to make it as simple as possible to install.
The only way to make Sandstorm easier to set up would be to partner with hosting providers to offer one-click installers. In the absence of that, Sandstorm's current install is about as easy as it could possibly be given the constraint of "should run on a generic Linux host".
(Aside: People have always given Linux flak for being hard to install. But have you ever tried to install Windows from scratch? It's sooooo much harder. But no one cares because it's preinstalled on every PC you buy.)
(Aside 2: How did Chrome become the most popular browser? Sure as hell not because it's so easy for people to install. Most people don't know what a browser is. It got there by Google paying a lot of people to bundle it. It's honestly amazing that Google pulled it off, against competitors who owned their respective operating systems.)
Anyway, back in the day, Sandstorm actually offered its own hosting service, which you didn't need to be technical to sign up for. So it was actually quite easy then. But it was a first-party service. In retrospect I think building the first-party service was a mistake. It distracted too much engineering time away from actually building out the platform and ecosystem. It meant we could offer a better price point than a VPS provider would, but I think the people actually interested in the product at that point weren't so price-conscious. If I did it again I'd focus on partnerships.
That's a useful bit of feedback. Indeed I would actually say that installation is quite simple as such things go. And maintenance is dead simple, it updates itself in the background.
I would say containers, and especially docker, turned out to be the best solution for this space. We just need some good non-commercial tools for getting the fancy nobrainer app-managment. Everything existing now is ever commecial or still too complicated for the laymen.
The one big plus for ucs is their samba server, but it's not the right choice if you are looking for an appstore to install apps from.
My personal favorite is Cloudron, as they have an app concept that only needs minimal human maintenance. And you could even combine it with an LDAP server like ucs for the user management.
When an idea like this strikes but doesn't take off there's definitely an element of "before it's time" but it's also really hard to get people to adopt opinionated development models unless it's part of a bigger language ecosystem or brand which can offer the scale of opportunity like an app store.
There's been attempts to do this without the opinions e.g deploy docker containers but the reality is we do actually need a cohesive vision for the end user for anything like this to take off. Catering to the developers alone in the self hosted model is very hard. And I know Kenton tried hosting too. It's just a tough business all around.
I used this for a few years at a previous job and really liked it — it didn't get a ton of adoption by other people there, though, unfortunately. It had a bunch of cool ideas and seemed well implemented. I'm sad that it didn't work out as a commercial endeavor.
I like how they use capability-based security [0] and use Cap'n Proto protocol. This is another technology that is slow to get broad adoption, but has many things going for when compared to e.g. Protocol Buffers (Cap'n Proto is created by the primary author of Protobuf v2, Kenton Varda).
I recently evaluated different protocol libraries for inter-language IPC. The state of Cap'n Proto is sadly quite a bit worse than Protobuf. Many languages either don't have an implementation, or it hasn't been updated for several years.
But it's such a good system, I wish we could use it.
On the other side of things, I've started using https://www.pikapods.com/ they follow the docker model but are a paid (~$2/mo) hosting provider focusing on open-source self-hostable apps. Seems to me like an easier path for getting non-techies to 'selfhost' their own apps.
Pikapods have provided my software for a while. Have not had much feedback from users yet, but no bad feedback is a good sign I guess. The person behind PikaPods was very receptive to my concerns that their website mentioning of "upstream revenue share" could be misleading (didn't apply for my project), and they quickly took action to improve things.
But yeah, Sandstorm has been in a state of "not dead but not moving fast" basically since the company went under; things picked up a bit in 2020, and I got oriented-enough on the codebase during that time to keep it floating along, but, per the post, it's never been easy going.
Anyway, I wish I were answering this question in 6 months time, when I'll be able to show off a variation of Tempest that is relatively usable and can do a few tricks that Sandstorm can't.
Just spent ten minutes delightedly reading your updates on this. It looks fantastic! I'm really excited to watch what happens next, and wish I had time to contribute.
Are there primary areas in which contributions would be most valuable, especially from those without Golang experience/skills?
always interested in self hosted solutions, not tried sandstorm and up for giving it a go, however if I was starting a small fairly simple personal project with no need to scale can I get wheels on it with Tempest, or should I consider building on sandstorm then porting?
Readit News
The overall UX is very similar to my vision of the future of selfhosting. It should be as easy as installing an app on your phone, then going through a quick OAuth2 flow to set up a tunnel on a domain, and you're up and running.
I don't think app packaging could be said to be the achilles heel -- how much work it involves depends a lot on the app (some things are very easy, others can be painful), but it is certainly not the case that Sandstorm didn't take off because packaging was too hard.
Sandstorm the business was shut down 6 years ago https://sandstorm.io/news/2017-02-06-sandstorm-returning-to-...
And Sandstorm Oasis the hosting service was shut down almost 4 years ago https://sandstorm.io/news/2019-09-15-shutting-down-oasis
This codebase is basically abandonware at this point and can’t be trusted for anything serious these days. If CF brought this on as an official product (and renamed it), it would go a long ways to build back trust
Security can just be a OTP: https://github.com/tinspin/rupy/blob/master/src/se/rupy/http...
But if you built your entire application assuming you're shipping it via Docker, untangling the sheer amount of redundant garbage you need to pare down to a single-document app that accepts outside authentication and security, ends up being a ton of work to package and then maintain it long-term.
Ideally, I think having apps not made for Sandstorm originally is mostly just a strategy to get enough people operating on Sandstorm's model for people to build Sandstorm-specific apps in earnest.
I've done plenty of manual self-hosting. I explored sandstorm for a year or two in the mid 2010s. It worked as promised, but the complexity of integrations, fraying, showed.
I switched to yunohost 6 years ago. I almost entirely don't think about it. I use a few services, wallabag and Uptime Kuma. Tried Nextcloud, HedgeDoc /HackMD/CodiMD, and Monica PRM because it was so painless to give them a go.
My initial motivation for exploring it, came from the hope it could install NextCloud. My attempts at installing OpenCloud and NextCloud, with scripts that would make non-idempotent changes that fail on a second run after first pass fixes, left me sensing an unfriendly forced upsell environment.
I have had to do some manual fixing to complete an upgrade or two, but it has workflows for backups, and some stringent, but fully automated configuration, sanity checking. I don't worry about the complexity.
Every app is run by a special user created to do just that. This app user only has permissions for the app and data folder.
(I don’t have much security/hardening experience)
I'm wondering if taking something like Fedora CoreOS/IoT, Cockpit, linuxserver.io images & develop a nice GUI & documentation on top wouldn't have much greater chance to succeed. People who will try to self host & will most likely stick to it are already familiar with tech & willing to tinker anyway.
Engineers and QA/useability/marketing people tend not to mingle in these projects. I am broken record at this point, but Blender and Godot tackled this problem.
Take sandstorm, it looks like a informative website but just looking at it I see a ton of friction installing it. Whether or not that is true, but the optics are against it.
The only way to make Sandstorm easier to set up would be to partner with hosting providers to offer one-click installers. In the absence of that, Sandstorm's current install is about as easy as it could possibly be given the constraint of "should run on a generic Linux host".
(Aside: People have always given Linux flak for being hard to install. But have you ever tried to install Windows from scratch? It's sooooo much harder. But no one cares because it's preinstalled on every PC you buy.)
(Aside 2: How did Chrome become the most popular browser? Sure as hell not because it's so easy for people to install. Most people don't know what a browser is. It got there by Google paying a lot of people to bundle it. It's honestly amazing that Google pulled it off, against competitors who owned their respective operating systems.)
Anyway, back in the day, Sandstorm actually offered its own hosting service, which you didn't need to be technical to sign up for. So it was actually quite easy then. But it was a first-party service. In retrospect I think building the first-party service was a mistake. It distracted too much engineering time away from actually building out the platform and ecosystem. It meant we could offer a better price point than a VPS provider would, but I think the people actually interested in the product at that point weren't so price-conscious. If I did it again I'd focus on partnerships.
Similiar and a little bit more mature is also YunoHost, https://yunohost.org/, or for professional environments, UCS https://www.univention.com/.
My personal favorite is Cloudron, as they have an app concept that only needs minimal human maintenance. And you could even combine it with an LDAP server like ucs for the user management.
There's been attempts to do this without the opinions e.g deploy docker containers but the reality is we do actually need a cohesive vision for the end user for anything like this to take off. Catering to the developers alone in the self hosted model is very hard. And I know Kenton tried hosting too. It's just a tough business all around.
[0] https://sandstorm.io/how-it-works#capabilities
[1] https://capnproto.org
But it's such a good system, I wish we could use it.
But yeah, Sandstorm has been in a state of "not dead but not moving fast" basically since the company went under; things picked up a bit in 2020, and I got oriented-enough on the codebase during that time to keep it floating along, but, per the post, it's never been easy going.
Anyway, I wish I were answering this question in 6 months time, when I'll be able to show off a variation of Tempest that is relatively usable and can do a few tricks that Sandstorm can't.
Are there primary areas in which contributions would be most valuable, especially from those without Golang experience/skills?
[0] https://github.com/zenhack/tempest
Deleted Comment