So you moved your apps off Heroku, where did you go to and how has it worked out?
Particularly interested in:
1. How much work it took to move apps (be honest)
2. How much experience you had at the time of the migration - e.g. on one extreme your entire devops experience may consist of just Heroku (that's me) or on the other extreme you may be a k8s guru (this helps others gauge how they'll go)
3. How valuable were your learnings? E.g. replacing Heroku with an IAAS instead of another PAAS might take longer but give more fundamental learnings, and hence be worth it for some
4. Cost comparison
5. Summary/description of your apps (e.g. 20 tiny apps with a few hits per month, 5 medium with ~20k hits per month, 2 large with 1-2m hits per month type thing). Please give language/framework.
6. Anything else you want to add
Lots of features are added day-by-day and I'm open for new ones. Visit our Github (https://github.com/coollabsio/coolify) or our feedback page (https://feedback.coolify.io/) to get more info.
A few months ago, I quit my job and started to work on it full-time (best decision I've made).
It is not VC funded (said no to 30+ investors - time will let me know if it was a good decision or not ), but funded by the awesome community! <3
Why I'm mentioning this? Because I would like to focus on the functionalities, the community & the users and not the revenue and other boring metrics. I just would like to make a good software, enjoy the process and make people's life easier. I do not want to make millions of dollars from it. If me and my family could live happily, that is totally fine.
Let me know if you have an questions. I'm happy to answer them.
(Also I'm working on a cloud/managed hosting version of Coolify (https://beta.coolify.io/))
Furthermore, with just a bunch of dollars you can get a really good VM from Hetzner and host everything there. At the moment I am using Coolify for small Node.js apps as well as using a few more services: - VaultWarden. - Minio as an object storage solution. - Glitchtip as a Sentry alternative.
Essentially, if you are considering privacy and avoid paying a dime to Jeff directly or sideways (most of the PaaS services are built on top of AWS) projects like Coolify, CapRover, Dokku etc are the way to go.
Coolify sounds cool, And i love the idea of being able to have self-hosting as a backup which it would allow, but i don't want to HAVE to self host. Would rather pay Coolify to host for me until they go out of business / change direction, at which point i can continue on with the self-hosted bit.
I saw you mention the K8s soonest: is that your scaling solution? Seems it would be simpler.
Regardless I think we may look into this.
If you are using a remote server, the building process of an application is done on the remote server and not locally (on Coolify's server), so Coolify won't be affected on how many builds are doing concurrently. The only limit is the CPU/Memory of your servers. :)
The ultimate & standard scaling solution will be K8s.
This way every time you need to set add a new service to coolify there's the need to ship a new version.
Why did you choose to go this way rather than a more generalized concept of a service? Wouldn't having a generic service architecture be more maintainable and ( imo ) more importantly add the capability to have a dynamic, global and customizable 'app store' of sort?
I agree and disagree.
I agree that this kind of approach to coding is something that would put anyone senior-ish off totally, but I disagree at the same time because it's only 20-ish services, and the code is easily refactorable.
My assumption - after having read code 30+ years - is that this is done on purpose, so that they want to stabilise the approach and refactor it later. Though, as we all know, there's no thing as "I'll fix this later" in programming, but still (in this case) not something that would put me off.
Also, it's on github, so I could always refactor it for them if I'm not happy with it.
I’m currently making a feature, that will change this. All services could be defined with a simple yaml file that won’t be hardcoded. They will be sitting in a repository and everyone can add any service with a simple yaml file.
It is definitely more maintainable than the current solution, but with the current solution, I could iterate faster. :)
As far as I can see, there is no 330 line long if-else chain in that file. Nothing even close to it. Can you link to what you're talking about more specifically?
No one should be running that without personally verifying each line. What if I want to uninstall it? (I haven’t looked at this script) but many don’t effectively clean up properly after uninstalling, as that’s an afterthought.
What is specifically wrong with the dozens of existing packaging solutions we already have?
Ok, if you feel so strongly about it, then before you paste the link into your terminal and hit enter to execute it, how about you uh... open the shell script it will execute in your browser and read the contents of it?
You use this idea that people are executing stuff they don't know and blaming the tools for it but don't think to read the script yourself?
Do you also verify each line of software you install? If you trust the author of certain software, why do you mistrust their install script?
specifically? Specifically: they are even WORSE from a security standpoint because you don't have the option of "personally verifying each line". Just because someone's stuck it in a nice easy-to-install package doesn't mean it's not a virus / malware / whatever.
are you suggesting that you read the source code of every app / tool you download before you execute it? If not, hello... pot. kettle?
Or those awful programs that give you a stub when you try to download the app, only to download the actual payload to, again, god-knows-where.
Or even worse, packaging your app in a docker container!
Seriously people, give a CURL-compatible URL to your actual payload and then get the hell out of the way
Because one of the cool things about Heroku is scale-up. A cool project here is https://sst.dev/. Would love to see EKS/GKE native integration there.
From what I saw, it looks like it does a lot of stuff behind the scenes relating to domains and SSL.. does it work with the server behind Cloudflare (the dashboard and hosted sites), or is this something I would probably want to be accessing directly?
In contrast, Coolify has a great GUI that abstracts away the most common things about PaaS hosting, like connecting to GitHub automatically for git push deploys, SSL certificates, reverse proxying and custom domain support, and best of all, having support for Heroku style buildpacks as well as Dockerfiles. I've been quite happy with it, the creator has a Discord and responds to issues very quickly.
With regards to non-self-hosted options, I did try out Render, Fly.io and Railway but I found that their free servers were too anemic. I was compiling a Rust backend and it simply could not compile on their free servers. On Hetzner, for 5 bucks I could get a 2 AMD vCPU and 2 GB RAM machine that was sufficient to compile my Rust apps in a way that the non-self-hosted ones were not. I have a JS frontend app that works fine though but I wanted to keep everything under the same VPS, plus I can run other types of self-hosted services on it too, like Plausible analytics and a Ghost blog. I'm not sure if those are allowed on non-self-hosted options.
All in all, it costs me 5 bucks a month, and I never have to worry about sudden upcharges for traffic à la AWS as in the very worst, my VPS goes down for a while. I'm now running about 20 different services on this 5 dollar box including databases and applications as well as other services, works just fine.
E.g. I use Fly.io for one of my apps. I build a container image as part of CI on Github Actions and then just push that image to Fly as part of a deploy. Fly never deals with a build step (although it can do that as well).
$5 is very cheap for the Hetzner server you describe though.
https://softwarerecs.stackexchange.com/a/84179/7653
In contrast, with Coolify, you simply login with your GitHub account and it installs a (one-time setup) GitHub app onto your account that automatically sets up the webhooks and I didn't need to do any configuration after that, it just worked for all subsequent apps. That level of ease of use is the experience I had with Heroku and I do not understand why more self-hosted PaaS don't seem to replicate that as well.
Also I like the Coolify GUI over the CapRover one, dark mode plus modern non-Bootstrap design is nice.
Which ones are you missing (except Docker Swarm)?
Also when I update docker some apps don't restart properly, but that may just be my (basic Ubuntu dedicated) server though.
2 are elixir apps that have been created on fly, 1 was an elixir app I had running on heroku, and 1 was rails app on heroku I recently migrated. I used their heroku migrator and it worked pretty much automatically. I had a few steps to follow to move the DB, but their docs covered it - https://fly.io/docs/rails/getting-started/migrate-from-herok...
I didn't want a self-hosted flavor of Heroku, and I value an opinionated and active community around the platform. I'll mirror another comment here:
> Everyone wants to be the new Heroku, but that should be table stakes. Fly is innovative in the same way Heroku was when it launched.
Overall, my pricing has been about the same or 5-10% less per month.
It doesn't have the same "Click this button to add this third-party service to your dynos, with billing included" eco-system that Heroku has (had?) but my Heroku usage was pretty vanilla so it wasn't an issue for me.
I get "marketplaces" are hard and there's a chicken/egg problem, but is anyone working on it? I think fly has the rest of it sorted and is clearly better. Would love to see the additional services as well.
Seems you have to buy an anycast-ip from somewhere or become your own ISP/network provider, configure a bunch of bgp routers & routes but that seems impractical for most. Stuff like Akamai, Cloudflare, Route53 have anycast-dns but that seem specifically for dns queries, not responding to a query and routing it somewhere.
Is it a problem that's just too big and difficult that you can't really self-host/spit-and-duct-tape your own system?
Check out https://docs.google.com/spreadsheets/d/1abmV_mXWWCsVxHLfouSi...
Most definitely. For most parts, it is hashicorp-as-a-service but without hashicorp!
Both services have been flawless so far at a much better cost/value proposition.
I've not had a chance to play with it yet, what are they doing that's innovative.
(Genuine question, not being cynical).
If you've got a worldwide audience you can set up your app in Dallas, Paris, Chennai, and Tokyo, have it auto-scale (and re-balance) instances based on which regions are the most in demand (eg, minimum 2 app servers per region, max 20 app servers, and it'll auto drift them throughout the day as needed). It also puts all your servers on a common VPN for communication, and they've got some tooling in place to help manage things like db replicas, replaying write requests back to the primary db env, etc.
It's not necessarily trivial to port any old app over (if you're doing writes every request for logging etc, that's something you'd want to change up backgrounding the writes) but it really can make a huge impact in responsiveness once you're set up.
- https://fly.io/blog/introducing-litefs/
- https://fly.io/blog/all-in-on-sqlite-litestream/
This allows for extremely cheap s3 database backups as well as fast local serving layer.
And many more deep dives on their blog - https://fly.io/blog/
Vs
Heroku with cloudflare in front Postgres from heroku and memcached with an add on.
1. Very little. I put the app in docker a long time ago for local development but still used heroku for prod. I dumped my db and imported it into a docker run db on Hetzner. One thing to note: I did not expose the db to the internet. The app connects to the db over a docker network.
2. I do DevOps for a living.
3. It did not add to my learning.
4. $30/month for a large Hetzner dedicated instance and $12 for S3 (backups) vs $25 for the app (I added 2 more for 8 hours a day during busy season) and $25 for a job runner and whatever they were charging for DB. My total cost now is $42/month and on Heroku it was around $120 during busy season and $100 during non busy season.
5. Rails Postgres memcached and delayed job. I had to expand my app tier during busy season by up to 3 instances (winter). With my load testing the Hetzner instance handles much more than the 3 instances ever did in my load testing.
6. I added db dump cron jobs along with restic backups to S3.
The advantage S3 offers is seamless integration with other AWS services
IMHO, the ease of integration has significant value, it would take a very sizable S3 bill for me to consider not using S3
I may look into B2 later. But again, I’m well below my original costs so I may not look for a while.
Heroku should’ve closed that free tier years ago! Now they added some very very cheap plans as well which is fantastic for them = gets rid of some of the horrendous cost from platform abuse they dealt with every quarter.
Good for you Heroku, you were always under appreciated by the enterprise, with their clingy sysadmins and ops people fighting you like a plague and at the same time you were taken advantage of by an army of penny pinchers!
Heroku
I sometimes feel this services "need" an affordable paid plan, so I don't "freeload" on their free plans, and I don't need the expensive big plan.
I.e.:
I monitor a bunch of my hobby sites with StatusCake. They're great and I use the free plan but I'd like to pay a little for very simple monitoring. But the first paid plan is around $200 a year. But I don't need the premium plan, I just need the free plan, but paying, so I feel like I'm contributing to the business.
AWS and the others... you've got a free plan, but once you begin to deploy... ;)
DnsMadeEasy. They don't have a free plan, but I'm using their "Small business 10 domain plan". That was $30 a year for years. Past year was $40. Then they got acquired by DigiCert and since last month they've posted their new prices: ~$200 a year for a 5 domain plan.
It happens in a lot of providers, so I guess is a regular VC tactic to acquire volume, clients, publicity or whatever.
Odd mentality. You don't want to feel like a freeloader, but don't want to pay the asking price.
Can you share a link to their new plans? The pricing listed on their old page (https://www.heroku.com/pricing) still showing $7/mo as their cheapest option and it's already an overkill for my applications. In comparison, the cheapest option on fly.io is $1.94/mo (https://fly.io/docs/about/pricing/).
Note: I know $7 is the maximum cost, but by default their hobby tier app will not sleep, so I guess it still costs $7/mo to run in the end?
Thanks!
I am not sure if it will meet your expectations though, it doesn't really provide something much cheaper than 7/month or for an always-on production service. As I see it, the "eco" dyno is mainly a way to more easily save money with transient dynos (used for things other than production services) -- it does meet your concern about hobby dynos not sleeping though!
There's also a new cheaper very limited postgres corresponding to the previous free postgres.
So, if you had a light-traffic, or demo, or staging project that was using the free dyno & postgres offering, you went from $0/mo to $16/mo. That's a big jump, depending on your use case.
I'm all for well-restricted free offerings that let you try a platform; Heroku absolutely needed to change its plan. But if the cheapest web app with a db plan is $16/mo, people are going to look for something closer to $5/mo for their low-load deployments.
If I'm wrong about any of this, I'd love to know.
$5-$10 / month is peanuts for most western people. In other places of the world, that could be equivalent to a solid 10%-20% of their monthly paycheck (if the hosting services do not offer prices based on location).
Are there IT workers who earn ~100 USD a month? (Okay, thinking about it there must be. Freelancers who are not really experienced, or working in very local very low productivity companies.)
Aren't there already small hosting providers regionally who price accordingly? (We tried to run one here in Hungary around 2009-2010, but there was no point. Free stuff provided by giant companies is a much better value, so the local market is composed of otherwise non-typical clients who actually want local hosting with something extra, all in all not a great business. Prices are very low, competition is fierce, etc.)
1. Almost no work. The app was Node.js + Postgres, it stayed that way and only needed a few environment variable changes plus a database migration.
2. Almost none, I use PaaS because I concentrate on making games, and want to spend as little time touching infrastructure as possible.
3. Learned mostly that switching PaaS is not a big deal if you stick to common offerings (Postgres instead of a fancy exact-purpose-fit DB).
4. ~$100 to $27/mo
5. A Node.js app that gets rather low traffic but needs to be stable, in the 1000s of req/day. It only handles highscores/ranked mode seeds, so low requirements.
6. Would heartily recommend Render if you want to stay with a Heroku-like PaaS approach and they already support what you need.
How has the Postgres DB worked out for you with Render? I've been weary of using a relational DB in general since it's a single point of failure and really disruptive if your instance goes down (even with a back-up DB to failover too can be problematic if data doesn't get replicated before a crash).
Also are you just using your Node.js app as a back-end server or are you also serving web pages from it?
It's an API consumed by the game client, no user-facing webpages.
I was looking for somewhere I could run web services and cron jobs both in the same place.
They have a Heroku importer, however I think you need to ask to have it turned on. I found that it was easier for me to build docker images for my apps and use a 'build service' instead however. YMMV
I found their support to be very responsive and enjoy using their UI. The UI, builds and so on all feel very fast.
Northflank can run databases, however for my databases I've been running them on ElephantSQL for some time (https://www.elephantsql.com) - even when I was on Heroku.
Free for Dev's PaaS list is worth a review: https://free-for.dev/#/?id=paas
I actually don't see a way to delete a project once created either.
You can delete your project navigating to the billing page inside a project. To delete your account you can send a support request and we can process your request (described in the privacy policy). We'd like to automate it more, however we'd like a formal opt-in via email of the account/team owner when deleting backups and stateful workloads right now.
Addon disk pricing is the same as our volume pricing for services.
The disks are SSDs. We've added a margin on-top of GCP, EC2 and Azure SSD pricing so I wouldn't say they are expensive in comparison to other providers.
It's possible to configure HDD storage which is much cheaper, would be happy to enable that feature flag for you. SSD $0.30 per GB, HDD $0.15 per GB. We'll add HDD pricing to the site and start to enable it by default for everyone.
I think PaaS is awesome. I actually consider moving to one of the managed offerings from time to time... then I look at the pricing.
As a systems engineer specialising in automation, it's hard for me to justify the $75-100/month fee from a PaaS provider (not including network bandwidth), when I currently pay that now for the same thing in AWS including network bandwidth (and I haven't even optimised it with CloudFront yet and static asset deployments to S3.)
I'm not saying PaaS should be a foregone conclusion based purely on my microscopic example and edge case, but if you can configure a CI/CD pipeline (I recommend GitLab CI or GitHub Actions), manage/configure an OS (I recommend Ansible), write Terraform to build out your infrastructure (loads and loads and loads of TF code libraries with example code you can just run with), and a few extra steps, then I'd recommend going IaaS.
You can keep the entire thing KISS, you'll know it inside out, you can grow it (or not) to any scale whilst keeping costs considerably lower than any managed PaaS available, you can hire someone else down the line to look after it when you've reached that level of scale/success/financial freedom, and more.
That being said, I can certainly see a huge value add in doing both: IaaS with a self-managed PaaS stack on top of it. That is, I think, the sweet spot.
Note: yep, maintaining your own infra' is undesirable to a lot of people. If you're one of those people, then stick to managed PaaS and pay the price (it's worth it in your case.)
Note: yep, your time does (technically) have a cost associated with it, but if you're building solutions based on IaaS everyday of your life, you've likely done a lot of the work already.
If you have 100 dev teams, infra experience is a lot of “overhead.” If you have 5, you just make sure some of the devs know how to strace and tcpdump and can collab in an incident. Frankly, building infrastructure is easy. Maintaining it is the hard part. Preventative maintenance, security, recovering from incidents are all much harder than figuring out enough for it to run.
It was a brilliant idea. Let devs be devs. But the price has always been so high that, at scale, it wasn’t far off the HR cost.
We’ve gone 180 on that with k8s. Now you have to understand a datacenter with different vocabulary to effectively leverage it. You see low adoption at “I have an app” because it’s more complicated than it’s worth. Higher adoption above that. But almost 0 adoption at I need 100+ clusters. And 90% of what I see assumes to exist in a single cluster as if etcd never hangs itself.
We’ve built all of these abstractions, but ultimately you have to understand the thing below them (ie Linux and VMs) when performance sucks or something goes boom.
I don’t know how someone walks in and learns it all at once vs my life of learning with the growing system from Unix on physicals to now.
It’s like my life with Java. Most devs don’t understand where to start troubleshooting performance or the JVM, despite rich tooling (magical APM, JFRs, etc).
I've read of smaller companies building their own PaaS over K8s, which is in fact how Kubernetes makes sense.