Honestly you can go very far with unmettered data plans. We consume 2GB/S, maxed out, which ends up being 5184TB a month. We have millions on users a day, all on a streaming video platform.
It cost less than $2K/Month.
The cloud is crazy expensive. Private servers are beasts, and they are cheap.
Of course, for this price, you don't have redundancy and horizontal scaling.
You also don't have to maintain and debug a system with redundancy and horizontal scaling.
> We consume 2GB/S, maxed out, which ends up being 5184TB a month. We have millions on users a day, all on a streaming video platform.
> It cost less than $2K/Month.
The solution in this article is serving on the order of 100TB/month for $400/month including a high speed global CDN, their API and database servers being hosted reliably, and redundancy and backup being handled by someone else.
Your solution is hosting on the order of 1000s of TBs of month (ignoring the database and other aspects of this website), but the price is an order of magnitude higher. You’ve also given up all of the automatic redundancy and hands-off management, and you don’t have the benefit of a high speed global CDN.
But more importantly, you have significantly higher engineering and on-call overhead, which you’re valuing at $0.
If anything, that only makes Polyhaven’s solution sound more impressive.
> Of course, for this price, you don't have redundancy and horizontal scaling.
Which is a huge caveat. The global CDN makes a big difference in how the site loads in global locations. Maybe not a big concern if you’re serving of static files with a lot of buffering, but they have a dynamic website and a global audience and they said fast load times are important.
> You also don't have to maintain and debug a system with redundancy and horizontal scaling.
But you have to do literally everything else manually and maintain it yourself, which is far from free.
All of these alternative proposals that value engineering time at $0/hour and assume your engineers are happy to be on-call 24/7 to maintain these servers are missing the point. You pay for turnkey solutions so you don’t have to deal with as much. Engineers don’t actually love to respond to on call events. If you can offload many of your problems to a cloud provider and free up your engineers for a nominal cost, do it.
A python that knows python and linux is enough to support the up time needs and evolutions of this particular product. Actually a part time one.
The entire team is composed of half of one dev.
I'm 100% sure it's way cheaper than anybody that has AWS on resume.
Of course, some things have to give, like the global CND, and some data guarantees.
Everything is a compromise. It's all depend of what is important for your project.
EDIT: also, my comment was not meant to oppose the article, but rather confirm the view that you should calibrate your setup to your project. Doing so will lead to great savings in hosting, and project complexity. A lot of projects don't need the cloud.
Whether or not a site is hosted in the cloud, it will break from time to time. S** invariably happens, no matter what. So, even if you host in the cloud, you're going to have problems, but they will be different problems. A developer to backup and support the site will be required, one way or the other. Case in point: polyhaven.com (the subject of this article) is not reachable as I write this.
Thanks! Exactly that! I always tell that customers and their response is:
-But google/facebook/amazon...
-But uptime needs to be 99.999
-But everyone uses cloud
Most businesses are not a trading-market, have less then 100 peoples (aka you are probably not another amazon), and no bonus using a cloud/kubernetes etc.
But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
And you can tell them all of those are possible, but they need to have a massive budget, not just for the monthly bills but also to hire experts to set things up with those constraints.
I worked at a company once that, from higher up, said that they had to have five nines of uptime. We had some really good cloud engineers there (one guy set up a server / internet container for the military in Afghanistan; in hindsight he said they should've just sent a container of porn dvd's), and they really went to town. For that five nines uptime, you're already pretty much required to set up your infrastructure to use multiple availability zones, everything redundant multiple times, etc.
Of course, the actual software we wrote was just a bunch of CRUD services written in nodejs (later scala because IDK), on top of a pile of shit java that abstracted away decades of legacy mainframes.
Either your compute demand is highly elastic, or your revenue and profit scale with usage, otherwise the Cloud is probably not for you. At least not in the longterm or for running websites.
Isn't AWS down like every two months for a few hours? That's far off the 99.999% mark. No one can guarantee 100% uptime and sometimes it's even better to have that under your control (eg. have a dedicated server and a backup one from different providers).
My point is that, if you want the highest possible uptime, you shouldn't rely on a single (cloud) provider.
You can have redundancy and horizontal scaling with private servers and still end up paying less than what you would on AWS and the like.
I have some clients who use AWS and others who prefer colo and/or dedicated servers from traditional datacenters. The latter group can afford to over-provision everything by 3-4x, even across different DC's if necessary. DC's aren't yesterday's dinosaurs anymore. The large ones have a bunch of hardware on standby that you can order at 3 a.m. and start running deployment scripts in minutes.
Not including extra human cost in the analysis is just disingenuous. I think to manage private servers of that size you would need at least two extra experts totalling at least $20k/month.
> I think to manage private servers of that size you would need at least two extra experts totalling at least $20k/month.
What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months, or to make minor quick tweaks and run some updates. Let's be generous, and say you need 10 h/month, which is about 1/16 of a person-month. And if things go horribly wrong, everybody drops what they are doing to fix things, anyway, no matter if you're on AWS, dedicated/colo or run your own data center.
When you significantly change your architecture/deployment, then you need to put in more time again, but if you build your code with need to scale and such things in mind from the get-go, then that won't come up much or at all.
What?! Maybe if you hire SV-skilled engineers on location in Sillicon Valley, but you can easily serve 2GB/second (on infra worth $2K/month) with one sys-admin dealing with it, and for way less than a whopping $10k/month.
If you are worried about getting your blog post to the top of Reddit or Hacker News (I've never been there myself); you can have a very modest web server or even a pay per request serverless sort of thing and pay $20 real quick to Cloudflare if you happen to get popular. It's the Bart Simpson method of highly scalability[0], for static content you can have global datacenter coverage in a couple minutes or so if you use them for DNS to start with. It even works if the origin server goes down.
> you can have a very modest web server or even a pay per request serverless sort of thing and pay $20 real quick to Cloudflare if you happen to get popular.
I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
$20/month for Cloudflare Pro is a steal for what you get. Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
> I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
You’re talking past the point here. It doesn’t matter how cheap if you’re fundamentally opposed to enabling cloud flare to reach its meat hooks further into the Internet.
This is no different from arguments about embedding google analytics or “just paying for windows” instead of using Linux.
> Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference? most of that group could probably do a lot of other optimizations that probably count for more.
I've had my personal $5/mo Digital Ocean VPS Wordpress site hit the top of HN before. I kept an eye on htop, but it handled it just fine. Exciting times.
As long as you have a decent backend it's no problem. If you're using some python/ruby/JS thing you're probably going to need some kind of reverse proxy to keep up with top of HN. If you're using a Haskell/Rust/C++/etc. compiled backend you're probably fine.
$20 a month for a Static site? that feels like a first world problem. I feel like everything is over engineered, unless you need subsecond delivery of your assets which are very huge, cdn doesn’t even makes sense. For a blog whats the point, your site is not going to receive heavy traffic every hour from global locations everyday. If you are concerned about returning users cache your assets on their machine. If your site is video heavy I can understand. I run a Django site for a zoo on a $10 instance, I have 2 others running on same instance. Never had issue with page speed even on 2G or under load my instance didn’t suffer. My storage on s3 is proxied via Nginx and cached on user device and in Nginx, I never even had a downtime due to traffic. I use fail2ban for basic protection. If it comes to DDOS im behind cloudflare free tier. $20 per month for a blog? Lol.
You missed the point, it's "pay $20 real quick to Cloudflare if you happen to get popular". There is a generous free tier, and for years I have paid nothing to Cloudflare, but I serve 2GB total every month to visitors, YMMV.
As a designer, it's really easy for me to put something on Cloudflare... doing what HN does would take at least more than a few hours (and the knowledge) to set up properly.
I had a blog post on the frontpage of hn for more tht 20 hours and my cheap Vps for 3€ a month could handle it perfectly since my website is just a statically generated website with hugo.
> This website blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/ is currently offline. Cloudflare's Always Online™ shows a snapshot of this web page from the Internet Archive's Wayback Machine. To check for the live version, click Refresh.
I tried looking, and I find it very irritating that in that whole web page, there is not one single link in the menu/footer/sidebar to polyhaven.com home page so I can click and discover what it actually is. Not one.
This occurs on many company blogs as well operating under a subdomain like blog.whatever.com
To be clear, this is a very tangential and irrelevant nitpick and I understand it does not contribute to the content of the website itself.
Yes! - the same goes with support sites on different domains (support.whatever.com or whatever.zendesk.com etc) that appear in google results and don't have a link to the company.
I think this is a result of app stores enforcing links to external digital products thing. That's why all these support pages which need to be linked from app, don't have a single link to main site.
To the polyhaven.com home page? I don't think it did. They are all sub pages which adds another navigation step to see the root page, which is what I find irritating.
Not having a link forces people to navigate to the page on their own. For a lot of people that takes the form of a Google search. Lots of people googling specifically for your brand increases your search rankings.
I handle ~150TB and 26M page views for ~$500 by simply renting a few dedicated servers at hetzner. And if I didn't need quite a lot of processing power (more than average website), it would be much lower. I only need so many servers for the CPU power, not traffic.
This! Hetzner root servers, there is nothing else I know on the market that comes close in price/power ratio when you use the "Serverbörse". I run multiple low traffic websites, databases and a heavy traffic elk instance + my complete homelab in proxmox / docker for 48€ a month and there is even room for more. Highly recommended!
That sounds very cool, did you happen to write about it? I currently run all this at home using a dedicated computer but electricity here isn’t cheap and I’d like to move everything to proxmox (although I’m struggling to figure out how to move all these more or less manually set up docker compose services and lxc containers)
Does DO even offer dedicated servers? I have used their vps in the past. It was ok, no issues, but more expensive than hetzners vps or dedicated for large traffic/cpu needs.
> This website blog.polyhaven.com/how-we-handle-80tb-and-5m-page-views-a-month-for-under-400/ is currently offline. Cloudflare's Always Online™ shows a snapshot of this web page from the Internet Archive's Wayback Machine. To check for the live version, click Refresh.
> To avoid this problem in the future, I decided to splurge a bit and go for a cloud solution where I wouldn’t have to worry about reliability, performance, scaling or integrity ever again: Google Firestore.
> Google Firebase is nice and convenient, but it is quite expensive. We could investigate some other managed database options in future.
I've only seen people get annoyed with Firestore over time, and migrating out of it. People do end up worrying. At first, They seem to choose Firestore because it's strongly marketed and seems suitable for a new project. And then data modeling, high prices or scalability becomes a problem.
What is a good solution compared to firebase ? I mean for a new project, alone something like user handling, registration, email sending, recovering passwords seem to cost some effort. Also be able to react to events, is kind of trivial in firebase. So is it really a bad choise (for the beginning)? Later you can always try to optimize.
so the problem here is if you have all of those needs, then really soon in the life of the project you are going to need good tooling to handle them.
So choosing something which makes it "one-click" to set up but total madness to manage is a really short-term optimization, only worth it for a pure prototype which you will throw away no matter how successful.
If you know you need those things to reach success, then it is better to make the up-front investment to get good tools for those.
If you still want to go with a cloud provider, AWS Amplify has some interesting tooling. I've build products both against Amplify and Firestore (and Firebase). Yes, Firebase is a few days to a week faster to set up (integrated user management, as you say), but AWS gives more sophisticated control and is built around scripted deployments.
You pay for it, of course, and I'm not arguing AWS vs running your own server. I am saying if the choice is AWS or Firebase, that a few days researching the choice would give you knowledge you could use for launching the next 10 prototypes you have in mind.
This is undoubtedly better than most chuck-it-on-AWS types would achieve, but it's still quite a bit more expensive than could achieved with marginally more effort. They are already on Digital Ocean - if they used their managed database offering, they could slash their Firestore bill and get rid of their Argo bill, bringing the cost sub $200 (but - massively reduced returns on investment of effort).
Readit News
It cost less than $2K/Month.
The cloud is crazy expensive. Private servers are beasts, and they are cheap.
Of course, for this price, you don't have redundancy and horizontal scaling.
You also don't have to maintain and debug a system with redundancy and horizontal scaling.
> It cost less than $2K/Month.
The solution in this article is serving on the order of 100TB/month for $400/month including a high speed global CDN, their API and database servers being hosted reliably, and redundancy and backup being handled by someone else.
Your solution is hosting on the order of 1000s of TBs of month (ignoring the database and other aspects of this website), but the price is an order of magnitude higher. You’ve also given up all of the automatic redundancy and hands-off management, and you don’t have the benefit of a high speed global CDN.
But more importantly, you have significantly higher engineering and on-call overhead, which you’re valuing at $0.
If anything, that only makes Polyhaven’s solution sound more impressive.
> Of course, for this price, you don't have redundancy and horizontal scaling.
Which is a huge caveat. The global CDN makes a big difference in how the site loads in global locations. Maybe not a big concern if you’re serving of static files with a lot of buffering, but they have a dynamic website and a global audience and they said fast load times are important.
> You also don't have to maintain and debug a system with redundancy and horizontal scaling.
But you have to do literally everything else manually and maintain it yourself, which is far from free.
All of these alternative proposals that value engineering time at $0/hour and assume your engineers are happy to be on-call 24/7 to maintain these servers are missing the point. You pay for turnkey solutions so you don’t have to deal with as much. Engineers don’t actually love to respond to on call events. If you can offload many of your problems to a cloud provider and free up your engineers for a nominal cost, do it.
The entire team is composed of half of one dev.
I'm 100% sure it's way cheaper than anybody that has AWS on resume.
Of course, some things have to give, like the global CND, and some data guarantees.
Everything is a compromise. It's all depend of what is important for your project.
EDIT: also, my comment was not meant to oppose the article, but rather confirm the view that you should calibrate your setup to your project. Doing so will lead to great savings in hosting, and project complexity. A lot of projects don't need the cloud.
Labour isnt expensive if you're operating towards a minimum needed to function, and your systems are sufficiently operationally stable.
-But google/facebook/amazon...
-But uptime needs to be 99.999
-But everyone uses cloud
Most businesses are not a trading-market, have less then 100 peoples (aka you are probably not another amazon), and no bonus using a cloud/kubernetes etc.
But it's the same old story, in the 00's i used the ~same arguments against buying OracleDB ;)
I worked at a company once that, from higher up, said that they had to have five nines of uptime. We had some really good cloud engineers there (one guy set up a server / internet container for the military in Afghanistan; in hindsight he said they should've just sent a container of porn dvd's), and they really went to town. For that five nines uptime, you're already pretty much required to set up your infrastructure to use multiple availability zones, everything redundant multiple times, etc.
Of course, the actual software we wrote was just a bunch of CRUD services written in nodejs (later scala because IDK), on top of a pile of shit java that abstracted away decades of legacy mainframes.
Isn't AWS down like every two months for a few hours? That's far off the 99.999% mark. No one can guarantee 100% uptime and sometimes it's even better to have that under your control (eg. have a dedicated server and a backup one from different providers).
My point is that, if you want the highest possible uptime, you shouldn't rely on a single (cloud) provider.
All means, no end.
But no matter how logically convincing your arguments were, most of the time upper manglement just went on buying Oracle, right...?
Some stuff makes sense to put on iaas, dns often does for example.
I have some clients who use AWS and others who prefer colo and/or dedicated servers from traditional datacenters. The latter group can afford to over-provision everything by 3-4x, even across different DC's if necessary. DC's aren't yesterday's dinosaurs anymore. The large ones have a bunch of hardware on standby that you can order at 3 a.m. and start running deployment scripts in minutes.
What? You set up the deployment once, and then you only need to touch it when things go horribly wrong, which is every couple of months, or to make minor quick tweaks and run some updates. Let's be generous, and say you need 10 h/month, which is about 1/16 of a person-month. And if things go horribly wrong, everybody drops what they are doing to fix things, anyway, no matter if you're on AWS, dedicated/colo or run your own data center.
When you significantly change your architecture/deployment, then you need to put in more time again, but if you build your code with need to scale and such things in mind from the get-go, then that won't come up much or at all.
You don’t even need a single employee to manage a single server…
In the real world, once most hosting platforms are up and running, the maintenance overhead is pretty low.
Where? Costs vary hugely across the world
[0]https://www.youtube.com/watch?v=aKU3hMvD31w
I get the impression that a lot of the critics in this thread don't really understand Cloudflare, how cheap it is, or even the concept of CDNs in general.
$20/month for Cloudflare Pro is a steal for what you get. Spinning up a dedicated server in a single datacenter somewhere isn't going to give the same results, especially if your users are geographically distributed like in this case.
You’re talking past the point here. It doesn’t matter how cheap if you’re fundamentally opposed to enabling cloud flare to reach its meat hooks further into the Internet.
This is no different from arguments about embedding google analytics or “just paying for windows” instead of using Linux.
Maybe not, but is the target audience that shills out $20/month really the type of people who have optimized their site to such an extent that shaving 50ms off the request latency by having your edge cache geolocated is really the type of thing that makes the difference? most of that group could probably do a lot of other optimizations that probably count for more.
Deleted Comment
Dead Comment
Once cloudflare captures the web market we'll all pay back with interest. They are not a charity.
I tend not to realise when my site goes viral, as I'm based in Australia whereas my largest audience is in b the US (and I'm a bit of a Luddite!)
Deleted Comment
hm
This occurs on many company blogs as well operating under a subdomain like blog.whatever.com
To be clear, this is a very tangential and irrelevant nitpick and I understand it does not contribute to the content of the website itself.
Or so goes the theory anyway.
oh the irony
> Google Firebase is nice and convenient, but it is quite expensive. We could investigate some other managed database options in future.
I've only seen people get annoyed with Firestore over time, and migrating out of it. People do end up worrying. At first, They seem to choose Firestore because it's strongly marketed and seems suitable for a new project. And then data modeling, high prices or scalability becomes a problem.
So choosing something which makes it "one-click" to set up but total madness to manage is a really short-term optimization, only worth it for a pure prototype which you will throw away no matter how successful.
If you know you need those things to reach success, then it is better to make the up-front investment to get good tools for those.
If you still want to go with a cloud provider, AWS Amplify has some interesting tooling. I've build products both against Amplify and Firestore (and Firebase). Yes, Firebase is a few days to a week faster to set up (integrated user management, as you say), but AWS gives more sophisticated control and is built around scripted deployments.
You pay for it, of course, and I'm not arguing AWS vs running your own server. I am saying if the choice is AWS or Firebase, that a few days researching the choice would give you knowledge you could use for launching the next 10 prototypes you have in mind.