We self-host sentry in Hetzner, but with a high-end server. 96c, 512gb. It ends up only costing around $300 a month, however with the scale of events that it processes, the managed version would be in the 10's of thousands.
The overhead at low volume is pretty high, but in the higher volumes (25M transactions/24h) it's a massive cost saving for us.
Edit:
There were just some initial headaches with needing to increase kafka partitions and add replications to the transaction processors, otherwise we didn't quite leverage the available compute and the backpressure would fill Redis up until OOM.
Same here with the community maintained Helm chart. Not the easiest thing but quite reasonable for almost two years now. This is for 50M transactions per month and we're seeing massive cost savings compared to SaaS at this volume as well.
For those interested in only errors, the self-hosted version recently introduced errors-only mode which should cut down on the containers.
Just be forewarned it doesn't seem to offer one iota of IAM, so whether or not one is "overpaying" for a cloud provider depends on what you're getting from them. If you mean "rent a machine," then likely. If you mean "have the machines heal themselves instead of Pagerduty waking me up" then reasonable people can differ about where that money is going
> It ends up only costing around $300 a month, however with the scale of events that it processes, the managed version would be in the 10's of thousands.
I think this is a repeated question but... are you considering the cost of the people managing the deployment, security oversight, dealing with downtime etc?
If you can keep the people doing all the things, they become cheaper over time. Because as your system settles and people become more competent, both downtime and effort required to mend these problems reduce dramatically, and you can give more responsibilities to the same people without overloading them.
I am the person, it's occasionally I log in to delete a log file that I just haven't setup to rotate. About once a month, apart from that, no intervention needed (so far).
I do have one major complaint though, in dotnet, the tracing/errors are always captured regardless of the sampling rate. So you end up with a lot more memory usage on high throughput/low memory services with no way to lower it.
There's a ticket now open to stop this, but it's still in progress.
Avoiding allocations when a transaction isn’t sampled should be pretty trivial. Only gotcha is that you’d still trace id propagation to tie errors together regardless of transactions. But tracing and transactions got decoupled a while ago so shouldn’t be a big problem.
I’ll leave a comment on ticket pointing to this post
This absolutely mirrors my experience. Sentry was straightforward to deploy years ago, but now seems like one of the more egregious offenders in the, 'self host-able but increasingly prohibitively complex by design' category.
As others have said, we've [0] found the only practical way to deploy this for our clients is Kuberentes + Helm chart, and that's on bare-metal servers (mostly Hetzner). It runs well if you can throw hardware and engineering time at it, which thankfully we can. But given the option we would love a simpler solution.
In our case we have a collection of Ansible roles we use for the purpose. We run databases using the Stackgres operator either using logical replication on local fast NVMe dives, on top of OpenEBS/Mayastor replicated block-storage.
But we specialise in this so that our clients don't have to. As much as I do actually love Kubernetes, the fact that the _easiest_ way to self-host Sentry is via Kubernetes is not a good sign. And choosing to spin up a Kubernetes cluster just to run Sentry would feel a lot like the lady who swallowed a fly[0].
When I posted this myself on Reddit, I said the following:
I've long held off on actually posting this article to a platform like this one (don't bash your competition and all that), but "isn't Sentry self-hosted?" _is_ one of the most asked questions I get, and multiple people have told me this blog-post actually explains the rationale for Bugsink better than the rest of the site, so there you have it.
Well done! I came to the same conclusion (with the exact same bewilderment steps) as I do love Sentry myself. I will definitely try Bugsink, it’s something i’ve been looking for ages.
Feedback on competition bashing: sometimes they deserve it, they should really just come out and say it: “open sourcing our stuff isn’t working for us, we want to keep making money on the hosting”, and that would be ok
We enable self-hosting because not everyone can use a cloud service (e.g. government regulation), otherwise we probably wouldn't even spend energy on it. We dont commercialize it at all, and likely never will. I strongly believe people should not run many systems themselves, and something that monitors your reliability is one such system. The lesson you learn building a venture backed company, and one that most folks miss: focus on growth, not cost-cutting. Self-hosting for many is a form of cost-cutting.
We do invest in making it easier, and its 100% a valid complaint that the entire thing is awful today to self-host, and most people dont need a lot of the functionality we ship. Its not intentional by any means, its just really hard to enable a tiny-scale use-case while also enabling someone like Disney Plus.
We also found the same problem as OP with self hosting sentry. Each release would unleash more containers and consume more memory until we couldn't run anything on the 32gb server except Sentry.
We looked at both GlitchTip and BugSink and only settled on GlitchTip as it was maintained by a bigger team. Feature wise they were quite similar and both good alternatives.
So far so good with GlitchTip.
And thanks Op for making BugSink, the more alternatives the better.
GlitchTip had replaced our Sentry (9.x, pre-Clickhouse madness). It was just a matter of updating DSN in a few Configmaps/Secrets, good to go from day one. The UI is a bit buggy and "resolve" doesn't always work, but it does 99% of what Sentry did with 10% of the effort to maintain a modern Sentry setup.
Although with Bugsink (which is what came out of this origin story of annoyance) I'm aiming for _even more_ simple (1 Docker container at minimum, rather than 4), 30 seconds up and running etc.
Hello, ping me (GlitchTip lead) if you want to collaborate. Your stack is also Django. I'd be open to simplifying - we could probably make Redis optional. I have an experimental script that runs celery and Django in one process. But I think Postgres is a must have. So that gets down to two. My hope is that GlitchTip works for super small use cases and scales with minimal thought required.
I saw the headline and wanted to make sure someone mentioned GlitchTip. It doesn't have all of the functionality of Sentry but has all of the functionality I need. We have been running it in production for a year with no problems. Given our small user base (<1000 users), Sentry did not make sense.
Also a fan of GlitchTip so far. I only recently (last month or so) started using it. I made a railway template for it and have been using it to monitor a handful sites. I used valkey and minio for storage. Makes it super easy to spool up an instance.
We run them both as we are evaluating glitchtip, but, at least for us, it has so many bugs vs sentry. But it's so much lighter so we try to stick with it.
The biggest issue I have with these solutions is indeed local debugging.
I use Sentry with most of my clients, and for effective debugging I need to spin my own Sentry in a Docker container which ends up being quite heavy on my machine especially when combined with Grafana and Prometheus.
I'm really unhappy with virtually all monitoring/telemetry/tracking solutions.
It really feels they are all designed to vendor lock you in their expensive cloud solutions and I really don't feel I'm getting my $s back at all. Worst of all those cloud vendors would rather add new features non-stop rather than honing what they currently have.
Their sales are everywhere, I've seen two different clients getting Datadog sales people join private Slacks to evangelize their products.
Both times I escalated to the CTO, both times I ended up suspecting someone in management had something to gain from pushing teams to adopt those solutions.
Sentry's sales team is incredibly aggressive. I've seen multiple colleagues hijacked for sales presentations over a few months. It would not surprise me at all if they just asked random employees to be added to the company slack, and even less if those people then just did it.
I can only commend the hustle on their part, but it does feel a little like a high pressure time share situation.
If you - or anyone reading this - ever ends up in a situation where we came off as aggressive send me a direct email and I will take care of it. This is not something we believe in at Sentry, and while you cant manage everything, its important to us that we never become "of those companies" like so many other successful companies become.
I still love Sentry, but it’s so enormous now that it isn’t practical to self-host for smaller businesses. A “small” alternative is always great to see!
I’m not sure how I feel about the license though (Polyform Shield, basically use-but-don’t-compete). It’s a totally valid choice – I just wish it would convert to FOSS at some point. (I understand the concern as I’ve had to make a similar decision when releasing https://lunni.dev/. I went with AGPL, but I’m still overthinking it :-)
Great ending, honestly. I hope that dev got fired and truly understood what he had done, and felt the appropriate amount of shame. Not for his error, because we all make mistakes, but for his hubris that allowed him to keep making that same mistake over and over, while insisting he was doing the right thing.
Ok so your dev can't be told off for bringing the stuff out (and for being a moron doing sync calls to a logging service) and this brought the company down
But it was a good call sending it to the cloud. Better than "my problem" it is something being "somebody else's problem"
The amount of shell script that needs to be executed to install is a bit of a no no for me. It also doesn't make sense to spin up a 16GB machine (minimum!) to track the errors on those 4-8GB VPS which are running my production services.
Readit News
The overhead at low volume is pretty high, but in the higher volumes (25M transactions/24h) it's a massive cost saving for us.
Edit:
There were just some initial headaches with needing to increase kafka partitions and add replications to the transaction processors, otherwise we didn't quite leverage the available compute and the backpressure would fill Redis up until OOM.
For those interested in only errors, the self-hosted version recently introduced errors-only mode which should cut down on the containers.
Bugsink's also quite scalable[0], but I wouldn't recommend it a 25M/day.
[0] https://www.bugsink.com/scalable-and-reliable/
Well, your homepage disagrees with this statement:
> Bugsink can deal with millions of events per day on dirt cheap hardware
Wow, that's really cheap. I'm seriously overpaying for my cloud provider and need to try Hetzner. I always assumed Hetzner was only European based.
FWIW, https://lowendbox.com/ is good fun for the former set of things, too
I think this is a repeated question but... are you considering the cost of the people managing the deployment, security oversight, dealing with downtime etc?
Disclosure: I'm a sysadmin.
There's a ticket now open to stop this, but it's still in progress.
Feel free to email - david at sentry
As others have said, we've [0] found the only practical way to deploy this for our clients is Kuberentes + Helm chart, and that's on bare-metal servers (mostly Hetzner). It runs well if you can throw hardware and engineering time at it, which thankfully we can. But given the option we would love a simpler solution.
[0]: https://lithus.eu
But we specialise in this so that our clients don't have to. As much as I do actually love Kubernetes, the fact that the _easiest_ way to self-host Sentry is via Kubernetes is not a good sign. And choosing to spin up a Kubernetes cluster just to run Sentry would feel a lot like the lady who swallowed a fly[0].
[0]: https://en.wikipedia.org/wiki/There_Was_an_Old_Lady_Who_Swal...
When I posted this myself on Reddit, I said the following:
I've long held off on actually posting this article to a platform like this one (don't bash your competition and all that), but "isn't Sentry self-hosted?" _is_ one of the most asked questions I get, and multiple people have told me this blog-post actually explains the rationale for Bugsink better than the rest of the site, so there you have it.
Feedback on competition bashing: sometimes they deserve it, they should really just come out and say it: “open sourcing our stuff isn’t working for us, we want to keep making money on the hosting”, and that would be ok
https://blog.sentry.io/building-an-open-source-service/
We enable self-hosting because not everyone can use a cloud service (e.g. government regulation), otherwise we probably wouldn't even spend energy on it. We dont commercialize it at all, and likely never will. I strongly believe people should not run many systems themselves, and something that monitors your reliability is one such system. The lesson you learn building a venture backed company, and one that most folks miss: focus on growth, not cost-cutting. Self-hosting for many is a form of cost-cutting.
We do invest in making it easier, and its 100% a valid complaint that the entire thing is awful today to self-host, and most people dont need a lot of the functionality we ship. Its not intentional by any means, its just really hard to enable a tiny-scale use-case while also enabling someone like Disney Plus.
In fact I did one last week, but it got only a fraction of today's article's traction... I'll try again in whatever the prescribed interval is :-)
https://glitchtip.com/
We also found the same problem as OP with self hosting sentry. Each release would unleash more containers and consume more memory until we couldn't run anything on the 32gb server except Sentry.
We looked at both GlitchTip and BugSink and only settled on GlitchTip as it was maintained by a bigger team. Feature wise they were quite similar and both good alternatives.
So far so good with GlitchTip.
And thanks Op for making BugSink, the more alternatives the better.
Although with Bugsink (which is what came out of this origin story of annoyance) I'm aiming for _even more_ simple (1 Docker container at minimum, rather than 4), 30 seconds up and running etc.
I use Sentry with most of my clients, and for effective debugging I need to spin my own Sentry in a Docker container which ends up being quite heavy on my machine especially when combined with Grafana and Prometheus.
I'm really unhappy with virtually all monitoring/telemetry/tracking solutions.
It really feels they are all designed to vendor lock you in their expensive cloud solutions and I really don't feel I'm getting my $s back at all. Worst of all those cloud vendors would rather add new features non-stop rather than honing what they currently have.
Their sales are everywhere, I've seen two different clients getting Datadog sales people join private Slacks to evangelize their products.
Both times I escalated to the CTO, both times I ended up suspecting someone in management had something to gain from pushing teams to adopt those solutions.
Killing flies with hammers and all, but since I really like my hammer I actually do all my local development with my full-blown error tracker too:
https://www.bugsink.com/blog/local-development/
I can only commend the hustle on their part, but it does feel a little like a high pressure time share situation.
david at sentry.io
I’m not sure how I feel about the license though (Polyform Shield, basically use-but-don’t-compete). It’s a totally valid choice – I just wish it would convert to FOSS at some point. (I understand the concern as I’ve had to make a similar decision when releasing https://lunni.dev/. I went with AGPL, but I’m still overthinking it :-)
Quite rare to hear this wise line these days. An I guess with AI coding assistant, this is only the beginning of this kind of horror story
But it was a good call sending it to the cloud. Better than "my problem" it is something being "somebody else's problem"