Readit News logoReadit News
Posted by u/prabhatsharma 3 years ago
Show HN: OpenObserve – Elasticsearch/Datadog alternativegithub.com/openobserve/op...
Hello folks,

We are launching OpenObserve. An open source Elasticsearch/Splunk/Datadog alternative written in rust and vue that is super easy to get started with and has 140x lower storage cost compared to elasticsearch. It offers logs, metrics, traces, dashboards, alerts, functions (run aws lambda like functions during ingestion and query to enrich, redact, transform, normalize and whatever else you want to do. Think redacting email IDs from logs, adding geolocation based on IP address, etc). You can do all of this from the UI, no messing up with configuration files.

OpenObserve can use local disk for storage in single node mode or s3/gcs/minio/azure blob or any s3 compatible store in HA mode.

We found that setting up observability often involved setting up 4 different tools (grafana for dashboarding, elasticsearch/loki/etc for logs, jaeger for tracing, thanos, cortex etc for metrics) and its not simple to do these things.

Here is a blog on why we built OpenObserve - https://openobserve.ai/blog/launching-openobserve.

We are in early days and would love to get feedback and suggestions.

gettodachoppa · 3 years ago
Hah, funny to see this on HN, I just tried this 3 days ago when I saw it being discussed on /r/selfhosted.

As someone running a homelab, and hadn't set up logging yet, it was a great find. I didn't have to learn and combine 3+ log technologies, it's just a single all-in-one monitoring server with web UI, dashboards, log filtering/search, etc. RAM usage of the Docker container was under 100MB.

prabhatsharma · 3 years ago
Glad you liked it. Yeah, resource usage is so low that some folks are running it in their homelab even on a raspberry-pi.
sureglymop · 3 years ago
How does this compare to Prometheus + Grafana + Loki?
adamm255 · 3 years ago
The amount of times I've started building an ELK stack in my home lab just to get pissed and trash it! Will be giving this a go.
gitowiec · 3 years ago
Once I wanted to use Data dog with our internal project. Unfortunately I couldn't use the trial, it ended when I was ill at home. So I asked to prolong the trial. Ohhh that was a big mistake. They came upon me with zoom meetings, with two persons from their side. Asking do many questions about everything made me angry. After our internal meeting we agreed that we will use another service. So I emailed Data dog that we will pass on their offer. And then emails started. I was like stalked about "let's do the meeting, tell us more for what do you need 3 weeks more of the trial .." and so on. That was worst corporate behaviour I experienced
amacneil · 3 years ago
Datadog sales were so annoying I banned their entire domain in google apps.

Then they started sending me emails from their personal gmail accounts…

https://www.linkedin.com/posts/adrianmacneil_sdrs-are-the-fr...

prabhatsharma · 3 years ago
Yeah, Reddit is filled with horror stories from Datadog. One thread here - https://www.reddit.com/r/devops/comments/zz4naq/datadog_i_do...
hamandcheese · 3 years ago
What differentiates you from the grafana stack? Grafana labs now have solutions for every pillar of observability (metrics, logs, traces), and that whole stack also touts cheap ingest (due to minimal upfront indexing and cheap blob storage) as one of its biggest selling points.
prabhatsharma · 3 years ago
Grafana stack is cool. It solved the problem around observability for the first time in a neat and light way. Think about it however, for someone setting up their observability stack they will need to setup 4 components: Grafana (dashboarding), Loki (logs), Mimir (Metrics) and Tempo (Traces). This is a lot of work and a lot of moving parts. It is also a lot of work to maintain and upgrade. Loki, Mimir and Tempo have their own querying languages and their own quirks. Good amount of stuff to learn. It is really hard for a small team to learn and maintain all of this. Junior engineers really struggle with this and seniors resent the time they have to spend on this. Loki also has issues around high cardinality, which is a problem for many teams.

We need something new and better.

We made sure to build a solution that is easy to setup and maintain. One single binary/container for single node setup and one stack to setup/upgrade/maintain for HA setup.

We also made sure that it is easy to use and learn. We allow standard SQL to be used for querying logs, metrics and traces - nothing extra to learn here. Metrics querying is supported using PromQL too. Drag and drop for creating panels and dashboards is there too :-).

Dashboarding is supported within the same stack. No need to setup Grafana or something else separately.

OpenObserve also offer functions that grafana stack does not offer. Give it a shot, you are going to love it.

Data is stored in open parquet format, allowing its usage even outside of OpenObserve if so desired.

Hope this helps.

vlovich123 · 3 years ago
One of the most annoying things of grafana is that panels can only be edited via the UI and the “changes” summary when you commit is near unusable with a bunch of auto generated garbage.

Having a simple language/YAML to define those panels and an easy way to preview them (eg paste in your file to preview the page, paste in a panel source to try it) in addition to editing via GUI and being able to copy changes back to revision control would be great

Too · 3 years ago
How does OpenObserve handle logs with high cardinality?

In Elasticsearch tuning these indices is one of the bigger operational burdens. Loki solves this by asking users to not put data in the indexed fields, instead brute force through it all.

Does OpenObserve have any other approach to this?

PlutoIsAPlanet · 3 years ago
Grafana log handling also isn't the best, especially when compared to New Relic etc.
valyala · 3 years ago
VictoriaMetrics core developer here.

OpenObserve looks very promising from the first sight! It prioritizes the same properties as VictoriaMetrics products:

- Easy setup and operation

- Low resource usage (CPU, RAM, storage)

- Fast performance

It would be interesting to compare the operation complexity, the performance and resource usage of OpenObserve for metrics vs VictoriaMetrics.

P.S. VictoriaMetrics is going to present its own log storage and log analytics solution - VictoriaLogs - at the upcoming Monitorama conference in Portland [1]. It will provide much smoother experience for ELK and Grafana Loki users, while requiring lower amounts of RAM, disk space and CPU [2].

[1] https://monitorama.com/2023/pdx.html

[2] https://youtu.be/Gu96Fj2l7ls

heliodor · 3 years ago
The folks at VictoriaMetrics have a track record of producing well-documented and easy-to-use software, and they're responsive.

It's easy to overlook these aspects but they will make all the difference to your team implementing the solution, so if you don't want to gamble, their products are a solid choice.

PS: I have no personal relation or connection with them. I am a user of VictoriaMetrics. Just want to point out things that matter but get ignored when choosing your software stack.

qxip · 3 years ago
I'm looking forward to learning more about VictoriaLogs! You guys always come up with elegant solutions and I'm sure this will be no exception.
e12e · 3 years ago
Sounds interesting!

Will you compare with qryn? Self-hosted sentry?

https://qryn.metrico.in/

https://develop.sentry.dev/self-hosted/

valyala · 3 years ago
VictoriaLogs will be easier to setup and operate than qryn and self-hosted Sentry. Single-node VictoriaLogs is a single small self-contained statically linked binary with zero external dependencies. It runs with optimal settings out of the box on any hardware - from Raspberry PI to high-end servers with hundreds CPU cores and terabytes of RAM.

Additionally, VictoriaLogs will provide more user-friendly query language - LogsQL, which is easier to use than SQL at ClickHouse or the query language provided by Grafana Loki.

samspenc · 3 years ago
This looks great, I had some questions about the company itself if that's OK, I'm just curious to understand the background:

At the bottom of your GitHub project home page, you say the best way to join the project is to join a WeChat group (in Chinese text), but likely only a very small minority of us outside China use WeChat, so that may be a stumbling block if you are trying to encourage people outside Asia to contribute to the project.

Per https://openobserve.ai/about , the address at the bottom says San Francisco, California, but in the same page it says "headquartered in Bangalore, India". So where are you based out of?

Also curious what the relationship is between OpenObserve the open-source project and Zinc Labs, which is referenced in the website (but not in the GitHub project).

prabhatsharma · 3 years ago
You can join Slack if you are outside of China, or wechat, if you are in China. We had some good early users whoBoth the options are available.

> headquartered in Bangalore, India This is embarrassing, just fixed it. We are a delaware based company, headquartered in San Francisco. Pure copy paste error.

Zinc Labs (Legal name) is the company behind the product OpenObserve.

lopkeny12ko · 3 years ago
How do you typo "San Francisco" into "Bangalore"? This seems extremely shady.

Copy and paste from where? Did you even build this product or just hire cheap labor out of India to build it?

aliencat · 3 years ago
It seems that the 140x lower storage cost comes from: 1. S3 (OO) vs EBS (ES): about 5x 2. No indexing: About 10x ? 3. No data duplication (due to using S3 I assume) in HA deployment: 3x

Is my math right? Or do you use something different for compression?

2 Orders of magnitude of storage saving is pretty impressive.

prabhatsharma · 3 years ago
You are right.
aliencat · 3 years ago
Thanks! Best of luck! Looks like a solid product
hamandcheese · 3 years ago
I poked around the user guide[0] and discovered this:

> Currently OpenObserve support steams of type logs, in future streams types metrics & traces will be supported.

- are metrics and traces queryable yet? I admit, I feel a little misled, if only logs are supported for now that should be made more clear.

- do (or will) metric and trace queries use a similar SQL syntax as log search?

Finally... is there a demo? I would love to be able to try out the product without actually putting in the effort to set it up.

[0]: https://openobserve.ai/docs/user-guide/streams/

prabhatsharma · 3 years ago
Here is a quick demo - https://www.youtube.com/watch?v=fZ-ErfMdF-o . Covers basics, logs, function, dashboards. Does not cover traces and metrics.
prabhatsharma · 3 years ago
I should fix it. Metrics and traces are supported today. Logs and traces are advanced.

Metrics is still in its infancy though.

prabhatsharma · 3 years ago
Logs, metrics and traces are all queryable using SQL. Metrics can additionally use PromQL for querying.
prabhatsharma · 3 years ago
You could also try the cloud version of OpenObserve without setting it up yourself.
prabhatsharma · 3 years ago
Logs, Metrics and traces are all queryable.
SergeAx · 3 years ago
Interesting product, thank you for your effort, definitely want to give it a try!

For me, thought, setting up a system is not the primary pain point today. FWIW, signing up for a cloud service is not hard.

The problem starts at the ingestion point. I am writing my apps according to 12 factors and running them in docker containers. What I want is an agent companion that will collect logs and metrics from these apps and containers and forward to the system. Datadog and Grafana has that, do OpenObserve?

Also, interesting that in your quickstart tutorial you have a step of unzipping logs file before sending to the system. I would suggest the ability to send them in (g)zipped format, as it is an organic format of keeping logs.

prabhatsharma · 3 years ago
Got your point. We think that there are 3 really good agents out there that can be leveraged by anyone to capture and send telemetry data. These are Vector, fluentbit and Otel-collector. Between these 3 they can handle pretty much anything. If there is something that needs to be handled that these cannot handle then its easy enough to build plugins for fluentbit and otel-collector.

We did not want to reinvent the wheel and recommend that you use one of these.

quickstart tutorial does not reflect what you will do in real life scenario.

You will want to learn how to use log forwarders like fluentbit and vector. These are the ones that you will generally use in real life. These log forwarders read the log files and send them to OpenObserve. Here is a blog on how to use fluentbit to capture kubernetes logs using fluentbit and send them to OpenObserve - https://openobserve.ai/blog/how-to-send-kubernetes-logs-usin...

We will add more tutorials and examples soon.

Hope this helps.

edenfed · 3 years ago
Hi SergeAx, I am building Odigos, which do exactly what you asked :) We combine OpenTelemetry and eBPF to automatically generate and deliver distributed traces, metrics and logs to Grafana, DataDog and other 15+ destinations. Check it out here: https://github.com/keyval-dev/odigos
SergeAx · 3 years ago
Hi, great to meet another builder here on HN) Am I understand correctly that your product is like grafana_agent, but vendor-agnostic?