bruth (u/bruth) - Readit News

bruth commented on TigerBeetle and Synadia pledge $512k to the Zig Software Foundation tigerbeetle.com/blog/2025... · Posted by u/jorangreef

webdevver · 2 months ago

i asked gemini to investigate on my behalf, and it basically said that Synadia is to NATS what HiveMQ is to MQTT. seems fair enough.

thank goodness for llms to spare us the marketing drivel!

bruth · 2 months ago

Unfortunately the LLM is leading you astray :)

bruth commented on TigerBeetle and Synadia pledge $512k to the Zig Software Foundation tigerbeetle.com/blog/2025... · Posted by u/jorangreef

lxe · 2 months ago

I'm absolutely struggling to understand what Synadia even does. it's been an infuriating experience navigating through their marketing site. It's been minutes and I still don't understand what it is. What is NATS?

Edit: I had to google what NATS.IO is. The marketing site is infuriatingly useless. Please, can we stop doing this?

Edit: At the bottom on the footer it says compare NATS to Kafka. It took me to a page that requires me to enter my email, name, and a message in order to download the white paper. I flipped over my desk in rage.

bruth · 2 months ago

(disclaimer: I am the VP of Prod/Eng at Synadia)

NATS - An application connectivity technology (L7). It was originally designed for low-latency M:N messaging, and that is still true today. In 2018, native multi-tenancy, clustering options, etc. got introduced. The persistence subsystem (JetStream) was introduced in 2021. It has a completely different design than Kafka, but with overlapping use cases. For better or worse, we get compared to Kafka a lot and virtually everyone who engages realizes the advantages and opportunities. NATS is much more flexible for application use cases, for example, it provides KeyValue and ObjectStore abstractions on top of the core persistent stream. There are a plethora of other details, but that is the basic gist. Overall, it has a lot of batteries included for building everything from embedded applications to large scale cloud-to-edge systems.

Synadia - The founder (Derek) created NATS. We are the primary maintainers of the project. We build products and services on top of NATS including a global SaaS where you can sign up and use "hosted NATS" with additional features. We offer a BYOC model, one of which we manage for you, or a Kubernetes-based self-service one that you deploy yourself. We also support fully self-hosted for customers that need to run in their own data centers or at the edge.

Regarding the comment re: the website, there are improvements we have in the works. Happy to engage and help clarify anything that is confusing.

bruth commented on Generative AI Is Not Going to Build Your Engineering Team for You simonwillison.net/2024/Ju... · Posted by u/duck

saberience · 2 years ago

SimonW is the king of hacker news clickbait these days.

He basically chooses the most clickbaity headline possible and then posts it up to hacker news. The actual content itself is almost always totally banal and adds nothing new or insightful to the world.

No one was arguing that AI was going to build an engineering team, I haven’t read or heard or seen this take from anyone.

Taking up this controversial notion as the premise of an article (as though it’s actually a thing), when no one was positing it in the first place, is a classic way of driving traffic to your blog.

bruth · 2 years ago

I don't know Simon personally, I do know he has been blogging for a very long time and many of his posts are for his own benefit of recall, not necessarily intended for an external audience. Also He did not post this, a different person did.

bruth commented on Show HN: Hatchet – Open-source distributed task queue github.com/hatchet-dev/ha... · Posted by u/abelanger

abelanger · 2 years ago

I'm assuming specifically you mean Nex functions? Otherwise NATS gives you connectivity and a message queue - it doesn't (or didn't) have the concept of task executions or workflows.

With regards to Nex -- it isn't fully stable and only supports Javascript/Webassembly. It's also extremely new, so I'd be curious to see how things stabilize in the coming year.

bruth · 2 years ago

(Disclaimer: I am a NATS maintainer and work for Synadia)

The parent comment may have been referring to the fact that NATS has support for durable (and replicated) work queue streams, so those could be used directly for queuing tasks and having a set of workers dequeuing concurrently. And this is regardless if you would want to use Nex or not. Nex is indeed fairly new, but the team on is iterating on it quickly and we are dog-fooding it internally to keep stabilizing it.

The other benefits of NATS is the built-in multi-tenancy which would allow for distinct applications/teams/contexts to have an isolated set of streams and messaging. It acts as a secure namespace.

NATS supports clustering within a region or across regions. For example, Synadia hosts a supercluster in many different regions across the globe and across the three major cloud providers. As it applies to distributed work queues, you can place work queue streams in a cluster within a region/provider closest to the users/apps enqueuing the work, and then deploy workers in the same region for optimizing latency of dequeuing and processing.

Could be worth a deeper look on how much you could leverage for this use case.

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

Kinrany · 2 years ago

It really doesn't help that Slack requires an account, isn't searchable and doesn't preserve history :)

bruth · 2 years ago

If a learning resource is lacking or confusing, Slack or GitHub issues/discussions is a way to engage and provide feedback so they can be improved. If the docs were confusing, there are other channels to get help to unblock folks. The outcome of that interaction would lead to improvements in the docs.

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

lakomen · 2 years ago

Regarding middleware, last time I tried there was a lot of information to process. Now I tried again and

  func loggingMiddleware(next micro.Handler) micro.Handler {
 return micro.HandlerFunc(func(req micro.Request) {
  spew.Dump(req.Data())
  next.Handle(req)
 })
  }

I'll remove the middleware part. EDIT: Seems I can't edit the post anymore.

Regarding auth, yes, spectrum of options, but for "production" use it's very complicated and you don't want a shared key, you need the whole operator, account, user thing, and external auth aka auth callout. The documentation is lacking. nkey, xkey what? It's confusing is complicated. Managing auth is a pain, more than once I've been locked out of my test server because of how complicated auth is and had to wipe it. It's so complicated because you have have isolated "namespaces" or operator spaces? Naming an organization operator, or a solution or service group doesn't help. user is synonymous with client. All that you don't get from the documentation, you have to watch Youtube videos to understand it. The documentation is bad.

How do I consume a stream from the deno/ws client without the whole client app blocking? No idea. The code snippets don't help. All there is, is a pay for company that offers to host NATS clusters, which you can pay to ask. Slack is not a good medium.

I'm guessing that the documentation is bad so one has to pay for support. And that's a no-go for me. I get that you spent time and effort into it, I'm assuming you're one of the NATS people, but what good is it when I don't know how to effectively use the software? The natsbyexample page is also hard to digest.

You're in your own little bubble and expect people to be mind readers.

If it's too complicated people won't adopt it. I know I won't because it's too complicated.

I would've loved to build upon it, because the core functionality is nice and works, easy to grasp.

I've not seen a single Jetstream stream and consumer tutorial with the new API and nats.ws and Go. Old API sure, STAN sure. New API no. Some real world stuff, not just code demo snippets. Why not? Because it's complicated.

Auth callout, that was the big thing everyone was waiting for? I would work on actually solving problems instead of building components so I can use this extra piece of software who's documentation is so bad.

You know how many Kafka tutorials there are? A LOT.

I want to like it, it seems very promising, but it suffers from the "coder no likey documentation" illness.

Today I saw a Twitch stream of some Netflix dev, talking about DX, developer experience.

NATS' DX has a lot of room for improvement, to put it friendly.

bruth · 2 years ago

> You're in your own little bubble and expect people to be mind readers.

Out of curiosity, have you asked questions in Slack or Github? If so, and you had a bad experience with the interaction, I get the sentiment and would offer help. But this comment is not constructive without context.

> I'm guessing that the documentation is bad so one has to pay for support.

The NATS project has been open source for around 12 years, and part of the CNCF since 2018. This is an incorrect statement and very poor assumption to make because the documentation doesn't make sense for you.

I 100% agree it can be improved and we are working on a new docs site, but it is not quite ready.

In case its helpful, there is an increasing collection of examples on https://natsbyexample.com with new JetStream client examples among others. If you have specific requests, feel free to open issues in the corresponding repo: https://github.com/ConnectEverything/nats-by-example

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

zinclozenge · 2 years ago

I was hoping the blog post would include a tentative roadmap for adding tiered storage.

bruth · 2 years ago

This post is about a product for NATS.

I presume you are talking about the roadmap after the 2.10 release?

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

wtatum · 2 years ago

Thanks that observation is extremely helpful. If I have it down then the intended flow looks something like?

- Web client is directed to some token vending service. This service implement authn in a manner of its choosing (i.e. OAuth) then sets a NATS client JWT in the cookie per https://docs.nats.io/running-a-nats-service/configuration/se... - Nats.ws client connection provides cookie during connection to perform client auth - If further authz/fine-grained control is needed the auth callout mechanism can be used. This would have access to the provided cookie/token so any claims needed for access control could be stapled on during step one and used at this point?

For GPs original question -- I'm running a fairly old Keycloak version (v8) but it does appear to set a JWT in KEYCLOAK_IDENTITY and KEYCLOAK_IDENTITY_LEGACY.

Am I right in understanding that IFF the token is signed with Ed25519 and both sub and iss are an NKEY value this is sufficient for NATS to accept that cookie as a credential?

bruth · 2 years ago

Yes that reads correct. The `sub` would a NATS user public nkey, the `iss` would be the NATS account public nkey (either the issuer nkey in config-mode or existing nkey in decentralized auth).

As long as it can verify the chain of trust for the user JWT that is returned, it should work.

The three schema types are shown here: https://docs.nats.io/running-a-nats-service/configuration/se...

auth request comes in -> generate user jwt, sign + encode -> respond with auth response.

As long as the necessary bits of the response and user JWT conform, it will work.

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

wtatum · 2 years ago

Same. I tried out some toy use-cases using nats.js over websocket a few months ago. The prospect of possibly being able to "directly" consume messaging or key/value store from the browser with only a thin gateway between was really interesting to me but I couldn't square up the NATS-internal JWT cookie thing with how you would handle auth in a traditional web-app (OAuth client on a gateway plus a session cookie).

I found some threads saying auth callout in 2.10 would solve this and decided to table the project until 2.10 but it's really really unclear how to work through the details of converting a "traditional" OAuth access token into the NATS-specified access token required by the auth callout contract.

bruth · 2 years ago

Auth callout is post-NATS client authentication, so it would not solve the "auth web flow" for authentication. Instead, the resulting token from that would be set as a cookie that then would be passed into the nats.ws client connection. The auth callout service would use that token to map to the concrete NATS user. The mechanism of doing that is up to the implementation. One option is to manage NATS claims into the OIDC provider (for the user authenticating) and then the auth service would decode that source JWT and extract the NATS claims and generate the NATS user JWT in the response.

bruth commented on Stream Compression in NATS qaze.app/blog/nats-stream... · Posted by u/SebastianM

lakomen · 2 years ago

Honestly, I'm more interested in auth callout, someone making a "how to use this stuff effectively" video and/or article, for dummies, ideally with keycloak.

If you look at [0] how does this explain anything? Seems like I need to write a service utilizing NATS which talks to the OIDC server.

NATS auth story is a complicated one, and now with auth callout it's even more complicated.

What I also quite didn't succeed in were JS client Jetstream consumers, async polling or w/e. All the examples, async or sync would block execution.

The regular request/response with microservices went well and straightforward. No issues at all. But writing middleware is also not very pretty, you have to have dedicated functions which you would call in the request handler (or microservice if you will) instead of being able to define a chain of middleware functions. Potejto potato but organization and structure and naming is half the code.

And more now you can't rely on http error codes, even if you can use them.

Then debugging is difficult, because messages are sent in binary format, and with no middleware to conditionally plug in, it's even more painful. Yeah you could write a dedicated service listening to >.* and printing deserialized JSON payloads but that's also extra work.

However dropping the HTTP overhead leads to a lot more throughput.

For now I'll stay with HTTP.

The reason I experimented with it was there is a company which required NATS knowledge.

[0] https://docs.nats.io/running-a-nats-service/configuration/se...

bruth · 2 years ago

> Seems like I need to write a service utilizing NATS which talks to the OIDC server.

Auth callout was designed to be a generic extension point to delegate authentication and generate dynamic a user JWT that NATS understands (permissions, limits, etc). It enables an arbitrary backend to be integrated with, not tied specifically to OIDC. But indeed, this requires implementing a service that does this integration.

> NATS auth story is a complicated one, and now with auth callout it's even more complicated.

There is a spectrum of auth options, starting with simple config-based, token or user/pass leading up to decentralized auth for use cases that need it. Auth callout is an opt-in thing, so it should only be adopted if it is truly necessary.

> you have to have dedicated functions which you would call in the request handler [...] instead of being able to define a chain of middleware functions

I don't quite understand this statement. Wrapping a NATS handler is the same approach as wrapping an HTTP handler (within the same client app). The function would take a handler and return a handler. There can be inspection of the message within that function and the choice of calling the next handler, responding early, doing some external call, or doing nothing.

If you want to distribute this middleware, then you would need to have separate, explicit subjects that each middleware would subscribe to and then publish to for the next element in the chain (for a choreography approach).

There is also the "message slip pattern" where the ingest component sets the path as metadata (e.g. headers) that each middleware component uses that to know (at request time) which subject to publish the result to next.