tqwewe (u/tqwewe) - Readit News

tqwewe commented on SierraDB: A distributed event store built in Rust tqwewe.com/blog/building-... · Posted by u/tqwewe

conceptme · 5 months ago

Does it also have snapshot capabilities, as mostly over time it becomes very difficult to replay events due to the shear amount of them.

tqwewe · 5 months ago

I haven't put any effort into any kind of snapshotting capabilities yet, since I won't want the scope to be too large and there's often ways of designing your system in ways where replaying isn't a big issue (the db scans are fast, designing aggregates to have a smaller scope/less overall events, etc).

But with increased resources, I can see some solutions being considered around snapshotting. For now the goal is heavily unix philosophy inspired: a really fast and purpose built event sourcing database

tqwewe commented on SierraDB: A distributed event store built in Rust tqwewe.com/blog/building-... · Posted by u/tqwewe

12_throw_away · 5 months ago

Very excited to see how this progresses! Honestly, it's always a little surprising to me that event store architectures aren't more widely used. The article is extremely correct about why:

> there's absolutely no clear cut way of approaching it for new projects

That's definitely my experience - there's no open source "batteries-included" event store where you can just `docker compose up` and start sourcing your events right away. (Maybe KurrentDB née EventStoreDB might offer something like this? But unfortunately, it has a weird license and feels heavily pivoted towards SaaS). And if you want to implement it yourself, a lot of the writing about event stores comes from the the Enterprise Design Patterns™ world.

tqwewe · 5 months ago

Yes exactly, I heard from an existing KurrentDB customer that the weird licensing change was actually a deal breaker causing them to move away from KurrentDB despite the migration pains.

I think a community, open source built project in Rust has been a missing piece I can hopefully help to solve.

tqwewe commented on Show HN: Kameo – Fault-tolerant async actors built on Tokio github.com/tqwewe/kameo... · Posted by u/tqwewe

throwawaymaths · a year ago

Thanks! It's not in the front page material.

tqwewe · a year ago

This is resolved now :) Added better material around distributed actors.

tqwewe commented on Show HN: Kameo – Fault-tolerant async actors built on Tokio github.com/tqwewe/kameo... · Posted by u/tqwewe

wildlogic · a year ago

Hi - any documentation regarding actor registration? Is there a conventional way to inform a remote actor about a new actor? Would this be sent in a message? How does the actor.register('name') work? Maybe could be a useful addition to the documentation. Thanks.

tqwewe · a year ago

I've added an indepth section to the kameo book about actor registration and lookup, including how it works: https://docs.page/tqwewe/kameo/distributed-actors/registerin...

tqwewe commented on Show HN: Kameo – Fault-tolerant async actors built on Tokio github.com/tqwewe/kameo... · Posted by u/tqwewe

tqwewe · a year ago

Thanks! Its definitely missing, I'll need to add that perhaps to the kameo book.

Its using libp2p under the hood with Kademlia distributed hash table for actor registrations.

tqwewe · a year ago

I've added an indepth section to the kameo book about distributed actors if you'd like to read more. https://docs.page/tqwewe/kameo/distributed-actors

tqwewe commented on Show HN: Kameo – Fault-tolerant async actors built on Tokio github.com/tqwewe/kameo... · Posted by u/tqwewe

snowboarder63 · a year ago

Thanks! I'm happy to see actors getting some solid use in the industry to provide better thread-management safety and remove a lot of concurrency headaches.

Question for you, I was poking around in the codebase and how do you handle your Signal priorities? Like if a link died, and there's 1000 messages in the queue already, or if it's a bounded mailbox, would the link died (or even stop) messages be delayed by that much?

Have you looked into prioritization of those messages such that it's not globally FIFO?

tqwewe · a year ago

Great question, I did some digging into the source code of beam to help answer if signals should have special priority, and the conclusion (with the help of someone else from the elixir community) was that signals have no special priority over regular messages in beam. So I decided to take this same approach, where a regular message is just a `Signal::Message(M)` variant, and everything sent to the mailbox is a signal.

So gracefully shutting down an actor with `actor_ref.stop_gracefully().await` will process all pending messages before stopping. But the actor itself can be forcefully stopped with `actor_ref.kill()`

tqwewe commented on Show HN: Kameo – Fault-tolerant async actors built on Tokio github.com/tqwewe/kameo... · Posted by u/tqwewe

snowboarder63 · a year ago

While the state is indeed a separate struct in ractor there's actually a good reason for this. It's because the state is constructed by the actor and it's guaranteed that the construction is the state is managed by the startup flow and panic safe.

Imagine opening a socket, if you have a mutable self the caller who is going to spawn that actor needs to open the socket themselves and risk the failure there. Instead of the actor who would eventually be responsible for said socket. This is outlined in our docs the motivation for this. Essentially the actor is responsible for creation of their state and any risks associated with that.

- for the async trait point we actually do support the native async traits without the boxing magic macro. It's a feature you can disable if you so wish but it impacts factories since you can't then box traits with native future returns https://github.com/slawlor/ractor/pull/202

(For transparency I'm the author of ractor)

tqwewe · a year ago

Thanks for the reply! The example you gave does make sense regarding the stage being used with the actors startup method.

I wasn't aware async_trait wasn't needed, thats nice to see.

Also congrats on it being used in such a big company, thats awesome! I have a lot of respect for ractor and appreciate your response