qianli_cs (u/qianli_cs)

qianli_cs commented on Oban, the job processing framework from Elixir, has come to Python dimamik.com/posts/oban_py... · Posted by u/dimamik

sorentwo · 16 days ago

There are other projects that implement the ideas in OSS, but that's the same in Elixir. Not that we necessarily invented DAGs/workflows, but our durable implementation on the Elixir side predates DBOS by several years. We've considered it an add-on to what Oban offers, rather than the entire product.

Having an entirely open source offering and selling support would be an absolute dream. Maybe we'll get there too.

qianli_cs · 16 days ago

That's fair, the idea itself isn't new. Workflows/durable execution have been around forever (same story in Elixir).

The differences are in the implementation and DX: the programming abstraction, how easy recovery/debugging is, and how it behaves once you're running a production cluster.

One thing that bit us early was versioning. In practice, you always end up with different workers running different code versions (rolling deploys, hotfixes, etc.). We spent a lot of time there and now support both workflow versioning and patching, so old executions can replay deterministically while still letting you evolve the code.

Curious how Oban handles versioning today?

qianli_cs commented on Oban, the job processing framework from Elixir, has come to Python dimamik.com/posts/oban_py... · Posted by u/dimamik

qianli_cs · 16 days ago

Thanks for sharing, interesting project! One thing that stood out to me is that some fairly core features are gated behind a Pro tier. For context, there are prior projects in this space that implement similar ideas fully in OSS, especially around Postgres-backed durable execution:

1. DBOS built durable workflows and queues on top of Postgres (disclaimer: I'm a co-founder of DBOS), with some recent discussions here: https://news.ycombinator.com/item?id=44840693

2. Absurd explores a related design as well: https://news.ycombinator.com/item?id=45797228

Overall, it's encouraging to see more people converging on a database-centric approach to durable workflows instead of external orchestrators. There's still a lot of open design space around determinism, recovery semantics, and DX. I'm happy to learn from others experimenting here.

qianli_cs commented on Lambda Durable Functions docs.aws.amazon.com/lambd... · Posted by u/john-shaffer

qianli_cs · 2 months ago

This post seems to be published in a hurry. Under "How it works" section a bunch of duplication, and I think they should make the blog post exactly once :) Excerpt from the blog post:

> During replay, your code runs from the beginning but skips over completed checkpoints, using stored results instead of re-executing completed operations. This replay mechanism ensures consistency while enabling long-running executions. > > ... During replay, your code runs from the beginning but skips over completed checkpoints, using stored results instead of re-executing completed operations. This replay mechanism ensures consistency while enabling long-running executions.

qianli_cs commented on Building a Durable Execution Engine with SQLite morling.dev/blog/building... · Posted by u/ingve

qianli_cs · 3 months ago

I really enjoyed this post and love seeing more lightweight approaches! The deep dive on tradeoffs between different durable-execution approaches was great. For me, the most interesting part is that Persistasaurus (cool name btw) use of bytecode generation via ByteBuddy is a clever way to improve DX: it can transparently intercept step functions and capture execution state without requiring explicit API calls.

(Disclosure: I work on DBOS [1]) The author's point about the friction from explicit step wrappers is fair, as we don't use bytecode generation today, but we're actively exploring it to improve DX.

[1]: https://github.com/dbos-inc

qianli_cs commented on Show HN: DBOS Java – Postgres-Backed Durable Workflows github.com/dbos-inc/dbos-... · Posted by u/KraftyOne

rtcoms · 3 months ago

Do you have any plans to support Ruby ?

qianli_cs · 3 months ago

Yeah, in the long term, we're supporting more languages. But we aren't actively building new languages right now.

qianli_cs commented on Show HN: DBOS Java – Postgres-Backed Durable Workflows github.com/dbos-inc/dbos-... · Posted by u/KraftyOne

nogridbag · 3 months ago

Are there any plans for supporting other databases? Our company primarily uses and has experience managing MySQL deployments. I evaluated Temporal some time ago as it seemed like a good fit for what we're building so I'm watching this closely. Thanks!

qianli_cs · 3 months ago

To build on what Peter said, we need to stay focused and make one backend solid before expanding. But architecturally, nothing prevents us from supporting more engines going forward.

qianli_cs commented on Make any TypeScript function durable useworkflow.dev/... · Posted by u/tilt

qianli_cs · 4 months ago

I'm excited about this because durable workflows are really important for making AI applications production ready :) Disclaimer: I'm working on DBOS, a durable workflow library built on Postgres, which looks complementary to this.

I asked their main developer Dillon about the data/durability layer and also the compilation step. I wonder if adding a "DBOS World" will be feasible. That way, you get Postgres-backed durable workflows, queues, messaging, streams, etc all in one package, while the "use workflow" interface remains the same.

Here is the response from Dillon, and I hope it's useful for the discussion here:

> "The primary datastore is dynamodb and is designed to scale to support tens of thousands of v0 size tenants running hundreds of thousands of concurrent workflows and steps."

> "That being said, you don't need to use Vercel as a backend to use the workflow SDK - we have created a interface for anyone to implements called 'World' that you can use any tech stack for https://github.com/vercel/workflow/blob/main/packages/world/..."

> "you will require a compiler step as that's what picks up 'use workflow' and 'use step` and applies source transformations. The node.js run time limitations only apply to the outer wrapper function w/ `use workflow`"