Dave_Rosenthal (u/Dave_Rosenthal)

Dave_Rosenthal commented on The Fancy Rug Dilemma epan.land/essays/2025-8_F... · Posted by u/ericpan64

Sure, let's take a rug as an example. I don't think there is one breakpoint. I think there are a set of axis of quality you invest into, roughly sequentially, as you go up the price scale of objects:

- $50 - Something rug-shaped exists

- $100 - Durability

- $200 - Materials

- $500 - Comfort and design

- $1000 - Basic craftsmanship

- $2000 - Refinement of craft

- $5000 - Artistry & identity

- $10000 - Tradition

- $20000 - Mastery

- $50000 - Rarity/historical importance

- $100000+ - ?

Because most people don't cross-shop $20k rugs and $200 rugs, most people are focused on one or two aspects around their personal budget. The essayist mentioned being amazing by the craftsmanship and artistry (see scale above). A broke college student might just want something that holds up in their dorm room and see what materials it's made out of and comfort as meaningless and abstract. And a billionaire shopping for a rug for their office might take everything other than rarity/historical importance as a given and just be thinking about that.

I think there is a large cognitive bias to consider everything you can easily afford "tangible and important improvements" and everything you can't as "abstract"!

Dave_Rosenthal commented on FoundationDB: From idea to Apple acquisition [video] youtube.com/watch?v=C1nZz... · Posted by u/zdw

niwtsol · 25 days ago

Lol - to find your one comment in this thread wayyy down at the bottom was delightful. That demo you reference from techcrunch disrupt ( start of this video - https://youtu.be/Nrb3LN7X1Pg) - that is a great demo. I'm curious - is that an idea/moment in your life you look back on and has that staying power, something that randomly pops in your mind that fills you with pride, or not really?

Dave_Rosenthal · 25 days ago

We thought of that demo with like a week or two to go before the event when we realized we needed something more visceral than a blinking cursor :)

So, it stayed with the world because it went semi viral in some tiny circles. But it didn’t stay with me because I personally learned little from it.

The thing that stays with me is all the things I learned spending years trying to implement an API you can describe in ~30 seconds really well. It’s a rare opportunity to be able to go so deep on a problem in a career.

Dave_Rosenthal commented on FoundationDB: From idea to Apple acquisition [video] youtube.com/watch?v=C1nZz... · Posted by u/zdw

Nican · 25 days ago

Yeah. I read over that page, and everything seems sensible.

The only part that is not well explained is that "FoundationDB has been tested with databases up to 100 TB".

Dave_Rosenthal · 25 days ago

The documentation is woefully out of date, sadly. Despite the code being in active development no one is touching the public docs. Though I don’t know for sure, that limitation was probably written something like 10 years ago.

Dave_Rosenthal commented on FoundationDB: From idea to Apple acquisition [video] youtube.com/watch?v=C1nZz... · Posted by u/zdw

majestik · a month ago

I can't put my finger on it but there's a weird tension between the two Dave's in this video. Almost like Rosenthal is trying to impress or earn the praise of Scherer.

Is there a backstory between these guys / FDB?

Dave_Rosenthal · a month ago

Ha, well I met Scherer ~30 years ago in a high school math class and we’ve done three companies together, so you could say we’ve known each other for a bit :)

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

jbverschoor · 4 months ago

2014: Beta launch of the SQL Layer: https://news.ycombinator.com/item?id=7593388

2014: Download SQL Layer: https://web.archive.org/web/20141209200002/https://foundatio...

2013: They talk about the SQL Layer: https://web.archive.org/web/20131220201001/https://foundatio...

2013: And someone can't start the SQL Layer on windows: https://web.archive.org/web/20131222063330/http://community....

Dave_Rosenthal · 3 months ago

Well, I stand very much corrected since that's me in the video! I forgot that an early version ever made it out into the world.

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

mike_hearn · 4 months ago

If it's configurable that's very new. It definitely has been hard coded and unconfigurable by design the entire history of the project that I've known it. They just tell you to work around it, not tweak it for your app.

Transactions holding locks for too long are indeed a problem though in Oracle transactions can have priorities and steal each other's locks.

Dave_Rosenthal · 4 months ago

It looks like it might be knobs (which can be changed via config file) called MAX_WRITE_TRANSACTION_LIFE_VERSIONS and MAX_READ_TRANSACTION_LIFE_VERSIONS now (defined in ServerKnobs.cpp)? It's in microseconds and probably needs to be synced client and server.

I don't know the details know, but it was definitely configurable when I wrote it :) I remember arguing for setting it to a default of 30/60 seconds we decided against as that would have impacted throughput at our default RAM budget. I thought might have been a good tradeoff to get people going, thinking they could tune it down (or up the RAM) if they needed to scale up perf later.

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

jakemoshenko · 4 months ago

Can you though? The protocol is not very well documented and it seems to iterate rather rapidly with the server version that it aims to be compatible with.

Dave_Rosenthal · 4 months ago

You might be able to, but are definitely not supposed to. The client is conceptually "part of the cluster".

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

jbverschoor · 4 months ago

AFAIK, the SQL layer was available and released

Dave_Rosenthal · 4 months ago

There have been some 3rd party toy projects in the past years, but pretty sure nothing was ever released by FoundationDB/Apple relating to SQL (until this additional SQL interface on the Apple "record layer").

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

mike_hearn · 4 months ago

There's a library here that implements a lot of database features and can be used on top of any sorted transactional K/V store, with FoundationDB being an exemplar backend:

https://github.com/permazen/permazen

It's pretty sophisticated, but the author uses it in his own projects and then just open sources it without trying to build a community around it so you may have to dig in to see that. It gives object mapping, indexing, composite indexing, triggers, query building and so on.

It's not "hard" to implement this stuff per se but it's a lot of work, especially to build enough test coverage to be convincing. I used to be quite into this idea of FoundationDB layers and especially Permazen, which I think is easily one of the best such layers even though it's not well known. I even wrote a SQL binding using Calcite so you could query your Permazen object stores in FDB using SQL!

I will say though, that in the recent past I started a job at Oracle Labs where I ended up using their database in a project, and that kind of gave me a new perspective on all this stuff. For example: scaling. Like a lot of people who spend too much time on Hacker News I used to think Postgres was state of the art, that RDBMS didn't scale well by design, and if you wanted one that did you'd need to use something exotic like layers on a FoundationDB cluster. But no. FoundationDB scales up to a few hundred nodes at most, and Oracle RAC/ExaData clusters can scale up that far too. There are people storing data from particle accelerators in ExaData clusters. The difference is the latter is a full SQL database with all the features you need to build an app right there already, instead of requiring you to depend on questionably well maintained upper layers that are very feature-light.

One place this hits you immediately is joins. Build out an ExaData cluster and you can model your data naturally whilst joining to your heart's content. The DB has lots of ways that it optimizes complex queries e.g. it pushes down predicates to the disk servers, it can read cached data directly out of other node's RAM over RDMA on a dedicated backbone network, and a whole lot more. Nearly every app requires complex queries, so this is a big deal. If you look at FoundationDB layers, then, well:

https://github.com/permazen/permazen/issues/31

Now in the last few years FoundationDB added for a very, very simple kind of push-down predicate in which a storage server can dereference a key to form another key, but if you look closely (a) it's actually a layering violation in which the core system understands data formats used by the Record layer specifically so it messes up their nice architecture, (b) the upper layers don't really support it anyway and (c) this is very, very far from the convenience or efficiency of a SQL join.

Another big problem I found with modeling real apps was the five second transaction timeout. This is not, as you might expect, a configurable value. It's hard-coded into the servers and clients. This turns into a hugely awkward limitation and routinely wrecks your application logic and forces you to implement very tricky concurrency algorithms inside your app, just to do basic tasks. For example, computing most reports over a large dataset does not work with FoundationDB because you can't get a consistent snapshot for more than five seconds! There are size limits on writes too. When I talked to the Permazen author about how he handled this, he told me he dumps his production database into an offline MySQL in order to do analytics queries. Well. This did cool my ardour for the idea somewhat.

There are nonetheless two big differences or advantages to FoundationDB. One is that Apple has generously made it open source, so it's free. If you're the kind of guy who is willing to self-support a self-hosted KV storage cluster without any backing from the team that makes it, this is a cost advantage. Most people aren't though so this is actually a downside because there's no company that will sell you a support contract, and your database is the core of the business so you don't want to take risks there usually. The second is it supports fully serializable transactions within that five second window, which Oracle doesn't. I used to think this was a killer advantage, and I still do love the simplicity of strict serializability, but the five second window largely kills off most of the benefits because the moment you even run the risk of going beyond it, you have to break up your transactions and lose all atomicity. It also requires care to achieve full idempotency. Regular read committed or snapshot isolation transactions offer a lower consistency level, but they can last as long as you need, don't require looping and in practice that's often easier to work with.

Dave_Rosenthal · 4 months ago

Thanks for the insight. A $XX million exa-data system is no doubt impressive :)

> Another big problem I found with modeling real apps was the five second transaction timeout. This is not, as you might expect, a configurable value. It's hard-coded into the servers and clients. This turns into a hugely awkward limitation and routinely wrecks your application logic and forces you to implement very tricky concurrency algorithms inside your app, just to do basic tasks. For example, computing most reports over a large dataset does not work with FoundationDB because you can't get a consistent snapshot for more than five seconds!

I'm pretty sure that the 5-second transaction timeout is configurable with a knob. You just need enough RAM to hold the key-range information for the transaction timeout period. Basically: throughput * transaction_time_limit <= RAM, since FDB enforces that isolation reconciliation runs in memory.

But, the other reason that 5 seconds is the default is that e.g. 1 hour read/write transactions don't really make sense in the optimistic concurrency world. This is the downside of optimistic concurrency. The upside is that your system never gets blocked by bad-behaved long-running transactions, which is a serious issue in real production systems.

Finally, I think that the current "Redwood" storage engine does allow long-lived read transactions, even though the original engine backing FDB didn't.

Dave_Rosenthal commented on Foundation DB Record Layer SQL API foundationdb.github.io/fd... · Posted by u/fidotron

bognition · 4 months ago

I remember learning about FoundationDB a decade ago and being deeply impressed with what they built. Then it was acquired by Apple and went silent. Since then we've seen an explosion in new database storage layers. I'm curious is FoundationDB still the new hotness or has it been replaced by newer better technologies?

Dave_Rosenthal · 4 months ago

FoundationDB's original promise was to combine a distributed storage engine with stateless layers on top to expose a variety of useful data structures (including SQL). The company was acquired before it could release the layers part of that equation. Apple open-sourced the core storage engine a few years later so FDB has kind of had a second life since then.

In that second life, the vast majority of the powerful databases around the industry built on FoundationDB have been built by companies making their own custom layers that are not public. This release is cool because it's a rare case that a company that has built a non-trivial layer on top of FDB is letting that source code be seen.

The group to which the FoundationDB storage engine itself appeals is fairly narrow--you have to want to go deep enough to build your own database/datastore, but not so deep to want to start from scratch. But, for this group, there is still nothing like FoundationDB in the industry--a distributed ACID KV store of extreme performance and robustness. So, yes, it's still the hotness in that sense. (As others have mentioned, see e.g. Deepseek's recent reveal of their 3FS distributed filesystem which relies on FDB.)