Currently bringing a similar kind of testing in to our workplace by writing our services to run on top of `madsim` [1]. This lets us continue writing async/await-style services in tokio but then (in tests) replace them with a deterministic executor that patches all sources of non-determinism (including dependencies that call out to the OS). It's pretty seamless.
The author of this article isn't joking when they say that the startup cost of this effort is monumental. Dealing with every possible source of non-determinism, re-writing services to be testable/sans-IO [2], etc. takes a lot of engineering effort.
Once the system is in place though, it's hard to describe just how confident you feel in your code. Combined with tools like quickcheck [3], you can test hundreds of thousands of subtle failure cases in I/O, event ordering, timeouts, dropped packets, filesystem failures, etc.
This kind of testing is an incredibly powerful tool to have in your toolbelt, if you have the patience and fortitude to invest in it.
As for Antithesis itself, it looks very very cool. Bringing the deterministic testing down the stack to below the OS is awesome. Should make it possible to test entire systems without wiring up a harness manually every time. Can’t wait to try it out!
[0]: https://sled.rs/simulation.html
[1]: https://github.com/madsim-rs/madsim?tab=readme-ov-file#madsi...
[2]: https://sans-io.readthedocs.io/
[3]: https://github.com/BurntSushi/quickcheck?tab=readme-ov-file#...
Are there public examples of what such a re-write looks like?
Also, are you working at a rust shop that's developing this way?
Final Note, TigerBeetle is another product that was written this way.
The drain is just crazy at this point.