Posix file system semantics are very complex. An in memory implementation is likely to have quality gaps that make it sub optimal. If you want fast tests /tmp is likely to be backed by tmpfs in memory. If you are paranoid you can use /dev/shm to be explicit about your desire.
Going this route means you're going to leverage all the well tested Linux VFS code and your tests will execute with higher fidelity.
It always surprised me somewhat that there isn't a set of traits covering some kind of `fs` like surface. It's not a trivial surface, but it's not huge either, and I've also found myself in a position of wanting to have multiple implementations of a filesystem-like structure (not even for the same reasons).
Tricky to make that kind of change to std lib now I appreciate, but it seems like an odd gap.
See David R. Hanson's "A Portable File Directory System" [0][1], for example: a 700 lines long implementation of early UNIX's filesystem API that piggy-backs on some sort of pre-existing (block-oriented) I/O primitives, which means you can do it entirely in-memory, with about another 300 lines of code or so.
I suspect that with OSes becoming much more UNIX-like the demand for such abstraction layers shrank almost to nothing.
Having traits in the stdlib would be nice, but there's also the type parameter pollution that results from mocking which I think is also a turn-off, as TFA says about rsfs.
I have a Rust library to implement the UAPI config spec (a spec that describes which files and directories a service should look for config files in), and initally wanted to test it with filesystem mocks. After making some effort to implement the mock types and traits, plus wrappers around the `<F = StdFs>` types to hide the `<F>` parameter because I didn't want to expose it in the public API, I realized it was much easier to not bother and just create all the directory trees I needed for the tests.
Yeah having traits for this in the stdlib would be nice.
You might find Lunchbox [1] interesting. I needed an async virtual filesystem interface for a project a few years ago (and didn't find an existing library that fit my needs) so I built one:
> Lunchbox provides a common interface that can be used to interact with any filesystem (e.g. a local FS, in-memory FS, zip filesystem, etc). This interface closely matches `tokio::fs::` ...
It includes a few traits (`ReadableFileSystem`, `WritableFileSystem`) along with an implementation for local filesystems. I also used those traits to build libraries that enable things like read-only filesystems backed by zip files [2] and remote filesystems over a transport (e.g. TCP, UDS, etc) [3].
This is something I'm hopeful will fall out accidentally from Zig's new IO interface. If everyone doing IO has the implementation injected, mocks (fault injection, etc) become trivial, along with any other sort of feature you might want like checking S3 if you don't have a sufficiently recent local copy.
It's not always about mocking (in my cases it hasn't been). Sometimes it is about multiple "real" implementations - a filesystem is itself an abstraction, and a very common one, it seems like it would at least sometimes be useful to be able to leverage that more flexibly.
I no mention of fsync/sync_all. That’s why your disk file system is acting as fast as your in memory file system (for small tests). Both are effectively in-memory.
I guess I wasn't sufficiently clear in the post, but the part I think is interesting is not that tmpfs and SSD bench at the same speed. I am aware of in-memory filesystem caches, and explicitly mention them twice in the last few paragraphs.
The interesting part, to me, was that using the vfs crate or the rsfs crate didn't produce any differences from using tmpfs or an SSD. In theory, those crates completely cut out the actual filesystem and the OS entirely. Somehow, avoiding all those syscalls didn't make it any faster? Not what I expected.
Anyway, if you have examples of in-process filesystem mocks that run faster than the in-memory filesystem cache, I'd love to hear about them.
A Rust-specific danger is that, if you don't explicitly sync a file before dropping it, any errors from syncing are ignored. So if you care about atomicity, call eg `File::sync_all()`.
This is actually true for all programs (on Linux at least) because closing a file does not mean it will be synced and so close(2) may not return an error even if the later sync will error out.
The more general issue (not checking close(2) errors) is mostly true for most programming languages. I can count on one hand how many C programs I've seen that attempt to check the return value from close(2) consistently, let alone programs in languages like Go where handling it is far more effort than ignoring it.
Also, close(2) doesn't consistently return errors. Because filesystem errors are often a property of the whole filesystem and data written during sync has been disassociated from the filesystem, the error usually can't be linked to a particular file descriptor. Most filesystems instead just return EIO if the filesystem had an error at all. This is arguably less useful than not returning an error at all because the error might be triggered by a completely unrelated process and (as above) you might not receive errors that you do care about.
Filesystems also have different approaches to which close(2) calls will get filesystem errors. Some only return the error to the first close(2) call, which means another thread or process could clear the error bit. Other filesystems keep the error bit set until a remount, which means that any program checking close(2) will get lots of spurious errors. From memory there was a data corruption bug in PostgreSQL a few years ago because they were relying on close(2) error semantics that didn't work for all filesystems.
On most filesystems close(2) is nearly a noop, so even if you surfaced errors from close it returning successfully would not guarantee an absence of errors.
close without fsync (or direct IO) essentially is telling the OS that you don't need immediate durability and prefer performance instead.
I'd almost never want do to fsync in normal code (unless implementing something transactional)... but I'd want an explicit close almost always (or drop should panic/abort).
For context - cppreference.com doesn't say anything about `fstream` syncing on drop, but it does have an explicit `sync` function. `QFile` from Qt doesn't even have a sync function, which I find odd.
> It turns out the intended primary use case of the crate is to store files inside Rust binaries but still have an API sort of like the filesystem API to interact with them. Unfortunately, that information is hidden away in a comment on a random GitHub issue, rather than included in the project readme.
A+ on technical prowess,
F- on being able to articulate a couple words about it on a text file.
That was 8 years ago, and even then mkfile needed a 512K buffer size to saturate the hardware. With the 512 byte default buffer it was 8x slower than the hardware.
In addition, as others have pointed out, if you are not doing something extra to ensure things are flushed to disk, you are just measuring the buffer cache in the first place.
Linux supports two different in-memory filesystems, ramfs and tmpfs. It should be easy enough to set up tests to use paths on such a mount point. Wonder why OP didn't mention these.
Readit News
Going this route means you're going to leverage all the well tested Linux VFS code and your tests will execute with higher fidelity.
Tricky to make that kind of change to std lib now I appreciate, but it seems like an odd gap.
I suspect that with OSes becoming much more UNIX-like the demand for such abstraction layers shrank almost to nothing.
[0] https://drh.github.io/documents/pds-spe.pdf
[1] https://drh.github.io/documents/pds.pdf
I have a Rust library to implement the UAPI config spec (a spec that describes which files and directories a service should look for config files in), and initally wanted to test it with filesystem mocks. After making some effort to implement the mock types and traits, plus wrappers around the `<F = StdFs>` types to hide the `<F>` parameter because I didn't want to expose it in the public API, I realized it was much easier to not bother and just create all the directory trees I needed for the tests.
You might find Lunchbox [1] interesting. I needed an async virtual filesystem interface for a project a few years ago (and didn't find an existing library that fit my needs) so I built one:
> Lunchbox provides a common interface that can be used to interact with any filesystem (e.g. a local FS, in-memory FS, zip filesystem, etc). This interface closely matches `tokio::fs::` ...
It includes a few traits (`ReadableFileSystem`, `WritableFileSystem`) along with an implementation for local filesystems. I also used those traits to build libraries that enable things like read-only filesystems backed by zip files [2] and remote filesystems over a transport (e.g. TCP, UDS, etc) [3].
[1] https://crates.io/crates/lunchbox
[2] https://crates.io/crates/zipfs
[3] https://github.com/VivekPanyam/carton/tree/main/source/anywh...
But the stdlib one is a bit barebones. So people created: https://github.com/spf13/afero
I think was trying to test something in Rust and I was surprised by how many people were OK with using real file's for unit testing.
It seems like a massive oversight for being able to use rust in a corporate environment.
Complicated logic can be in pure functions and not be intertwined with IO if it needs to be tested.
Mocking IO seems like it won’t really capture the problems you might encounter in reality anyway.
The interesting part, to me, was that using the vfs crate or the rsfs crate didn't produce any differences from using tmpfs or an SSD. In theory, those crates completely cut out the actual filesystem and the OS entirely. Somehow, avoiding all those syscalls didn't make it any faster? Not what I expected.
Anyway, if you have examples of in-process filesystem mocks that run faster than the in-memory filesystem cache, I'd love to hear about them.
The more general issue (not checking close(2) errors) is mostly true for most programming languages. I can count on one hand how many C programs I've seen that attempt to check the return value from close(2) consistently, let alone programs in languages like Go where handling it is far more effort than ignoring it.
Also, close(2) doesn't consistently return errors. Because filesystem errors are often a property of the whole filesystem and data written during sync has been disassociated from the filesystem, the error usually can't be linked to a particular file descriptor. Most filesystems instead just return EIO if the filesystem had an error at all. This is arguably less useful than not returning an error at all because the error might be triggered by a completely unrelated process and (as above) you might not receive errors that you do care about.
Filesystems also have different approaches to which close(2) calls will get filesystem errors. Some only return the error to the first close(2) call, which means another thread or process could clear the error bit. Other filesystems keep the error bit set until a remount, which means that any program checking close(2) will get lots of spurious errors. From memory there was a data corruption bug in PostgreSQL a few years ago because they were relying on close(2) error semantics that didn't work for all filesystems.
close without fsync (or direct IO) essentially is telling the OS that you don't need immediate durability and prefer performance instead.
A+ on technical prowess,
F- on being able to articulate a couple words about it on a text file.
https://blog.metaobject.com/2017/02/mkfile8-is-severely-sysc...
That was 8 years ago, and even then mkfile needed a 512K buffer size to saturate the hardware. With the 512 byte default buffer it was 8x slower than the hardware.
In addition, as others have pointed out, if you are not doing something extra to ensure things are flushed to disk, you are just measuring the buffer cache in the first place.