This is a request for comments for a new file system for zoned block devices.

The pull request based on top of another open pull request : #6878 that enables db_bench and db_stress to be used with custom file systems.

With this pull request I am mainly asking for comments on the high-level architecture and feedback on the following:

What applicable testing is available? I have mainly run smoke testing using db_bench and db_stress up till now and looking for ways to do i.e. recovery/crash/power fail testing.

Would a completely self-contained file system be preferable? Currently ZenFS stores logs and lock files on the default file system. The reason for this if to avoid duplicating already working code and for easy access to the log files.

What kind of workloads are most interesting to optimize for?

Looking forward to feedback as I finish up my laundry list of todos and optimizations.

Thanks!

ZenFS is a simple file system that utilizes RockDBs FileSystem interface to place files into zones on a raw zoned block device. By separating files into zones and utilizing the write life time hints to co-locate data of similar life times, the system write amplification is greatly reduced(compared to conventional block devices) while keeping the ZenFS capacity overhead at a very reasonable level.

ZenFS is designed to work with host-managed zoned spinning disks as well as NVME SSDs with Zoned Namespaces.

Some of the ideas and concepts in ZenFS are based on earlier work done by Abutalib Aghayev and Marc Acosta.

ZenFS depends on libzbd and Linux kernel 5.4 or later to perform zone management operations.

ZenFS implements the FileSystem API, and stores all data files on to a raw zoned block device. Log and lock files are stored on the default file system under a configurable directory. Zone management is done through libzbd and zenfs io is done through normal pread/pwrite calls.

Optimizing the IO path is on the TODO list.

This example issues 100 million random inserts followed by as many overwrites on a 100G memory backed zoned null block device. Target file sizes are set up to align with zone size.

This graph below shows the capacity usage over time. As ZenFS does not do any garbage collection the write amplification is 1.

Files are mapped into into a set of extents:

ZenFS is exceptionally lazy at current state of implementation and does not do any garbage collection whatsoever. As files gets deleted, the used capacity zone counters drops and when it reaches zero, a zone can be reset and reused.

Metadata is stored in a rolling log in the first zones of the block device.

Each valid meta data zone contains:

The metadata format is currently experimental. More extensive testing is needed and support for differential updates is planned to be implemented before bumping up the version to 1.0.