ZFS Pool for Virtual Machines

Running VirtualBox on ZFS pool intended for general use is not exactly the smoothest experience. Due to it’s disk access pattern, what works for all your data will not work for virtual machine disk access. Yes, you can play with record size and adding SLOG device but you can also go slightly different route. Add disk specifically for VirtualBox.

My testing has found that simple SSD with the following settings does wonders:

zpool create -o autoexpand=on -m /VirtualBox \
    -O compression=off -O recordsize=4K -O atime=off \
    -O utf8only=on -O normalization=formD -O casesensitivity=sensitive \
    VirtualBox /dev/diskid/^^DISK.eli^^

First of all, you don’t want compression. Not because data is not compressible but because compression can lead you to believe you have more space than you actually do. Even when you use fixed disk, you can run out of disk space just because some uncompressible data got written within VM. Due to copy-on-write architecture, you can still get into the trouble but exposure is greatly limited.

Ideally record size should match your expected load. In case of VirtualBox that’s 512 bytes. However, tracking 512 byte records takes so much metadata that 4K records are actually both more space efficient and perform better. Depending on your exact hardware you might find that going to 8K or even higher might hit the sweet spot. Testing is the only way to know for sure but 4K is a reasonable starting point.

All other options are just plumbing - of course you want UTF-8 and no access time tracking.

Now you can run VirtualBox without complicating your main data pool.

PS: This assumes that you have disks enumerated by diskid and fully encrypted.

PPS: I usually just spin up temporary virtual machines for testing and thus I don’t care much about them long term. If you plan to kick something up long-term, do consider mirrored ZFS.