Btrfs
btrfs is a modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration.
Installation
See "Using configuration.nix from the installer" below for a BIOS installation, or if you want an installation more similar to what the installer would create.
Partition the disk
# printf "label: gpt\n,550M,U\n,,L\n" | sfdisk /dev/sdX
Format partitions and create subvolumes
# nix-shell -p btrfs-progs
# mkfs.fat -F 32 /dev/sdX1
# mkfs.btrfs /dev/sdX2
# mkdir -p /mnt
# mount /dev/sdX2 /mnt
# btrfs subvolume create /mnt/root
# btrfs subvolume create /mnt/home
# btrfs subvolume create /mnt/nix
# umount /mnt
Mount the partitions and subvolumes
# mount -o compress=zstd,subvol=root /dev/sdX2 /mnt
# mkdir /mnt/{home,nix}
# mount -o compress=zstd,subvol=home /dev/sdX2 /mnt/home
# mount -o compress=zstd,noatime,subvol=nix /dev/sdX2 /mnt/nix
# mkdir /mnt/boot
# mount /dev/sdX1 /mnt/boot
Install NixOS
# nixos-generate-config --root /mnt
# nano /mnt/etc/nixos/configuration.nix # manually add mount options
# nixos-install
Configuration
Compression
nixos-generate-config --show-hardware-config doesn't detect mount options automatically, so to enable compression, you must specify it and other mount options in a persistent configuration:
fileSystems = {
"/".options = [ "compress=zstd" ];
"/home".options = [ "compress=zstd" ];
"/nix".options = [ "compress=zstd" "noatime" ];
"/swap".options = [ "noatime" ];
};
Swap file
Optionally, create a separate subvolume for the swap file. Be sure to regenerate your hardware-configuration.nix if you choose to do this.
# mkdir -p /mnt
# mount /dev/sdXY /mnt
# btrfs subvolume create /mnt/swap
# umount /mnt
# mkdir /swap
# mount -o subvol=swap /dev/sdXY /swap
Then, create the swap file and adjust its size as desired:
# btrfs filesystem mkswapfile --size 8g --uuid clear /swap/swapfile
Finally, add the swap file to your configuration and nixos-rebuild switch:
swapDevices = [ { device = "/swap/swapfile"; } ];
Scrubbing
Btrfs filesystem by default keeps checksums for all files, and this allows to check if contents of the file has not changed due to hardware malfunctions and other external effects.
Scrubbing - is the process of checking file consistency (for this it may use checksums and/or duplicated copies of data, from raid for example). Scrubbing may be done "online", meaning you don't need to unmount a subvolume to scrub it.
You can enable automatic scrubbing with
services.btrfs.autoScrub.enable = true;
Automatic scrubbing by default is performed once a month, but you can change that with
services.btrfs.autoScrub.interval = "weekly";
interval syntax is defined by systemd.timer's Calendar Events
By default, autoscrub will scrub all detected btrfs mount points. However, in case of mounted nested subvolumes (like in example above /nix and /home are nested subvolumes under /), you only need to scrub the top-most one. So an example configuration may look like this:
services.btrfs.autoScrub = {
enable = true;
interval = "monthly";
fileSystems = [ "/" ];
};
The result of periodic auto scrub will be save to system journal, however you can also always check the status of the last scrub with
btrfs scrub status /
You can also start a scrubbing in background manually
btrfs scrub start /
You can check the status of the ongoing scrubbing process with the same status command from above
Usage
Subvolume
Create a subvolume
btrfs subvolume create /mnt/nixos
Removing a subvolume
btrfs subvolume delete /mnt/nixos
Snapshots
A snapshot in btrfs is simply a subvolume that shares its data (and metadata) with some other subvolume, using btrfs's COW capabilities.
Because of that, there is no special location for snapshots - you need to decide where you want to store them for yourself. It can be a simple directory inside root subvolume, or a directory inside a dedicated "snapshots" subvolume.
For this example we are going to store snapshots in a simple directory /snapshots, that has to be created beforehand with sudo mkdir /snapshots
Taking a read-only (-r) snapshot called home_snapshot_202302 of the subvolume mounted at /home
btrfs subvolume snapshot -r /home /snapshots/home_snapshot_202302
You can also snapshot the root subvolume. But keep in mind, that nested subvolumes are not part of a snapshot. So if you have subvolumes /nix /home, taking snapshot of / will not include them.
btrfs subvolume snapshot -r / /snapshots/nixos_snapshot_202302
Make snapshot read-write again
btrfs property set -ts /snapshots/home_snapshot_202302 ro false
However, changing read-only property of a snapshot in-place may causes issues with any future incremental send/receive.
Instead, a read-only snapshot itself (being a simple subvolume) can be snapshoted again as a read-write snapshot like this:
btrfs subvolume snapshot /snapshots/home_snapshot_202302 /snapshots/home_snapshot_202302_rw
Or it can be restored directly to /home straight away like this:
btrfs subvolume delete /home
btrfs subvolume snapshot /snapshots/home_snapshot_202302 /home
After this you can mount /home again./
Transfer snapshot
Sending the snapshot /snapshots/nixos_snapshot_202302 compressed to a remote host via ssh at root@192.168.178.110 and saving it to a subvolume mounted or directory at /mnt/nixos
sudo btrfs send /snapshots/nixos_snapshot_202302 | zstd | ssh root@192.168.178.110 'zstd -d | btrfs receive /mnt/nixos'
Installation with encryption
Using Luks2:
cryptsetup --verify-passphrase -v luksFormat "$DISK"p2
cryptsetup open "$DISK"p2 enc
You can use any device paritition for your bootloader # Notice that this bootloader is unencrypted on default:
mkfs.vfat -n BOOT "$DISK"p1
Creating Subvolumes
mkfs.btrfs /dev/mapper/enc # Creating btrfs partition
mount -t btrfs /dev/mapper/enc /mnt
# Create the subvolumes
btrfs subvolume create /mnt/root # The subvolume for /, which will be cleared on every boot
btrfs subvolume create /mnt/home # The subvolume for /home, which should be backed up
btrfs subvolume create /mnt/nix # The subvolume for /nix, which needs to be persistent but is not worth backing up, as it’s trivial to reconstruct
btrfs subvolume create /mnt/persist # The subvolume for /persist, containing system state which should be persistent across reboots and possibly backed up
btrfs subvolume create /mnt/log # The subvolume for /var/log.
# Take an empty *readonly* snapshot of the root subvolume, which can be rollback to on every boot.
btrfs subvolume snapshot -r /mnt/root /mnt/root-blank
Add the following nix config to clear the root volume on every boot(Erase your darlings):
{
boot.initrd.postDeviceCommands = lib.mkAfter ''
mkdir /mnt
mount -t btrfs /dev/mapper/enc /mnt
btrfs subvolume delete /mnt/root
btrfs subvolume snapshot /mnt/root-blank /mnt/root
'';
}
Unmount to mount on the subvolumes for the next steps:
umount /mnt
Once the subvolumes has been created, mount them with the options. Example with Zstandard compression with noatime:
mount -o subvol=root,compress=zstd,noatime /dev/mapper/enc /mnt
mkdir /mnt/home
mount -o subvol=home,compress=zstd,noatime /dev/mapper/enc /mnt/home
mkdir /mnt/nix
mount -o subvol=nix,compress=zstd,noatime /dev/mapper/enc /mnt/nix
mkdir /mnt/persist
mount -o subvol=persist,compress=zstd,noatime /dev/mapper/enc /mnt/persist
mkdir -p /mnt/var/log
mount -o subvol=log,compress=zstd,noatime /dev/mapper/enc /mnt/var/log
# do not forget to create and mount the bootloader
mkdir /mnt/boot
mount "$DISK"p1 /mnt/boot
Configure hardware-configuration.nix
# enable btrfs support
boot.supportedFilesystems = [ "btrfs" ];
fileSystems."/var/log" =
{ device = "/dev/disk/by-uuid/X";
fsType = "btrfs";
# enable noatime and zstd to the other subvolumes aswell
options = [ "subvol=log" "compress=zstd" "noatime" ];
# to have a correct log order
neededForBoot = true;
};
Generate Nixconfig:
nixos-generate-config --root /mnt
Using configuration.nix from the installer
The basic idea here, is that nixos-generate-config --root /mnt generates a configuration.nix file, but the one generated by the installer is better, so let's use that one. In particular, it sets up any boot.loader.grub options correctly for your hardware.
Run the NixOS installer and install gnome. Wait for it to finish, but do not reboot.
# These are the file systems the installer created
$ sudo sfdisk -l
<bla bla bla>
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 52420094 52418047 25G 83 Linux
# It didn't umount the disk, so lets copy the configuration.nix from it, and umount it afterwards:
$ mount | grep vda
/dev/vda1 on /tmp/calamares-root-efk4r5hz type ext4 (rw,relatime)
$ cp /tmp/calamares-root-efk4r5hz/etc/nixos/configuration.nix ~
$ sudo umount /tmp/calamares-root-efk4r5hz
# I'm an idiot with parted, but find gparted so much easier to use, so:
$ sudo gparted /dev/vda
# I created a small bootable boot partition (/dev/vda1)
# and a btrfs partition for the rest (/dev/vda2)
$ sudo sfdisk -l /dev/vda
<bla bla bla>
Device Boot Start End Sectors Size Id Type
/dev/vda1 * 2048 1128447 1126400 550M 83 Linux
/dev/vda2 1128448 52428799 51300352 24.5G 83 Linux
Now continue with from the #Format partitions and create subvolumes section above, but just before nixos-install:
$ sudo mv /mnt/etc/nixos/configuration.nix /mnt/etc/nixos/nixos-generate-config
.configuration.nix
$ sudo mv ~/configuration.nix /mnt/etc/nixos/configuration.nix
And then run sudo nixos-install. Enter a root password when prompted.
$ mount | grep vda
umount all mountpoints shown prior to reboot.
Reboot. When the gdm screen is shown, there will be no users. This is because any user created in configuration.nix doesn't have a password yet. Type Ctrl+Alt+F2 to get to a tty console, login as root and set a password for that user, in my case the "test" user:
# passwd test
New password:
Retype new password:
passwd: password updated successfully
# reboot
Now you can login and enjoy.