NixOS Testing library
The primary documentation for the NixOS VM testing framework is in the NixOS manual, and in the Nixpkgs manual. A tutorial can be found at [1].
The test infrastructure entry point is nixos/lib/testing.nix. Alternatively, for out-of-tree tests you can invoke it via Nixpkgs as the nixosTest function, which reuses your already evaluated Nixpkgs to generate your node configurations. The test infra relies on the qemu build-vm code to generate virtual machines.
It will generate a test driver (a wrapper of nixos/lib/test-driver/test-driver.py) in charge of creating the network. It will start one vde-switch and its associated socket per vlan (defined in virtualisation.vlans). IPs are assigned declaratively according to the number of vlan via the function `assignIPAddresses`.
The driver (of the form /nix/store/668bqxvsv6rn9hy8n4nmaps9ma2i5k4r-nixos-test-driver-<TESTNAME>) will launch the different vms passed as arguments. The wrapper `bin/nixos-run-vms` is in charge to start the driver with the correct VM script as arguments.
Once the driver is loaded, depending on the environment variables `tests` it will run in an interactive mode or run some perl code (`testScript`). In interactive mode, you can run `start_all` followed by `join_all` to start and keep the VM alive
How to debug tests ?
You can run the tests interactively as described in [2]. When you run `nix-build ./nixos/tests/login.nix`, the resulting output gives you a summary of the results, but to gain access to the VM, you can run
nix repl ./nixos/tests/login.nix
and see the ran VM via `driver.outPath`.
I don't see any prompt ? (qemu window pitch black)
Check the output for `malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "\x{0}\x{0}\x{0}\x{0}...") at /nix/store/1hkp2n6hz3ybf2rvkjkwrzgbjkrrakzl-update-users-groups.pl line 11`. You should purge the state present in rm -rf /tmp/vm-state-<VM_NAME>
Setting `virtualisation.vlans` does not create the expected interfaces
There are two sides to the problem: 1. By default the qemu-vm setups a `user` based nic: virtualisation.qemu.networkingOptions. You need to override the option to get rid of this interface. 2. As of this writing nixpkgs will generate interfaces starting from `eth1` (instead of `eth0`).
Keys: https://en.wikibooks.org/wiki/QEMU/Monitor#sendkey_keys
home-manager example
It is possible to use home-manager to manage packages per user. This example shows how to add home-manager to a single file configuration.
The complete `hmtest.nix` file content looks like the following:
let nixpkgs = builtins.fetchTarball "https://github.com/nixOS/nixpkgs/archive/22.05.tar.gz"; pkgs = import nixpkgs {}; home-manager = builtins.fetchTarball "https://github.com/nix-community/home-manager/archive/release-22.05.tar.gz"; in pkgs.nixosTest { nodes.machine = { config, pkgs, ... }: { imports = [ (import "${home-manager}/nixos") ]; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; services.xserver.enable = true; services.xserver.displayManager.gdm.enable = true; services.xserver.desktopManager.gnome.enable = true; users.users.alice = { isNormalUser = true; extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user. }; home-manager.users.alice = { home.packages = [ pkgs.firefox pkgs.thunderbird ]; }; system.stateVersion = "22.05"; }; testScript = {nodes, ...}: machine.wait_for_unit("default.target") machine.succeed("su -- alice -c 'which firefox'") machine.fail("su -- root -c 'which firefox'") ; }
wayland application example
The configuration we are using is starting the gnome desktop manager using wayland. To test if a wayland application is working is more complicated because we need to automate the login into gnome and automated startup of the application. Additionally we need to enable access to gnome dbus interface. To do this we need to modify the configuration the automated start of the application including automated login to gnome/wayland
In the machine configuration we need to enable autologin for the user alice.
services.xserver.displayManager.autoLogin.enable = true;
services.xserver.displayManager.autoLogin.user = "alice";
To simplify our script we pin the uid of the user to 1000.
uid = 1000;
We specify a service that auto start firefox after login, which is easier than doing this in the test script.
environment.systemPackages = [
(pkgs.makeAutostartItem {
name = "firefox";
package = pkgs.firefox;
})
];
Because gnome doesn't allow the evaluation of javascript to get information about open windows we need to override the gnome-shell startup service to start gnome-shell in unsafe mode:
systemd.user.services = {
"org.gnome.Shell@wayland" = {
serviceConfig = {
ExecStart = [
# Clear the list before overriding it.
""
# Eval API is now internal so Shell needs to run in unsafe mode.
"${pkgs.gnome.gnome-shell}/bin/gnome-shell --unsafe-mode"
];
};
};
The test script utilizes the gnome dbus interface to get a list of open wayland windows. we wait until firefox appear to be started and make a screenshot that will be found in the result folder.
testScript = {nodes, ...}: let
user = nodes.machine.config.users.users.alice;
bus = "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/${toString user.uid}/bus";
gdbus = "${bus} gdbus";
su = command: "su - ${user.name} -c '${command}'";
gseval = "call --session -d org.gnome.Shell -o /org/gnome/Shell -m org.gnome.Shell.Eval";
wmClass = su "${gdbus} ${gseval} global.display.focus_window.wm_class";
in
machine.wait_until_succeeds("${wmClass} | grep -q 'firefox'")
machine.sleep(20)
machine.screenshot("screen")
;
The complete `firefoxtest.nix` file looks like the following:
let nixpkgs = builtins.fetchTarball "https://github.com/nixOS/nixpkgs/archive/22.05.tar.gz"; pkgs = import nixpkgs {}; home-manager = builtins.fetchTarball "https://github.com/nix-community/home-manager/archive/release-22.05.tar.gz"; in pkgs.nixosTest { nodes.machine = {...}: { imports = [ (import "${home-manager}/nixos") ]; boot.loader.systemd-boot.enable = true; boot.loader.efi.canTouchEfiVariables = true; services.xserver.enable = true; services.xserver.displayManager.gdm.enable = true; services.xserver.desktopManager.gnome.enable = true; services.xserver.displayManager.autoLogin.enable = true; services.xserver.displayManager.autoLogin.user = "alice"; users.users.alice = { isNormalUser = true; extraGroups = ["wheel"]; # Enable ‘sudo’ for the user. uid = 1000; }; home-manager.users.alice = { home.packages = [ pkgs.firefox pkgs.thunderbird ]; }; system.stateVersion = "22.05"; environment.systemPackages = [ (pkgs.makeAutostartItem { name = "firefox"; package = pkgs.firefox; }) ]; systemd.user.services = { "org.gnome.Shell@wayland" = { serviceConfig = { ExecStart = [ # Clear the list before overriding it. "" # Eval API is now internal so Shell needs to run in unsafe mode. "${pkgs.gnome.gnome-shell}/bin/gnome-shell --unsafe-mode" ]; }; }; }; }; testScript = {nodes, ...}: let user = nodes.machine.config.users.users.alice; #uid = toString user.uid; bus = "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/${toString user.uid}/bus"; gdbus = "${bus} gdbus"; su = command: "su - ${user.name} -c '${command}'"; gseval = "call --session -d org.gnome.Shell -o /org/gnome/Shell -m org.gnome.Shell.Eval"; wmClass = su "${gdbus} ${gseval} global.display.focus_window.wm_class"; in machine.wait_until_succeeds("${wmClass} | grep -q 'firefox'") machine.sleep(20) machine.screenshot("screen") ; }