Difference between revisions of "Install NixOS on a Server With a Different Filesystem"

From NixOS Wiki
Jump to: navigation, search
(add notice about official nixos manual section at the top)
(Debian9 is outdated in the meantime)
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{note|There is now an ''official'' section in the [https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro  nixos manual] which should be followed before trying to reproduce the steps in this article.}}
+
{{note|If you don't need to change the filesystem, you can follow the new section in the [https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro  nixos manual].}}
  
 
Usually when installing NixOS, you boot from an external USB device containing the installer, which makes it easy to change the underlying filesystem. On a remote server however, this is usually not possible. This guide shows you how you can still make it work. Here it is shown with a DigitalOcean (DO) droplet initially running Debian, then replacing the original filesystem with ZFS and installing NixOS on it.
 
Usually when installing NixOS, you boot from an external USB device containing the installer, which makes it easy to change the underlying filesystem. On a remote server however, this is usually not possible. This guide shows you how you can still make it work. Here it is shown with a DigitalOcean (DO) droplet initially running Debian, then replacing the original filesystem with ZFS and installing NixOS on it.
Line 10: Line 10:
  
 
* A running Linux installation (This guide uses Debian)
 
* A running Linux installation (This guide uses Debian)
* At least 1GB of RAM
+
* At least 2GB of RAM
 
* root ssh access
 
* root ssh access
 
* The same architecture as your local machine
 
* The same architecture as your local machine
  
Note: If you need a $5 DO droplet which only has 500MB of RAM, get a higher tier one temporarily and resize it to your desired size later. To be able to resize it, the installation droplet needs to have less than or equal amounts of disk space than your target size. E.g. get the $15 droplet which has 20GB of disk space like the $5 one (but the $10 one currently doesn't work as it has 30GB disk space).
+
Note: DigitalOcean allows you to resize the droplet temporarily, which you can use to get enough RAM to do this, while reverting it once done.
  
 
== Building the system ==
 
== Building the system ==
Line 64: Line 64:
  
 
<source lang="bash">cd /
 
<source lang="bash">cd /
apt-get update && apt-get install xz-utils # Needed to unpack on Debian 9
+
tar -xf ~/nixos-system-x86_64-linux.tar.xz
tar -xf ~/nixos-system-x86_64-linux.tar.gz
 
 
./kexec_nixos
 
./kexec_nixos
 
</source>
 
</source>
Line 156: Line 155:
 
== NIXOS_LUSTRATE ==
 
== NIXOS_LUSTRATE ==
 
See the relevant [https://github.com/NixOS/nixpkgs/pull/17784 PR] or [https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro  section of the manual]
 
See the relevant [https://github.com/NixOS/nixpkgs/pull/17784 PR] or [https://nixos.org/nixos/manual/index.html#sec-installing-from-other-distro  section of the manual]
 +
 +
[[Category:Server]]

Latest revision as of 17:26, 18 December 2022

Note: If you don't need to change the filesystem, you can follow the new section in the nixos manual.

Usually when installing NixOS, you boot from an external USB device containing the installer, which makes it easy to change the underlying filesystem. On a remote server however, this is usually not possible. This guide shows you how you can still make it work. Here it is shown with a DigitalOcean (DO) droplet initially running Debian, then replacing the original filesystem with ZFS and installing NixOS on it.

The trick to making this work is by building a kexec compatible ramdisk NixOS system locally, transfering it to the server and use the kexec command to boot into it. Afterwards, you can install NixOS like you usually do.

Requirements

To follow this guide you need a server with:

  • A running Linux installation (This guide uses Debian)
  • At least 2GB of RAM
  • root ssh access
  • The same architecture as your local machine

Note: DigitalOcean allows you to resize the droplet temporarily, which you can use to get enough RAM to do this, while reverting it once done.

Building the system

To create the installation system, we use clever's kexec config with some modifications. Clone the repository and create a file myconfig.nix with the following contents:

{
  imports = [
    ./configuration.nix
  ];

  # Make it use predictable interface names starting with eth0
  boot.kernelParams = [ "net.ifnames=0" ];

  networking = {
    defaultGateway = "x.x.x.x";
    # Use google's public DNS server
    nameservers = [ "8.8.8.8" ];
    interfaces.eth0 = {
      ipAddress = "y.y.y.y";
      prefixLength = z;
    };
  };

  kexec.autoReboot = false;

  users.users.root.openssh.authorizedKeys.keys = [
    "ssh-rsa ..."
  ];
}

Replace x.x.x.x with your servers IPv4 gateway, y.y.y.y with its IPv4 address and z with the subnet mask prefix. In DigitalOcean, you can find this info in your droplet's Networking tab in the Public Network section. Finally, put your ssh public key in the users.users.root.openssh.authorizedKeys.keys option.

Build the system configuration with

nix-build '<nixpkgs/nixos>' -A config.system.build.kexec_tarball -I nixos-config=./myconfig.nix -Q

This may take a while. When it finishes you will find the finished system as a tarball in ./result/tarball.

Starting the built NixOS system on the server

Transfer the tarball to the server and ssh into it

scp result/tarball/nixos-system-x86_64-linux.tar.xz root@y.y.y.y:.
ssh root@y.y.y.y

Then unpack the tarball and run the kexec script

cd /
tar -xf ~/nixos-system-x86_64-linux.tar.xz
./kexec_nixos
  1. Wait until the + kexec -e line shows up, then terminate the ssh connection by pressing the following keys one after the other: RETURN + ~ + ..
  2. Wait until you have a ping again by doing ping y.y.y.y.
  3. Reconnect with ssh, you should see a warning about the host identification having changed, which is a good sign in our case.
  4. Remove your server's previous entry in ~/.ssh/known_hosts and try again.

If everything worked, you should now see the [root@kexec:~]# prompt. You're now running NixOS entirely in RAM!

Installing NixOS

Install NixOS like normal, and make sure to include the following:

  • boot.kernelParams = [ "net.ifnames=0" ];
  • The same network configuration from above

Example installation with ZFS

Repartition your main disk using fdisk to such a configuration (you can remove all previous partitions):

/dev/vda1 1M BIOS boot partition (BIOS boot)
/dev/vda2 200M boot partition (EFI System)
/dev/vda3 2GB swap partition (Linux swap)
/dev/vda4 rest, zfs partition (Linux Filesystem)

Create the file systems:

mkfs.ext4 /dev/vda2
mkswap /dev/vda3
zpool create -O compress=on -O mountpoint=legacy tank /dev/vda4
zfs create -o xattr=off -o atime=off tank/nix

Mount them:

swapon /dev/vda3
mount -t zfs tank /mnt
mkdir /mnt/boot /mnt/nix
mount -t zfs tank/nix /mnt/nix
mount /dev/vda2 /mnt/boot

Generate the configs:

nixos-generate-config --root /mnt

Edit /mnt/etc/nixos/configuration.nix to something like this:

{ config, pkgs, ... }:

{
  imports =
    [
      ./hardware-configuration.nix
    ];

  boot.loader.grub.enable = true;
  boot.loader.grub.version = 2;

  boot.kernelParams = [ "net.ifnames=0" ];

  boot.zfs.devNodes = "/dev";
  boot.loader.grub.device = "/dev/vda";

  networking = {
    hostName = "foobar";
    hostId = "12345678";
    defaultGateway = "x.x.x.x";
    nameservers = [ "8.8.8.8" ];
    interfaces.eth0 = {
      ipAddress = "y.y.y.y";
      prefixLength = z;
    };
  };

  services.openssh.enable = true;

  users.users.root.openssh.authorizedKeys.keys = [
    "ssh-rsa ..."
  ];

  system.stateVersion = "18.03"; # Did you read the comment?
}

And finally, install nixos and cross fingers:

nixos-install
reboot

NIXOS_LUSTRATE

See the relevant PR or section of the manual