NixOS on ARM

From NixOS Wiki
Revision as of 10:06, 2 April 2025 by KaladinB4 (talk | contribs) (Updated flake stable arm64 and mobile noxios suggetsions)
Jump to: navigation, search
    1. NixOS installation & configuration

The installation image contains a MBR partition table with two partitions: a FAT16/32 `/boot` partition and an ext4 root filesystem. This design allows you to directly reuse the SD image's partition layout and "install" NixOS on the same storage device by replacing the default configuration and running `nixos-rebuild`. On first boot, the image automatically resizes the rootfs partition to utilize all available storage space.

NixOS on ARM supports two approaches to system configuration: traditional configuration.nix files and the now-standard Flakes approach (recommended since NixOS 25.05). Both methods are covered below.

      1. Boot Process Overview

All ARM boards in NixOS use U-Boot as the bootloader, alongside U-Boot's Generic Distro Configuration Concept to communicate boot information (kernel zImage path, initrd, DTB, command line arguments). U-Boot scans storage devices for `/extlinux/extlinux.conf` or `/boot/extlinux/extlinux.conf` files (generated by NixOS) and uses the bootable partition flag.

U-Boot provides both an interactive shell and generation selection menu (similar to GRUB). Board-specific input/display support varies - check your device's wiki page for details.

      1. Basic Setup with configuration.nix

To generate a default `/etc/nixos/configuration.nix` file and detect hardware:

```bash sudo nixos-generate-config ```

Here's a modern configuration template suitable for most ARM boards:

```nix { config, pkgs, lib, ... }:

{ 

 # NixOS wants to enable GRUB by default
 boot.loader.grub.enable = false;
 
 # Enables the generation of /boot/extlinux/extlinux.conf
 boot.loader.generic-extlinux-compatible.enable = true;

 # Import hardware configuration generated by nixos-generate-config
 # This should correctly set up your file systems
 imports = [ ./hardware-configuration.nix ];
 
 # File systems configuration (if not correctly detected)
 # Usually nixos-generate-config handles this properly,
 # but this is provided as a fallback
 /*
 fileSystems = {
   "/" = {
     device = "/dev/disk/by-label/NIXOS_SD";
     fsType = "ext4";
   };
 };
 */
 
 # Adding a swap file is optional, but recommended for RAM-constrained devices
 # Size is in MiB
 # swapDevices = [ { device = "/swapfile"; size = 1024; } ];
 
 # Enable basic networking
 networking.networkmanager.enable = true;
 
 # Basic firewall
 networking.firewall.enable = true;
 
 # Set your time zone
 time.timeZone = "UTC";
 
 # System state version - IMPORTANT: Do not change this after initial install
 system.stateVersion = "25.05";

} ```

Apply your configuration with:

```bash sudo nixos-rebuild switch ```

      1. Kernel Selection

Kernel selection depends on your specific ARM device:

- **For boards with good mainline support** (including Raspberry Pi 4/5 on aarch64): 

 ```nix
 boot.kernelPackages = pkgs.linuxPackages_latest;
 ```

- **For legacy Raspberry Pi models on armv6/armv7**: 

 ```nix
 boot.kernelPackages = pkgs.linuxPackages_rpi;
 ```

- **For boards requiring specialized kernels**: 

 Refer to the board-specific wiki page for recommended kernel packages.
      1. Modern Flakes-Based Approach (Recommended)

Since NixOS 25.05, Flakes are the standard recommended approach for system configuration. They provide reproducible builds, locked dependencies, and simplify cross-device management.

1. Create a `flake.nix` file:

```nix { 

 description = "NixOS configuration for my ARM device";
 
 inputs = {
   nixpkgs.url = "github:nixos/nixpkgs/nixos-25.05";
   nixos-hardware.url = "github:NixOS/nixos-hardware";
 };
 
 outputs = { self, nixpkgs, nixos-hardware, ... }: {
   nixosConfigurations.mydevice = nixpkgs.lib.nixosSystem {
     system = "aarch64-linux"; # Or "armv6l-linux"/"armv7l-linux" for 32-bit ARM
     modules = [
       # Hardware-specific modules (if available)
       # nixos-hardware.nixosModules.raspberry-pi-4
       # nixos-hardware.nixosModules.raspberry-pi-5
       
       # Your main configuration file
       ./configuration.nix
     ];
   };
 };

} ```

2. Apply your configuration with:

```bash sudo nixos-rebuild switch --flake .#mydevice ```

The `#mydevice` part corresponds to the `nixosConfigurations.mydevice` entry in your flake.nix file.

      1. Binary Caches
        1. AArch64 (ARM64)

The official NixOS Hydra instance builds full binary sets for the AArch64 architecture, available on https://cache.nixos.org for both the nixpkgs-unstable and stable channels:

```nix nix.settings = { 

 substituters = [
   "https://cache.nixos.org"
 ];
 trusted-public-keys = [
   "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
 ];

}; ```

        1. armv6l and armv7l (32-bit ARM)

There are currently no official binary caches for 32-bit ARM. Consider: - Building packages natively (slow on low-power devices) - Using cross-compilation (see below) - Setting up your own binary cache

For better performance on bandwidth-constrained devices, limit parallel connections:

```nix nix.settings = { 

 max-substitution-jobs = 2;

}; ```

      1. Building on Resource-Constrained ARM Devices

ARM devices, especially older or low-power models, may struggle with building large packages. Consider these approaches:

        1. 1. Remote Builders

Configure a more powerful machine (ideally ARM64) as a remote builder:

```nix nix.buildMachines = [{ 

 hostName = "builder";
 system = "aarch64-linux";
 maxJobs = 4;
 speedFactor = 2;
 supportedFeatures = [ "nixos-test" "benchmark" "big-parallel" "kvm" ];

}]; nix.distributedBuilds = true; ```

For setup instructions, see the [Distributed Build](https://nixos.wiki/wiki/Distributed_build) wiki page.

        1. 2. Cross-Compilation using QEMU with binfmt_misc

On an x86_64 NixOS system, enable ARM emulation:

```nix boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; ```

Then build ARM packages from your x86_64 machine using:

```bash nix build --system aarch64-linux .#package ```

Or for non-Flake commands:

```bash nix-build '<nixpkgs>' -A pkgs.hello --argstr system aarch64-linux ```

        1. 3. Building through QEMU/KVM

While slower than other methods, full emulation through QEMU/KVM is possible for ARM development. See the [NixOS on ARM/QEMU](https://nixos.wiki/wiki/NixOS_on_ARM/QEMU) page for detailed setup.

      1. Creating Custom ARM Images
        1. Building with Flakes (Recommended)

Create a custom SD card image with a Flake:

```nix { 

 description = "Custom ARM image";
 
 inputs = {
   nixpkgs.url = "github:nixos/nixpkgs/nixos-25.05";
 };
 
 outputs = { self, nixpkgs }: {
   nixosConfigurations.armimage = nixpkgs.lib.nixosSystem {
     system = "aarch64-linux"; # For building 64-bit ARM images
     modules = [
       # Base SD image module for your architecture
       "${nixpkgs}/nixos/modules/installer/sd-card/sd-image-aarch64.nix"
       # Your customizations
       ({ ... }: {
         # Add your customizations here
         users.users.root.openssh.authorizedKeys.keys = [
           "ssh-ed25519 AAAAC3NzaC1lZDI1.... username@tld"
         ];
       })
     ];
   };
   
   # Make the SD image the default output
   packages.aarch64-linux.default = 
     self.nixosConfigurations.armimage.config.system.build.sdImage;
 };

} ```

Build with:

```bash nix build ```

        1. Building the Traditional Way

For non-Flake builds:

```bash nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage -I nixos-config=./sd-image.nix ```

Where `sd-image.nix` contains:

```nix { ... }: { 

 imports = [
   <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64.nix>
 ];
 
 # Your customizations here
 users.users.root.openssh.authorizedKeys.keys = [
    "ssh-ed25519 AAAAC3NzaC1lZDI1.... username@tld"
 ];

} ```

      1. U-Boot Customization

Some boards may require custom U-Boot builds. For boards supported by upstream U-Boot, you can cross-compile from an x86_64 host:

```bash nix-shell -p 'let plat = pkgsCross.aarch64-multiplatform; in plat.buildUBoot{defconfig = "board_defconfig"; extraMeta.platforms = ["aarch64-linux"];}' ```

Replace `board_defconfig` with your board's U-Boot configuration name.

For manual builds (when NixOS packages are unavailable):

```bash nix-shell -E 'with import <nixpkgs> {}; stdenv.mkDerivation { name = "arm-shell"; buildInputs = [git gnumake gcc gcc-arm-embedded dtc bison flex python3 swig]; }' git clone git://git.denx.de/u-boot.git cd u-boot make CROSS_COMPILE=arm-none-eabi- board_defconfig make CROSS_COMPILE=arm-none-eabi- ```

Flash the resulting U-Boot image to your storage device (SD card/eMMC/etc.) at the appropriate offset, typically:

```bash sudo dd if=u-boot-sunxi-with-spl.bin of=/dev/sdX bs=1024 seek=8 ```

The seek value and filename vary by board - check your board's wiki page or documentation.

      1. Optimizing Performance on ARM

For better performance on ARM devices:

```nix { 

 # CPU frequency scaling (if supported by your hardware)
 powerManagement.cpuFreqGovernor = "ondemand";
 
 # IO scheduler tuning (improves SD card performance)
 services.udev.extraRules = 
   # Set deadline scheduler for SD cards
   ACTION=="add|change", KERNEL=="mmcblk[0-9]", ATTR{queue/scheduler}="mq-deadline"
 ;
 
 # Zram swap (better than SD card swap)
 zramSwap = {
   enable = true;
   algorithm = "zstd";
 };
 
 # Using tmpfs for temp directories
 boot.tmpOnTmpfs = true;

} ```

      1. Security Hardening for ARM Devices

Since NixOS 25.05, additional security features are available that are relevant for ARM devices:

```nix { 

 # Kernel hardening options
 boot.kernelParams = [ "slab_nomerge" "init_on_alloc=1" "init_on_free=1" ];
 boot.kernel.sysctl = {
   "kernel.kptr_restrict" = 2;
   "kernel.dmesg_restrict" = 1;
 };
 
 # Mandatory Access Control (if supported by your hardware)
 security.apparmor.enable = true;  # Less resource-intensive than SELinux
 
 # Memory protection
 security.protectKernelImage = true;

} ```

Note: Always understand the security implications and compatibility impact of hardening options before applying them. Some settings may affect hardware compatibility or performance on specific ARM devices.

      1. Wayland Support on ARM64

Recent NixOS releases provide good Wayland support on ARM64 devices:

```nix { 

 # For GNOME
 services.xserver = {
   enable = true;
   displayManager.gdm.enable = true;
   desktopManager.gnome.enable = true;
   displayManager.gdm.wayland = true; # Explicitly enable Wayland
 };
 
 # For KDE Plasma
 services.xserver = {
   enable = true;
   displayManager.sddm.enable = true;
   desktopManager.plasma5.enable = true;
   displayManager.defaultSession = "plasmawayland";
 };
 
 # Required for GPU acceleration
 hardware.opengl = {
   enable = true;
   driSupport = true;
 };

} ```

For Raspberry Pi 5, add `dtoverlay=vc4-kms-v3d-pi5` in `/boot/config.txt` to enable GPU drivers compatible with Wayland.

      1. Common ARM Device Troubleshooting

1. **UART Console Access**: If your device isn't booting properly, enable UART: 

  ```nix
  boot.kernelParams = [
    "console=ttyS0,115200n8"  # Adjust ttyS0 to match your device's UART
  ];
  ```

  The actual device (ttyAMA0, ttyS0, ttyS1) depends on your hardware. See the [Enable UART](https://nixos.wiki/wiki/NixOS_on_ARM#Enable_UART) section for details.

2. **Memory Constraints**: For devices with limited RAM: 

  ```nix
  nix.settings.cores = 1; # Limit parallel builds
  nix.settings.max-jobs = 1;
  boot.tmp.cleanOnBoot = true; # Clean /tmp on boot
  ```

3. **Network Issues**: For better wireless support: 

  ```nix
  networking.networkmanager.enable = true;
  hardware.enableRedistributableFirmware = true; # For WiFi firmware
  ```
      1. Related Projects
        1. NixOS Mobile

For mobile devices like smartphones and tablets, check out the [Mobile NixOS project](https://github.com/mobile-nixos/mobile-nixos). This project extends NixOS to run on devices like the PinePhone and select OnePlus models, with features specifically designed for mobile use cases.

Mobile NixOS provides specialized modules for: - Touch input and mobile-friendly UI - Power management - Cellular modem support - Mobile hardware integration

Visit the [Mobile NixOS documentation](https://mobile.nixos.org/) for detailed setup and configuration guidance.

        1. QEMU Emulation

For detailed instructions on running NixOS ARM in QEMU, see the dedicated [NixOS on ARM/QEMU](https://nixos.wiki/wiki/NixOS_on_ARM/QEMU) wiki page.

Retrieved from "https://nixos.wiki/index.php?title=NixOS_on_ARM&oldid=13891"