Difference between revisions of "Flakes"

From NixOS Wiki
Jump to: navigation, search
m (Fix typo 'attr' -> 'name')
m (rollback unauthorized mass edits)
Tag: Rollback
 
(54 intermediate revisions by 25 users not shown)
Line 1: Line 1:
'''Nix flakes''' are an [https://nixos.org/manual/nix/stable/contributing/experimental-features.html experimental feature] of the Nix package manager.
+
'''Nix flakes''' is an [https://nixos.org/manual/nix/stable/contributing/experimental-features.html experimental feature] of the Nix package manager. Flakes was introduced with Nix 2.4 ([https://nixos.org/manual/nix/unstable/release-notes/rl-2.4.html see release notes]).
  
== Introduction ==
+
====Introduction====
  
The flakes experimental feature introduces a policy for managing dependencies between Nix expressions and adds supporting convenience functionality.
+
Nix flakes provide a standard way to write Nix expressions (and therefore packages) whose dependencies are version-pinned in a lock file, improving reproducibility of Nix installations. The experimental <code>nix</code> CLI lets you evaluate or build an expression contained within a flake, install a derivation from a flake into an [[User Environment]], and operate on flake outputs much like the original nix-{build,eval,...} commands would.
  
Technically, a [https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#description flake] is a file system tree that contains a Nix file named <code>flake.nix</code> in its root directory.
+
* A [https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#description flake] refers to a file-system tree whose root directory contains the Nix file specification called <code>flake.nix</code>.
 +
* An installation may contain any number of flakes, independent of each other or even call each other.
 +
* The contents of <code>flake.nix</code> file follow the uniform naming schema for expressing packages and dependencies on Nix.
 +
*  Flakes use the standard Nix protocols, including the [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references URL-like syntax] for specifying repositories and package names.
 +
* To simplify the long URL syntax with shorter names, [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-registry.html flakes uses a registry] of symbolic identifiers.
 +
* Flakes also allow for locking references and versions that can then be easily queried and updated programmatically.
 +
*  [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix.html Nix command-line interface] accepts flake references for expressions that build, run, and deploy packages.
  
Flakes add the following behavior to Nix:
+
====Enable flakes temporarily====
  
# A <code>flake.nix</code> file offers a uniform [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-format schema], where:
+
When using any <code>nix</code> command, add the following command-line options:
#* Other flakes can be referenced as dependencies providing Nix language code or other files.
+
<syntaxHighlight lang=text>
#* The values produced by the Nix expression in <code>flake.nix</code> are structured according to pre-defined use cases.
+
--experimental-features 'nix-command flakes'
# References to other flakes can be specified using a dedicated [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references URL-like syntax]. A [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-registry.html flake registry] allows using symbolic identifiers for further brevity. References can be automatically locked to their current specific version and later updated programmatically.
+
</syntaxHighlight>
# A [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix.html new command line interface], implemented as a separate experimental feature, leverages flakes by accepting flake references in order to build, run, or deploy software defined as a flake.
 
  
== Enable flakes ==
+
====Enable flakes permanently in NixOS====
  
Flakes have been introduced with Nix 2.4 ([https://nixos.org/manual/nix/unstable/release-notes/rl-2.4.html release notes]).
+
Add the following to the [https://nixos.wiki/wiki/Overview_of_the_NixOS_Linux_distribution#Declarative_Configuration system configuration] ([https://nixos.wiki/wiki/Flakes#Using_nix_flakes_with_NixOS flakes]):
  
See [https://lazamar.co.uk/nix-versions/?channel=nixpkgs-unstable&package=nix Nix package versions] to find a channel or <code>nixpkgs</code> revision providing the required Nix version.
+
<syntaxHighlight lang=nix>
 +
  nix.settings.experimental-features = [ "nix-command" "flakes" ];
 +
</syntaxHighlight>
  
Note: Flakes is automatically enabled if you installed Nix using the [https://github.com/DeterminateSystems/nix-installer Determinate Installer].
+
=====Other Distros, with Home-Manager=====
  
===Temporary===
+
Add the following to your home-manager config:
  
Add <code>--experimental-features 'nix-command flakes'</code> when calling the <code>nix</code> command.
+
<syntaxHighlight lang=nix>
 +
  nix = {
 +
    package = pkgs.nix;
 +
    settings.experimental-features = [ "nix-command" "flakes" ];
 +
  };
 +
</syntaxHighlight>
  
=== Permanent ===
+
=====Other Distros, without Home-Manager=====
  
Edit either <code>~/.config/nix/nix.conf</code> or <code>/etc/nix/nix.conf</code> and add:
+
{{Note | The  [https://github.com/DeterminateSystems/nix-installer Nix Determinate Installer] enables flakes by default.}}
 +
 
 +
Add the following to <code>~/.config/nix/nix.conf</code> or <code>/etc/nix/nix.conf</code>:
  
 
<syntaxHighlight lang=text>
 
<syntaxHighlight lang=text>
experimental-features = nix-command flakes  
+
experimental-features = nix-command flakes
 
</syntaxHighlight>
 
</syntaxHighlight>
  
Here's a handy copy-paste:
+
===Basic Usage of Flake===
 
 
<syntaxHighlight lang=text>
 
mkdir -p ~/.config/nix
 
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf
 
</syntaxHighlight>
 
  
If the Nix installation is in multi-user mode, don’t forget to restart the <code>nix-daemon</code>.
+
Before running any nix commands at this point, please note the two warnings below: one for encryption and the other for git.
  
=== NixOS ===
+
====Encryption WARNING====
  
On NixOS set the following options in <code>configuration.nix</code> and run <code>nixos-rebuild</code>.
+
{{Warning | Since contents of flake files are copied to the world-readable Nix store folder, do not put any unencrypted secrets in flake files. You should instead use a [[Comparison of secret managing schemes|secret managing scheme]].}}
  
<syntaxHighlight lang=nix>
+
====Git WARNING====
{ pkgs, ... }: {
+
For flakes in git repos, only files in the working tree will be copied to the store.
  nix.settings.experimental-features = [ "nix-command" "flakes" ];
 
}
 
</syntaxHighlight>
 
  
== Basic project usage ==
+
Therefore, if you use <code>git</code> for your flake, ensure to <code>git add</code> any project files after you first create them.
  
{{warning | All files tracked by the version control system (e.g. git or mercurial) will be copied to the nix store when the flake is evaluated. So be careful when putting secrets in version control (which is not optimal by itself) around a flake.}}
+
See also https://www.tweag.io/blog/2020-05-25-flakes/
  
In your repo, run <code>nix flake init</code> to generate the flake.nix file. Then run <code>git add flake.nix</code> to add it to the git staging area, otherwise nix will not recognize that the file exists.
+
====Generate flake.nix file====
  
See also https://www.tweag.io/blog/2020-05-25-flakes/
+
To start the basic usage of flake, run the flake command in the project directory:
  
To install a flake when using nix outside of nixOS, use <code>nix profile install /path/to/flake</code>. The path may also be an URL (e.g.: <code>nix profile install git+https://example.com/my-repo?dir=subdirectory</code>).
+
<syntaxHighlight lang=text>
 +
nix flake init
 +
</syntaxHighlight>
  
 
== Flake schema ==
 
== Flake schema ==
Line 72: Line 80:
 
* <code>description</code> is a string describing the flake.
 
* <code>description</code> is a string describing the flake.
 
* <code>inputs</code> is an attribute set of all the dependencies of the flake. The schema is described below.
 
* <code>inputs</code> is an attribute set of all the dependencies of the flake. The schema is described below.
* <code>outputs</code> is a function of one argument that takes an attribute set of all the realized inputs, and outputs another attribute set which schema is described below.
+
* <code>outputs</code> is a function of one argument that takes an attribute set of all the realized inputs, and outputs another attribute set whose schema is described below.
 
* <code>nixConfig</code> is an attribute set of values which reflect the [https://nixos.org/manual/nix/stable/command-ref/conf-file.html values given to nix.conf]. This can extend the normal behavior of a user's nix experience by adding flake-specific configuration, such as a binary cache.
 
* <code>nixConfig</code> is an attribute set of values which reflect the [https://nixos.org/manual/nix/stable/command-ref/conf-file.html values given to nix.conf]. This can extend the normal behavior of a user's nix experience by adding flake-specific configuration, such as a binary cache.
  
 
=== Input schema ===
 
=== Input schema ===
  
This is not a complete schema but should be enough to get you started:
+
[https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-inputs The nix flake inputs manual].
 +
[https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references The nix flake references manual].
  
<syntaxHighlight lang=nix>
+
=== Output schema ===
{
 
  inputs = {
 
    # GitHub example, also supports GitLab:
 
    nixpkgs.url = "github:Mic92/nixpkgs/master";
 
    # Git urls
 
    git-example.url = "git+https://git.somehost.tld/user/path?ref=branch&rev=fdc8ef970de2b4634e1b3dca296e1ed918459a9e";
 
    # Local directories (for absolute paths you can omit 'path:')
 
    directory-example.url = "path:/path/to/repo";
 
    # The above url will also copy .git, use this for (shallow) local Git repos
 
    git-directory-example.url = "git+file:/path/to/repo?shallow=1"
 
    # Use this for non-flakes
 
    bar.url = "github:foo/bar/branch";
 
    bar.flake = false;
 
    # Overwrite inputs in a flake
 
    # This is useful to use the same nixpkgs version in both flakes
 
    sops-nix.url = "github:Mic92/sops-nix";
 
    sops-nix.inputs.nixpkgs.follows = "nixpkgs";
 
    # Pin flakes to a specific revision
 
    nix-doom-emacs.url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3";
 
    nix-doom-emacs.flake = false;
 
    # To use a subdirectory of a repo, pass dir=
 
    nixpkgs.url = "github:foo/bar?dir=shu";
 
  }
 
}
 
</syntaxHighlight>
 
Also see [https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references the nix flake manual].
 
  
The bar input is then passed to the output schema
+
Once the inputs are resolved, they're passed to the function `outputs` along with with `self`, which is the directory of this flake in the store. `outputs` returns the outputs of the flake, according to the following schema.
 
 
=== Output schema ===
 
  
 
This is described in the nix package manager [https://github.com/NixOS/nix/blob/master/src/nix/flake.cc src/nix/flake.cc] in CmdFlakeCheck.
 
This is described in the nix package manager [https://github.com/NixOS/nix/blob/master/src/nix/flake.cc src/nix/flake.cc] in CmdFlakeCheck.
  
 
Where:
 
Where:
+
 
 
* <code><system></code> is something like "x86_64-linux", "aarch64-linux", "i686-linux", "x86_64-darwin"
 
* <code><system></code> is something like "x86_64-linux", "aarch64-linux", "i686-linux", "x86_64-darwin"
 
* <code><name></code> is an attribute name like "hello".
 
* <code><name></code> is an attribute name like "hello".
Line 144: Line 125:
 
   overlays."<name>" = final: prev: { };
 
   overlays."<name>" = final: prev: { };
 
   # Default overlay
 
   # Default overlay
   overlays.default = {};
+
   overlays.default = final: prev: { };
 
   # Nixos module, consumed by other flakes
 
   # Nixos module, consumed by other flakes
   nixosModules."<name>" = { config }: { options = {}; config = {}; };
+
   nixosModules."<name>" = { config, ... }: { options = {}; config = {}; };
 
   # Default module
 
   # Default module
   nixosModules.default = {};
+
   nixosModules.default = { config, ... }: { options = {}; config = {}; };
   # Used with `nixos-rebuild --flake .#<hostname>`
+
   # Used with `nixos-rebuild switch --flake .#<hostname>`
 
   # nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
 
   # nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
 
   nixosConfigurations."<hostname>" = {};
 
   nixosConfigurations."<hostname>" = {};
Line 244: Line 225:
 
== Install packages with `nix profile` ==
 
== Install packages with `nix profile` ==
  
You can install Nix packages into your local profile from any flakes with `nix profile install. For example from `nixpkgs`` flake :
+
[https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-profile-install.html <code>nix profile install</code> in the manual]
<syntaxHighlight lang=bash>
 
nix profile install nixpkgs#hello
 
</syntaxHighlight>
 
 
 
On NixOS you can syncronize your system and your profile references to `nixpkgs` with:
 
<syntaxHighlight lang=bash>
 
nix registry pin nixpkgs github:NixOS/nixpkgs/$(nixos-version --revision)
 
</syntaxHighlight>
 
  
 
== Using nix flakes with NixOS ==
 
== Using nix flakes with NixOS ==
Line 274: Line 247:
 
<syntaxHighlight lang=nix>
 
<syntaxHighlight lang=nix>
 
{
 
{
   inputs.nixpkgs.url = github:NixOS/nixpkgs;
+
   inputs.nixpkgs.url = github:NixOS/nixpkgs/nixos-unstable;
 
   inputs.home-manager.url = github:nix-community/home-manager;
 
   inputs.home-manager.url = github:nix-community/home-manager;
 
+
 
 
   outputs = { self, nixpkgs, ... }@attrs: {
 
   outputs = { self, nixpkgs, ... }@attrs: {
 
     nixosConfigurations.fnord = nixpkgs.lib.nixosSystem {
 
     nixosConfigurations.fnord = nixpkgs.lib.nixosSystem {
Line 290: Line 263:
 
{ config, lib, nixpkgs, home-manager, ... }: {
 
{ config, lib, nixpkgs, home-manager, ... }: {
 
   # do something with home-manager here, for instance:
 
   # do something with home-manager here, for instance:
   imports = [ home-manager.nixosModule ];
+
   imports = [ home-manager.nixosModules.default ];
 
   ...
 
   ...
 
}
 
}
Line 308: Line 281:
  
 
To switch a remote configuration, use:
 
To switch a remote configuration, use:
<syntaxHighlight lang=console>
+
<syntaxHighlight lang=bash>
 
$ nixos-rebuild --flake .#mymachine \
 
$ nixos-rebuild --flake .#mymachine \
   --target-host mymachine-hostname --build-host localhost \
+
   --target-host mymachine-hostname --build-host mymachine-hostname --fast \
 
   switch
 
   switch
 
</syntaxHighlight>
 
</syntaxHighlight>
+
 
{{warning|Remote building seems to be broken at the moment, which is why the build host is set to “localhost”.}}
+
{{warning|Remote building seems to have an issue that's [https://github.com/NixOS/nixpkgs/issues/134952#issuecomment-1367056358 resolved by setting the <code>--fast</code> flag].}}
 +
 
 +
== Pinning the registry to the system pkgs on NixOS ==
 +
 
 +
<syntaxHighlight lang=nix>
 +
  nix.registry = {
 +
    nixpkgs.to = {
 +
      type = "path";
 +
      path = pkgs.path;
 +
    };
 +
  };
 +
</syntaxHighlight>
 +
 
 +
{{warning|The above example seems to break nix's eval cache, and may slow down hdd systems significantly!}}
 +
 
 +
To make sure the registry entry is "locked", use the following:
 +
 
 +
<syntaxHighlight lang=nix>
 +
  nix.registry = {
 +
    nixpkgs.to = {
 +
      type = "path";
 +
      path = pkgs.path;
 +
      narHash = builtins.readFile
 +
          (pkgs.runCommandLocal "get-nixpkgs-hash"
 +
            { nativeBuildInputs = [ pkgs.nix ]; }
 +
            "nix-hash --type sha256 --sri ${pkgs.path} > $out");
 +
    };
 +
  };
 +
</syntaxHighlight>
 +
 
 +
This has the unfortunate side-effect of requiring import-from-derivation and slowing down build times, however it may greatly speed up almost every eval. Full-time flakes users may be able to just use <code>narHash = pkgs.narHash</code>.
  
 
== Super fast nix-shell ==
 
== Super fast nix-shell ==
  
One of the nix feature of the Flake edition is that Nix evaluations are cached.
+
A feature of the nix Flake edition is that Nix evaluations are cached.
  
 
Let’s say that your project has a <code>shell.nix</code> file that looks like this:
 
Let’s say that your project has a <code>shell.nix</code> file that looks like this:
Line 365: Line 368:
 
{{warning|TODO: there is an alternative version where the defaultPackage is a pkgs.buildEnv that contains all the dependencies. And then nix shell is used to open the environment.}}
 
{{warning|TODO: there is an alternative version where the defaultPackage is a pkgs.buildEnv that contains all the dependencies. And then nix shell is used to open the environment.}}
  
=== Direnv integration ===
+
=== Automatically switch nix shells with nix-direnv ===
 
 
Assuming that the flake defines a <code>devShell</code> output attribute and that you are using direnv. Here is how to replace the old use nix stdlib function with the faster flake version:
 
 
 
<syntaxHighlight lang=sh>
 
use_flake() {
 
  watch_file flake.nix
 
  watch_file flake.lock
 
  eval "$(nix print-dev-env --profile "$(direnv_layout_dir)/flake-profile")"
 
}
 
</syntaxHighlight>
 
 
 
Copy this in <code>~/.config/direnv/lib/use_flake.sh</code> or in <code>~/.config/direnv/direnvrc</code>
 
or directly in your project specific <code>.envrc</code>.
 
 
 
Note: You may not need to create <code>use_flake()</code> yourself; as of [https://github.com/direnv/direnv/releases/tag/v2.29.0#:~:text=add%20use_flake%20function direnv 2.29,] <code>use flake</code> is part of direnv's standard library.
 
  
With this in place, you can now replace the use nix invocation in the <code>.envrc</code> file with <code>use flake</code>:
+
You can easily switch nix shells when you cd into different projects with nix-direnv. [https://github.com/nix-community/nix-direnv View their guide here]
 
 
<syntaxHighlight lang=text>
 
# .envrc
 
use flake
 
</syntaxHighlight>
 
 
 
The nice thing about this approach is that evaluation is cached.
 
 
 
==== Optimize the reloads ====
 
 
 
Nix Flakes has a Nix evaluation caching mechanism. Is it possible to expose that somehow to automatically trigger direnv reloads?
 
 
 
With the previous solution, direnv would only reload if the flake.nix or flake.lock files have changed. This is not completely precise as the flake.nix file might import other files in the repository.
 
 
 
==== Setting the bash prompt like nix-shell ====
 
 
 
A [https://github.com/NixOS/nix/pull/4189 new experimental feature of flakes] allow to setup a bash-prompt per flake:
 
<syntaxHighlight lang=nix>
 
{
 
  description = "...";
 
  nixConfig.bash-prompt = "\[nix-develop\]$ ";
 
  ...
 
}
 
</syntaxHighlight>
 
 
 
Otherwise it's also possible to set the <code>nix develop</code> bash prompt system wide using the [https://nixos.org/manual/nix/unstable/command-ref/conf-file.html nix.conf option bash-prompt]. (On nixos I think it is set in <code>nix.extraOptions</code>)
 
  
 
== Pushing Flakes to Cachix ==
 
== Pushing Flakes to Cachix ==
Line 444: Line 406:
  
 
   inputs = {
 
   inputs = {
     nixpkgs.url = "nixpkgs/nixos-21.11";  
+
     nixpkgs.url = "nixpkgs/nixos-21.11";
     nixpkgs-unstable.url = "nixpkgs/nixos-unstable";  
+
     nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
 
   };
 
   };
  
Line 478: Line 440:
 
   environment.systemPackages = [pkgs.firefox pkgs.unstable.chromium];
 
   environment.systemPackages = [pkgs.firefox pkgs.unstable.chromium];
 
   # ...
 
   # ...
}  
+
}
 
</syntaxHighlight>
 
</syntaxHighlight>
 
Same can be done with the NURs, as it already has an ''overlay'' attribute in the flake.nix of the project, you can just add <syntaxHighlight lang=nix>nixpkgs.overlays = [ nur.overlay ];</syntaxHighlight>
 
Same can be done with the NURs, as it already has an ''overlay'' attribute in the flake.nix of the project, you can just add <syntaxHighlight lang=nix>nixpkgs.overlays = [ nur.overlay ];</syntaxHighlight>
Line 493: Line 455:
  
 
<syntaxHighlight lang=text>
 
<syntaxHighlight lang=text>
# nix repl  
+
# nix repl
 
>> :lf /etc/nixos
 
>> :lf /etc/nixos
 
>> nixosConfigurations.myhost.config
 
>> nixosConfigurations.myhost.config
Line 501: Line 463:
 
Or out of your current flake:
 
Or out of your current flake:
 
<syntaxHighlight lang=text>
 
<syntaxHighlight lang=text>
# nix repl  
+
# nix repl
 
>> :lf .#
 
>> :lf .#
 
</syntaxHighlight>
 
</syntaxHighlight>
Line 507: Line 469:
 
You can then access to the inputs, outputs… For instance if you would like to check the default version of the kernel present in nixpgs:
 
You can then access to the inputs, outputs… For instance if you would like to check the default version of the kernel present in nixpgs:
 
<syntaxHighlight lang=text>
 
<syntaxHighlight lang=text>
nix-repl> inputs.nixpkgs.legacyPackages.x86_64-linux.linuxPackages.kernel.version      
+
nix-repl> inputs.nixpkgs.legacyPackages.x86_64-linux.linuxPackages.kernel.version
 
"5.15.74"
 
"5.15.74"
 
</syntaxHighlight>
 
</syntaxHighlight>
Line 556: Line 518:
 
== Enable unfree software ==
 
== Enable unfree software ==
  
Because flake evalutations are hermetic, they will ignore the system configuration on nonfree software and the <code>NIXPKGS_ALLOW_UNFREE</code> environment variable by default.
+
Refer to [[Unfree Software]].
  
To use nonfree software with CLI tools like <code>nix shell</code> or <code>nix run</code>, the <code>--impure</code> flag must be used for Nixpkgs to access the current environment variables:
+
== Development tricks ==
<syntaxHighlight lang=console>
+
=== Build a package added in a PR ===
$ NIXPKGS_ALLOW_UNFREE=1 nix run --impure nixpkgs#discord
+
<syntaxHighlight>
 +
nix build github:nixos/nixpkgs?ref=pull/<PR_NUMBER>/head#<PACKAGE>
 
</syntaxHighlight>
 
</syntaxHighlight>
 +
this allows building a package that has not yet been added to nixpkgs.
  
To use nonfree software in a flake, add <code>nixpkgs</code> as an input in your flake and import it with the <code>allowUnfree</code> option:
+
note that this will download a full source tarball of nixpkgs.  if you already have a local clone, using that may be faster due to delta compression:
<syntaxHighlight lang=nix>
+
<syntaxHighlight>
pkgs = import nixpkgs { config = { allowUnfree = true; }; };
+
git fetch upstream pull/<PR_NUMBER>/head && git checkout FETCH_HEAD && nix build .#PACKAGE
</syntaxHighlight>
 
 
 
=== Enable unfree software in home-manager ===
 
 
 
If you want to install software using home-manager via nix flakes in non NixOS systems (like darwin) you can use the home-manager <code>nixpkgs.config</code> option for example
 
 
 
<syntaxHighlight lang=nix>
 
nixpkgs.config.allowUnfree = true;
 
 
</syntaxHighlight>
 
</syntaxHighlight>
 
+
this allows building a package that has not yet been added to nixpkgs.
== Development tricks ==
 
 
=== How to add a file locally in git but not include it in commits ===
 
=== How to add a file locally in git but not include it in commits ===
  
Line 583: Line 538:
 
<syntaxHighlight>
 
<syntaxHighlight>
 
git add --intent-to-add extra/flake.nix
 
git add --intent-to-add extra/flake.nix
git update-index --assume-unchanged extra/flake.nix
+
git update-index --skip-worktree --assume-unchanged extra/flake.nix
 
</syntaxHighlight>
 
</syntaxHighlight>
  
 
=== Rapid iteration of a direct dependency ===
 
=== Rapid iteration of a direct dependency ===
One common pain point with using Nix as a development environment is the need to completely rebuild dependencies and re-enter the dev shell every time they are updated. The <code>nix develop --redirect <flake> <directory></code> command allows you to provide a mutable dependency to your shell as if it were built by Nix.  
+
One common pain point with using Nix as a development environment is the need to completely rebuild dependencies and re-enter the dev shell every time they are updated. The <code>nix develop --redirect <flake> <directory></code> command allows you to provide a mutable dependency to your shell as if it were built by Nix.
  
 
Consider a situation where your executable, <code>consumexe</code>, depends on a library, <code>libdep</code>. You're trying to work on both at the same time, where changes to <code>libdep</code> are reflected in real time for <code>consumexe</code>. This workflow can be achieved like so:
 
Consider a situation where your executable, <code>consumexe</code>, depends on a library, <code>libdep</code>. You're trying to work on both at the same time, where changes to <code>libdep</code> are reflected in real time for <code>consumexe</code>. This workflow can be achieved like so:
Line 612: Line 567:
 
* [https://www.tweag.io/blog/2020-06-25-eval-cache/ Nix Flakes, Part 2: Evaluation caching] (Eelco Dolstra, 2020)
 
* [https://www.tweag.io/blog/2020-06-25-eval-cache/ Nix Flakes, Part 2: Evaluation caching] (Eelco Dolstra, 2020)
 
* [https://www.tweag.io/blog/2020-07-31-nixos-flakes/ Nix Flakes, Part 3: Managing NixOS systems] (Eelco Dolstra, 2020)
 
* [https://www.tweag.io/blog/2020-07-31-nixos-flakes/ Nix Flakes, Part 3: Managing NixOS systems] (Eelco Dolstra, 2020)
 +
* [https://github.com/ryan4yin/nixos-and-flakes-book NixOS & Flakes Book](Ryan4yin, 2023) - 🛠️ ❤️ An unofficial NixOS & Flakes book for beginners.
 +
* [https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html Nix flake command reference manual] - Many additional details about flakes, and their parts.
 
* [https://xeiaso.net/blog/nix-flakes-1-2022-02-21 Nix Flakes: an Introduction] (Xe Iaso, 2022)
 
* [https://xeiaso.net/blog/nix-flakes-1-2022-02-21 Nix Flakes: an Introduction] (Xe Iaso, 2022)
 
* [https://serokell.io/blog/practical-nix-flakes Practical Nix Flakes] (Alexander Bantyev, 2021) - Intro article on working with Nix and Flakes
 
* [https://serokell.io/blog/practical-nix-flakes Practical Nix Flakes] (Alexander Bantyev, 2021) - Intro article on working with Nix and Flakes

Latest revision as of 11:01, 6 April 2024

Nix flakes is an experimental feature of the Nix package manager. Flakes was introduced with Nix 2.4 (see release notes).

Introduction

Nix flakes provide a standard way to write Nix expressions (and therefore packages) whose dependencies are version-pinned in a lock file, improving reproducibility of Nix installations. The experimental nix CLI lets you evaluate or build an expression contained within a flake, install a derivation from a flake into an User Environment, and operate on flake outputs much like the original nix-{build,eval,...} commands would.

  • A flake refers to a file-system tree whose root directory contains the Nix file specification called flake.nix.
  • An installation may contain any number of flakes, independent of each other or even call each other.
  • The contents of flake.nix file follow the uniform naming schema for expressing packages and dependencies on Nix.
  • Flakes use the standard Nix protocols, including the URL-like syntax for specifying repositories and package names.
  • To simplify the long URL syntax with shorter names, flakes uses a registry of symbolic identifiers.
  • Flakes also allow for locking references and versions that can then be easily queried and updated programmatically.
  • Nix command-line interface accepts flake references for expressions that build, run, and deploy packages.

Enable flakes temporarily

When using any nix command, add the following command-line options:

 --experimental-features 'nix-command flakes'

Enable flakes permanently in NixOS

Add the following to the system configuration (flakes):

  nix.settings.experimental-features = [ "nix-command" "flakes" ];
Other Distros, with Home-Manager

Add the following to your home-manager config:

  nix = {
    package = pkgs.nix;
    settings.experimental-features = [ "nix-command" "flakes" ];
  };
Other Distros, without Home-Manager
Note: The Nix Determinate Installer enables flakes by default.

Add the following to ~/.config/nix/nix.conf or /etc/nix/nix.conf:

experimental-features = nix-command flakes

Basic Usage of Flake

Before running any nix commands at this point, please note the two warnings below: one for encryption and the other for git.

Encryption WARNING

Warning: Since contents of flake files are copied to the world-readable Nix store folder, do not put any unencrypted secrets in flake files. You should instead use a secret managing scheme.

Git WARNING

For flakes in git repos, only files in the working tree will be copied to the store.

Therefore, if you use git for your flake, ensure to git add any project files after you first create them.

See also https://www.tweag.io/blog/2020-05-25-flakes/

Generate flake.nix file

To start the basic usage of flake, run the flake command in the project directory:

nix flake init

Flake schema

The flake.nix file is a Nix file but that has special restrictions (more on that later).

It has 4 top-level attributes:

  • description is a string describing the flake.
  • inputs is an attribute set of all the dependencies of the flake. The schema is described below.
  • outputs is a function of one argument that takes an attribute set of all the realized inputs, and outputs another attribute set whose schema is described below.
  • nixConfig is an attribute set of values which reflect the values given to nix.conf. This can extend the normal behavior of a user's nix experience by adding flake-specific configuration, such as a binary cache.

Input schema

The nix flake inputs manual. The nix flake references manual.

Output schema

Once the inputs are resolved, they're passed to the function `outputs` along with with `self`, which is the directory of this flake in the store. `outputs` returns the outputs of the flake, according to the following schema.

This is described in the nix package manager src/nix/flake.cc in CmdFlakeCheck.

Where:

  • <system> is something like "x86_64-linux", "aarch64-linux", "i686-linux", "x86_64-darwin"
  • <name> is an attribute name like "hello".
  • <flake> is a flake name like "nixpkgs".
  • <store-path> is a /nix/store.. path
{ self, ... }@inputs:
{
  # Executed by `nix flake check`
  checks."<system>"."<name>" = derivation;
  # Executed by `nix build .#<name>`
  packages."<system>"."<name>" = derivation;
  # Executed by `nix build .`
  packages."<system>".default = derivation;
  # Executed by `nix run .#<name>`
  apps."<system>"."<name>" = {
    type = "app";
    program = "<store-path>";
  };
  # Executed by `nix run . -- <args?>`
  apps."<system>".default = { type = "app"; program = "..."; };

  # Formatter (alejandra, nixfmt or nixpkgs-fmt)
  formatter."<system>" = derivation;
  # Used for nixpkgs packages, also accessible via `nix build .#<name>`
  legacyPackages."<system>"."<name>" = derivation;
  # Overlay, consumed by other flakes
  overlays."<name>" = final: prev: { };
  # Default overlay
  overlays.default = final: prev: { };
  # Nixos module, consumed by other flakes
  nixosModules."<name>" = { config, ... }: { options = {}; config = {}; };
  # Default module
  nixosModules.default = { config, ... }: { options = {}; config = {}; };
  # Used with `nixos-rebuild switch --flake .#<hostname>`
  # nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
  nixosConfigurations."<hostname>" = {};
  # Used by `nix develop .#<name>`
  devShells."<system>"."<name>" = derivation;
  # Used by `nix develop`
  devShells."<system>".default = derivation;
  # Hydra build jobs
  hydraJobs."<attr>"."<system>" = derivation;
  # Used by `nix flake init -t <flake>#<name>`
  templates."<name>" = {
    path = "<store-path>";
    description = "template description goes here?";
  };
  # Used by `nix flake init -t <flake>`
  templates.default = { path = "<store-path>"; description = ""; };
}

You can also define additional arbitrary attributes, but these are the outputs that Nix knows about.

nix run

When output apps.<system>.myapp is not defined, nix run myapp runs <packages or legacyPackages.<system>.myapp>/bin/<myapp.meta.mainProgram or myapp.pname or myapp.name (the non-version part)>

Using flakes with stable Nix

There exists the flake-compat library that you can use to shim default.nix and shell.nix files. It will download the inputs of the flake, pass them to the flake’s outputs function and return an attribute set containing defaultNix and shellNix attributes. The attributes will contain the output attribute set with an extra default attribute pointing to current platform’s defaultPackage (resp. devShell for shellNix).

Place the following into default.nix (for shell.nix, replace defaultNix with shellNix) to use the shim:

(import (
  fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/12c64ca55c1014cdc1b16ed5a804aa8576601ff2.tar.gz";
    sha256 = "0jm6nzb83wa6ai17ly9fzpqc40wg1viib8klq8lby54agpl213w5"; }
) {
  src =  ./.;
}).defaultNix

You can also use the lockfile to make updating the hashes easier using nix flake lock --update-input flake-compat. Add the following to your flake.nix:

  inputs.flake-compat = {
    url = "github:edolstra/flake-compat";
    flake = false;
  };

and add flake-compat to the arguments of outputs attribute. Then you will be able to use default.nix like the following:

(import (
  let
    lock = builtins.fromJSON (builtins.readFile ./flake.lock);
  in fetchTarball {
    url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
    sha256 = lock.nodes.flake-compat.locked.narHash; }
) {
  src =  ./.;
}).defaultNix

Accessing flakes from Nix expressions

If you want to access a flake from within a regular Nix expression on a system that has flakes enabled, you can use something like (builtins.getFlake "path:/path/to/directory").packages.x86_64-linux.default, where 'directory' is the directory that contains your flake.nix.

Making your evaluations pure

Nix flakes run in pure evaluation mode, which is underdocumented. Some tips for now:

  • fetchurl and fetchtar require a sha256 argument to be considered pure.
  • builtins.currentSystem is non-hermetic and impure. This can usually be avoided by passing the system (i.e., x86_64-linux) explicitly to derivations requiring it.
  • Imports from channels like <nixpkgs> can be made pure by instead importing from the output function in flake.nix, where the arguments provide the store path to the flake's inputs:
 outputs = { self, nixpkgs, ... }:
  {
    nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
      # Note that you cannot put arbitrary configuration here: the configuration must be placed in the files loaded via modules
      system = "x86_64-linux";
      modules = [
        (nixpkgs + "/nixos/modules/<some-module>.nix")
        ./machine.nix
      ];
    };
  };

The nix flakes command

The nix flake subcommand is described in command reference page of the unstable manual.

Install packages with `nix profile`

nix profile install in the manual

Using nix flakes with NixOS

nixos-rebuild switch will read its configuration from /etc/nixos/flake.nix if it is present.

A basic nixos flake.nix could look like this:

{
  outputs = { self, nixpkgs }: {
    # replace 'joes-desktop' with your hostname here.
    nixosConfigurations.joes-desktop = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [ ./configuration.nix ];
    };
  };
}

If you want to pass on the flake inputs to external configuration files, you can use the specialArgs attribute:

{
  inputs.nixpkgs.url = github:NixOS/nixpkgs/nixos-unstable;
  inputs.home-manager.url = github:nix-community/home-manager;

  outputs = { self, nixpkgs, ... }@attrs: {
    nixosConfigurations.fnord = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      specialArgs = attrs;
      modules = [ ./configuration.nix ];
    };
  };
}

Then, you can access the flake inputs from the file configuration.nix like this:

{ config, lib, nixpkgs, home-manager, ... }: {
  # do something with home-manager here, for instance:
  imports = [ home-manager.nixosModules.default ];
  ...
}

nixos-rebuild also allows to specify different flake using the --flake flag (# is optional):

$ sudo nixos-rebuild switch --flake '.#'

By default nixos-rebuild will use the currents system hostname to lookup the right nixos configuration in nixosConfigurations. You can also override this by using appending it to the flake parameter:

$ sudo nixos-rebuild switch --flake '/etc/nixos#joes-desktop'

To switch a remote configuration, use:

$ nixos-rebuild --flake .#mymachine \
  --target-host mymachine-hostname --build-host mymachine-hostname --fast \
  switch
Warning: Remote building seems to have an issue that's resolved by setting the --fast flag.

Pinning the registry to the system pkgs on NixOS

  nix.registry = {
    nixpkgs.to = {
      type = "path";
      path = pkgs.path;
    };
  };
Warning: The above example seems to break nix's eval cache, and may slow down hdd systems significantly!

To make sure the registry entry is "locked", use the following:

  nix.registry = {
    nixpkgs.to = {
      type = "path";
      path = pkgs.path;
      narHash = builtins.readFile
          (pkgs.runCommandLocal "get-nixpkgs-hash"
            { nativeBuildInputs = [ pkgs.nix ]; }
            "nix-hash --type sha256 --sri ${pkgs.path} > $out");
    };
  };

This has the unfortunate side-effect of requiring import-from-derivation and slowing down build times, however it may greatly speed up almost every eval. Full-time flakes users may be able to just use narHash = pkgs.narHash.

Super fast nix-shell

A feature of the nix Flake edition is that Nix evaluations are cached.

Let’s say that your project has a shell.nix file that looks like this:

{ pkgs ? import <nixpkgs> { } }:
with pkgs;
mkShell {
  buildInputs = [
    nixpkgs-fmt
  ];

  shellHook = ''
    # ...
  '';
}

Running nix-shell can be a bit slow and take 1-3 seconds.

Now create a flake.nix file in the same repository:

{
  description = "my project description";

  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem
      (system:
        let pkgs = nixpkgs.legacyPackages.${system}; in
        {
          devShells.default = import ./shell.nix { inherit pkgs; };
        }
      );
}

Run git add flake.nix so that Nix recognizes it.

And finally, run nix develop. This is what replaces the old nix-shell invocation.

Exit and run again, this command should now be super fast.

Warning: TODO: there is an alternative version where the defaultPackage is a pkgs.buildEnv that contains all the dependencies. And then nix shell is used to open the environment.

Automatically switch nix shells with nix-direnv

You can easily switch nix shells when you cd into different projects with nix-direnv. View their guide here

Pushing Flakes to Cachix

https://docs.cachix.org/pushing#flakes

To push all flake outputs automatically, use devour-flake.

Build specific attributes in a flake repository

When in the repository top-level, run nix build .#<attr>. It will look in the legacyPackages and packages output attributes for the corresponding derivation.

Eg, in nixpkgs:

$ nix build .#hello

Building flakes from a Git repo url with submodules

As per nix 2.9.1, git submodules in package srcs won't get copied to the nix store, this may cause the build to fail. To workaround this, use:

$ nix build .?submodules=1#hello

See: https://github.com/NixOS/nix/pull/5434

Importing packages from multiple channels

A NixOS config flake skeleton could be as follows:

{
  description = "NixOS configuration with two or more channels";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-21.11";
    nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs, nixpkgs-unstable }:
    let
      system = "x86_64-linux";
      overlay-unstable = final: prev: {
        unstable = nixpkgs-unstable.legacyPackages.${prev.system};
        # use this variant if unfree packages are needed:
        # unstable = import nixpkgs-unstable {
        #   inherit system;
        #   config.allowUnfree = true;
        # };

      };
    in {
      nixosConfigurations."<hostname>" = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          # Overlays-module makes "pkgs.unstable" available in configuration.nix
          ({ config, pkgs, ... }: { nixpkgs.overlays = [ overlay-unstable ]; })
          ./configuration.nix
        ];
      };
    };
}
# NixOS configuration.nix, can now use "pkgs.package" or "pkgs.unstable.package"
{ config, pkgs, ... }: {
  environment.systemPackages = [pkgs.firefox pkgs.unstable.chromium];
  # ...
}

Same can be done with the NURs, as it already has an overlay attribute in the flake.nix of the project, you can just add

nixpkgs.overlays = [ nur.overlay ];

If the variable nixpkgs points to the flake, you can also define pkgs with overlays with:

pkgs = import nixpkgs { overlays = [ /*the overlay in question*/ ]; };

Getting Instant System Flakes Repl

How to get a nix repl out of your system flake:

# nix repl
>> :lf /etc/nixos
>> nixosConfigurations.myhost.config
{ ... }

Or out of your current flake:

# nix repl
>> :lf .#

You can then access to the inputs, outputs… For instance if you would like to check the default version of the kernel present in nixpgs:

nix-repl> inputs.nixpkgs.legacyPackages.x86_64-linux.linuxPackages.kernel.version
"5.15.74"

However, this won't be instant upon evaluation if any file changes have been done since your last configuration rebuild. Instead, if one puts:

nix.nixPath = let path = toString ./.; in [ "repl=${path}/repl.nix" "nixpkgs=${inputs.nixpkgs}" ];

In their system flake.nix configuration file, and includes the following file in their root directory flake as repl.nix:

let
  flake = builtins.getFlake (toString ./.);
  nixpkgs = import <nixpkgs> { };
in
{ inherit flake; }
// flake
// builtins
// nixpkgs
// nixpkgs.lib
// flake.nixosConfigurations

(Don't forget to git add repl.nix && nixos-rebuild switch --flake "/etc/nixos") Then one can run (or bind a shell alias):

source /etc/set-environment && nix repl $(echo $NIX_PATH | perl -pe 's|.*(/nix/store/.*-source/repl.nix).*|\1|')

This will launch a repl with access to nixpkgs, lib, and the flake options in a split of a second.

An alternative approach to the above shell alias is omitting repl from nix.nixPath and creating a shell script:

nix.nixPath = [ "nixpkgs=${inputs.nixpkgs}" ];
environment.systemPackages = let
  repl_path = toString ./.;
  my-nix-fast-repl = pkgs.writeShellScriptBin "my-nix-fast-repl" ''
    source /etc/set-environment
    nix repl "${repl_path}/repl.nix" "$@"
  '';
in [
  my-nix-fast-repl
];

Enable unfree software

Refer to Unfree Software.

Development tricks

Build a package added in a PR

nix build github:nixos/nixpkgs?ref=pull/<PR_NUMBER>/head#<PACKAGE>

this allows building a package that has not yet been added to nixpkgs.

note that this will download a full source tarball of nixpkgs. if you already have a local clone, using that may be faster due to delta compression:

git fetch upstream pull/<PR_NUMBER>/head && git checkout FETCH_HEAD && nix build .#PACKAGE

this allows building a package that has not yet been added to nixpkgs.

How to add a file locally in git but not include it in commits

When a git folder exists, flake will only copy files added in git to maximize reproducibility (this way if you forgot to add a local file in your repo, you will directly get an error when you try to compile it). However, for development purpose you may want to create an alternative flake file, for instance containing configuration for your preferred editors as described here… of course without committing this file since it contains only your own preferred tools. You can do so by doing something like that (say for a file called extra/flake.nix):

git add --intent-to-add extra/flake.nix
git update-index --skip-worktree --assume-unchanged extra/flake.nix

Rapid iteration of a direct dependency

One common pain point with using Nix as a development environment is the need to completely rebuild dependencies and re-enter the dev shell every time they are updated. The nix develop --redirect <flake> <directory> command allows you to provide a mutable dependency to your shell as if it were built by Nix.

Consider a situation where your executable, consumexe, depends on a library, libdep. You're trying to work on both at the same time, where changes to libdep are reflected in real time for consumexe. This workflow can be achieved like so:

cd ~/libdep-src-checkout/
nix develop # Or `nix-shell` if applicable.
export prefix="./install" # configure nix to install it here
buildPhase   # build it like nix does
installPhase # install it like nix does

Now that you've built the dependency, consumexe can take it as an input. In another terminal:

cd ~/consumexe-src-checkout/
nix develop --redirect libdep ~/libdep-src-checkout/install
echo $buildInputs | tr " " "\n" | grep libdep
# Output should show ~/libdep-src-checkout/ so you know it worked

If Nix warns you that your redirected flake isn't actually used as an input to the evaluated flake, try using the --inputs-from . flag. If all worked well you should be able to buildPhase && installPhase when the dependency changes and rebuild your consumer with the new version without exiting the development shell.

See also