Difference between revisions of "Overview of the Nix Language"

From NixOS Wiki
Jump to: navigation, search
(no need to expand on that)
 
(53 intermediate revisions by 27 users not shown)
Line 1: Line 1:
 +
<blockquote>The Nix language is designed for conveniently creating and composing <i>derivations</i> – precise descriptions of how contents of existing files are used to derive new files. </blockquote>
 +
<cite>[https://nixos.org/manual/nix/stable/language/ Nix Reference Manual]</cite>
  
This [[:Category:Discussion|discussion]] article covers the syntax, semantics, typing, compilation, tooling and libraries of the Nix Expression Language.
+
{{Note|Check the [https://nixos.org/manual/nix/stable/language/ Nix Reference Manual on the Nix Language] for up-to-date documentation and [https://nix.dev/tutorials/nix-language Nix language basics] for a gentle introduction.}}
  
<blockquote>The Nix expression language is a pure, lazy, functional language. Purity means that operations in the language don't have side-effects (for instance, there is no variable assignment). Laziness means that arguments to functions are evaluated only when they are needed. Functional means that functions are “normal” values that can be passed around and manipulated in interesting ways. The language is not a full-featured, general purpose language. Its main job is to describe packages, compositions of packages, and the variability within packages.</blockquote>
+
== See also ==
<cite>From the [https://nixos.org/nix/manual/#ch-expression-language Nix manual]</cite>
+
* [[Nix Expression Language: Learning resources|Learning resources]]
 +
* [[Editor Modes for Nix Files]]
 +
* [[Nix Language: Tips & Tricks]]
 +
* [[Nix Language Quirks]]
  
The language was designed especially for the [[Nix Package Manager]].
+
[[Category:Pedias]]
 
 
== Learning resources ==
 
 
 
The [https://nixos.org/nix/manual/#ch-expression-language manual] provides a '''reference''' of the Nix language. All language constructs you may use in nix are defined here, together with code snippets.
 
 
 
[https://medium.com/@MrJamesFisher/nix-by-example-a0063a1a4c55 Nix By Example] is a step-by-step tutorial.
 
The [https://nixos.org/nixos/nix-pills/index.html nix pills] also provide a lot of insight into the language and functional package management in general.
 
[https://nixcloud.io/tour/ The Nix Tour] is an interactive tour that uses the actual package manager to learn you the language by example, in the browser.
 
 
 
== Language Paradigms ==
 
 
 
=== Lazy ===
 
 
 
Not all expressions in nixpkgs will be evaluated and instantiated as nix performs evaluation only when needed for a finished output. In the following example <code>abort</code> will never be triggered as the variable it belongs to is unused:
 
<syntaxHighlight lang=nix>
 
let
 
  a = abort "will never happen";
 
  b = "hello";
 
  c = "world"
 
in b + c
 
</syntaxHighlight>
 
 
 
=== Functional ===
 
Functional Programming is a style of building the structure and elements of computer programs—that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data. It is a declarative programming paradigm, which means programming is done with expressions or declarations instead of statements.
 
 
 
see also: [https://en.wikipedia.org/wiki/Functional_programming]
 
 
 
=== Pure ===
 
 
 
A pure function is a function where the return value is only determined by its input values, without observable side effects. In Nix, all build operations try to be as pure as possible to achieve reproducible builds. This means that wherever you build the packages as few side effects as possible should have an impact onto the build.
 
 
 
== Language Features ==
 
This section describes the main language features of the nix expression language
 
 
 
=== Expressions ===
 
 
 
When Nix tutorials talk about '''Nix Expressions''' they typically mean the definition of a function with multiple inputs which as a result in a derivation. However a Nix expression can be everything, from a simple string, to function to a set of expressions.
 
 
 
=== Types ===
 
 
 
The nix language provides a number of basic types:
 
{| class="wikitable"
 
|-
 
! Type !! Description !! Example
 
|-
 
| Strings || Strings either start with <code>double quotes</code> or <code>double single quotes</code>. They also support antiquotation (templating). Leading spaces are stripped with double single quotes. || <code>"Say ${pkgs.hello.name}"</code>,
 
<syntaxHighlight lang="nix">''first line
 
  second line
 
''</SyntaxHighlight>
 
|-
 
| Integers || || <code>5</code>
 
|-
 
| Floating-point numbers || Precision is limited || <code>1.2</code>
 
|-
 
| Path || relative paths will become absolute when evaluated, paths must contain a slash. <code><nixpkgs/pkgs></code> is also possible and will resolve to the folder incl. subfolders in your NIX_PATH || <syntaxHighlight>./hello/world
 
> /abs/path/to/hello/world
 
</syntaxHighlight>
 
<syntaxHighlight><nixpkgs/lib>
 
> /path/to/your/nixpkgs/lib</syntaxHighlight>
 
|-
 
| URI ||  || <code>http://example.org/foo.tar.bz2</code>
 
|-
 
| Boolean ||  || <code>true</code>,<code>false</code>
 
|-
 
| Null ||  || <code>null</code>
 
|-
 
| Lists || items are not separated by comma, can contain any other type || <syntaxHighlight lang=nix>[ 1 ./example.bin { hello="world"; }] </syntaxHighlight>
 
|-
 
| Sets || In other languages called <code>dicts</code>(py),<code>objects</code>(js) <code>hashes</code>(ruby), essentially a list of key-value pairs || <syntaxHighlight lang=nix>{ hello="world"; }.hello
 
> "world"</syntaxHighlight>
 
|-
 
| Functions || see below || <code>pattern: body</code>
 
|}
 
A detailed description of all types can be found in [https://nixos.org/nix/manual/#ssec-values The Nix manual]
 
 
 
=== Functions ===
 
 
 
Functions have the following form: <code>pattern: body</code>
 
 
 
There are 3 patterns in how functions can be written:
 
 
 
# <code>f = a: a*a;</code> a single identifier, can be extended with multiple identifiers as <code>f = a: b: a*b;</code> as a partial function would be returned.
 
# <code>{ x, y ? "foo", z ? "bar", ... }: z + y + x</code> a set pattern, which can also set defaults. ellipse means that the function may receive extra arguments.
 
# <code>args@{ x, y, z, ... }: z + y + x + args.a</code> an <code>@</code> pattern which can be used to match extra arguments and store them in the <code>args</code> set.
 
 
 
=== Operators ===
 
 
 
Lower precedence means a stronger binding; ie. this list is sorted from strongest to weakest binding, and in the case of equal precedence between two operators, the associativity decides the binding.
 
 
 
{| class="wikitable"
 
!width="6%"| Prec
 
!width="13%"| Abbreviation
 
!width="24%"| Example
 
!width="13%"| Assoc
 
!width="42%"| Description
 
|-
 
| 1
 
| SELECT
 
| <code>e . attrpath [or def]</code>
 
| none
 
| Select attribute denoted by the attribute path <code>attrpath</code> from set <code>e</code>. (An attribute path is a dot-separated list of attribute names.) If the attribute doesn’t exist, return <code>default</code> if provided, otherwise abort evaluation.
 
|-
 
| 2
 
| APP
 
| <code>e1 e2</code>
 
| left
 
| Call function <code>e1</code> with argument <code>e2</code>.
 
|-
 
| 3
 
| NEG
 
| <code>-e</code>
 
| none
 
| Numeric negation.
 
|-
 
| 4
 
| HAS_ATTR
 
| <code>e ? attrpath</code>
 
| none
 
| Test whether set <code>e</code> contains the attribute denoted by <code>attrpath</code>; return true or false.
 
|-
 
| 5
 
| CONCAT
 
| <code>e1 ++ e2</code>
 
| right
 
| List concatenation.
 
|-
 
| 6
 
| MUL
 
| <code>e1 * e2</code>
 
| left
 
| Numeric multiplication.
 
|-
 
| 6
 
| DIV
 
| <code>e1 / e2</code>
 
| left
 
| Numeric division.
 
|-
 
| 7
 
| ADD
 
| <code>e1 + e2</code>
 
| left
 
| Numeric addition, or string concatenation.
 
|-
 
| 7
 
| SUB
 
| <code>e1 - e2</code>
 
| left
 
| Numeric subtraction.
 
|-
 
| 8
 
| NOT
 
| <code>!e</code>
 
| left
 
| Boolean negation.
 
|-
 
| 9
 
| UPDATE
 
| <code>e1 // e2</code>
 
| right
 
| Return a set consisting of the attributes in <code>e1</code> and <code>e2</code> (with the latter taking precedence over the former in case of equally named attributes).
 
|-
 
| 10
 
| LT
 
| <code>e1 &lt; e2</code>
 
| left
 
| Less than.
 
|-
 
| 10
 
| LTE
 
| <code>e1 &lt;= e2</code>
 
| left
 
| Less than or equal.
 
|-
 
| 10
 
| GT
 
| <code>e1 &gt; e2</code>
 
| left
 
| Greater than.
 
|-
 
| 10
 
| GTE
 
| <code>e1 &gt;= e2</code>
 
| left
 
| Greater than or equal.
 
|-
 
| 11
 
| EQ
 
| <code>e1 == e2</code>
 
| none
 
| Equality.
 
|-
 
| 11
 
| NEQ
 
| <code>e1 != e2</code>
 
| none
 
| Inequality.
 
|-
 
| 12
 
| AND
 
| <code>e1 &amp;&amp; e2</code>
 
| left
 
| Logical AND.
 
|-
 
| 13
 
| OR
 
| <code><nowiki>e1 || e2</nowiki></code>
 
| left
 
| Logical OR.
 
|-
 
| 14
 
| IMPL
 
| <code>e1 -&gt; e2</code>
 
| none
 
| Logical implication (equivalent to <code><nowiki>!e1 || e2</nowiki></code>).
 
|}
 
 
 
Source: [https://gist.github.com/joepie91/c3c047f3406aea9ec65eebce2ffd449d Gist of joepie91]
 
 
 
=== Imports ===
 
 
 
<code>import</code> loads, parses and imports the nix expression stored in path. This keyword is essentially a builtin of nix but not a part of the language itself.
 
 
 
Usage:
 
<syntaxHighlight lang=nix>
 
  x = import <nixpkgs> {};
 
  y = trace x.pkgs.hello.name x;
 
</syntaxHighlight>
 
 
 
=== Notable constructs ===
 
 
 
Nix looks a lot like JSON with functions but also provides a number of very specialized constructs which can help you build clean and easy to read expressions. In this sub-chapter the most notable constructs will be shown by example:
 
 
 
==== <code>with</code> statement ====
 
The <code>with</code> statement introduces the lexical scope of a set into the expression which follows.
 
Common usages are:
 
'''On top of expressions''':
 
 
 
You will see the with statement a lot at the beginning of expression definition. Most of the time it is used to load the lib functions into the namespace for quick access.
 
<syntaxHighlight lang=nix>
 
{lib, ... }:
 
 
 
with lib;
 
{
 
  options = {
 
    networking.hosts = mkOption {
 
      type = with types; attrsOf ( listOf str);
 
      default = {};
 
    };
 
  };
 
  ... 
 
}
 
</syntaxHighlight>
 
instead of:
 
<syntaxHighlight lang=nix>
 
{lib, ... }:
 
{
 
  options = {
 
    networking.hosts = lib.mkOption {
 
      type = lib.types.attrsOf ( lib.types.listOf lib.types.str);
 
      default = {};
 
    };
 
  };
 
  ... 
 
}
 
</syntaxHighlight>
 
 
 
'''In package input definitions''':
 
<syntaxHighlight lang=nix>
 
{pkgs}:
 
{
 
  ...
 
  buildInputs = with pkgs; [ curl php coreutils procps ffmpeg ];
 
}
 
</syntaxHighlight>
 
Instead of :
 
<syntaxHighlight lang=nix>
 
{pkgs}:
 
{
 
  ...
 
  buildInputs = [ pkgs.curl pkgs.php pkgs.coreutils pkgs.procps pkgs.ffmpeg ];
 
}
 
</syntaxHighlight>
 
 
 
'''In the package meta tag''':
 
<syntaxHighlight lang=nix>
 
{stdenv, ...}:
 
{
 
  ...
 
  meta = with stdenv.lib; {
 
    license = with licenses; [ lgp3 gpl3 ];
 
    maintainers = with maintainers; [ adisbladis lassulus ];
 
  };
 
}
 
</syntaxHighlight>
 
Instead of :
 
<syntaxHighlight lang=nix>
 
{stdenv, ...}:
 
{
 
  ...
 
  meta = {
 
    license = [ stdenv.lib.licenses.lgp3 stdenv.lib.licenses.gpl3 ];
 
    maintainers = [ stdenv.lib.maintainers.adisbladis stdenv.lib.maintainers.lassulus ];
 
  };
 
}
 
</syntaxHighlight>
 
 
 
'''In a default.nix of an external package''':
 
<syntaxHighlight lang=nix>
 
with import <nixpkgs> {};
 
stdenv.mkDerivation rec {
 
    name = "mytool-env";
 
    src = ./.;
 
    buildInputs = with pkgs;[
 
      python34
 
      python34Packages.docopt
 
    ];
 
 
 
    shellHook =''
 
      export HISTFILE=$PWD/histfile
 
    '' ;
 
}
 
</syntaxHighlight>
 
 
 
==== <code>let ... in</code> statement ====
 
 
 
With <code>let</code> you can define local variables which can also reference to self without the need of the <code>rec</code> construct. This feature is use inside expressions to prepare variables which become part of an output.
 
The usage of <code>let</code> is comparable to the [http://zvon.org/other/haskell/Outputsyntax/letQexpressions_reference.html Haskell let expression]
 
<syntaxHighlight lang=nix>
 
let
 
  a = 1;
 
  b = 2;
 
in  a + b
 
=> 3
 
</syntaxHighlight>
 
 
 
==== <code>inherit</code> statement ====
 
 
 
The <code>inherit</code> expression can be used to copy variables from the surrounding lexical scope. A typical use case is to declare the version or name of a derivation in the expression and reuse this parameter in the function to fetch the source.
 
 
 
This is a typical python package derivation as the fetchPypi function also requires <code>pname</code> and <code>version</code> as input:
 
<syntaxHighlight lang=nix>
 
buildPythonPackage rec {
 
  pname = "hello";
 
  version = "1.0";
 
  src = fetchPypi {
 
    inherit pname version;
 
  sha256 = "01ba..0";
 
  };
 
}
 
</syntaxHighlight>
 
 
 
==== <code>rec</code> statement ====
 
 
 
The <code>rec</code> expression turns a basic set into a set where self-referencing is possible. This can be used when the <code>let</code> expression would create too much clutter. It is often seen in package derivation descriptions.
 
 
 
Sample usage:
 
<syntaxHighlight lang=nix>
 
rec {
 
  x = y - 100;
 
  y = 123;
 
}.x
 
=> 23
 
</syntaxHighlight>
 
 
 
== Development tools ==
 
 
 
=== Syntax highlighting & editor modes ===
 
 
 
Nix language has decent syntax highlighting (SH) support among popular code editors, but refactoring/autocomplete is still rare.
 
 
 
Reference: [[Editor Modes for Nix Files]]
 
 
 
== Tips & Tricks ==
 
 
 
=== Finding a (function/package/…) definition ===
 
 
 
Nix is often criticized that it has no working “jump to definition”. The good news is that you can have something very similar by using a regular expression:
 
 
 
If your package is named <code>hello</code>, searching for the regular expression <code>hello\ =</code> lists all nix symbol definitions that are named this way. In many cases there’s only two or three results. You should find what you are searching for easily.
 
 
 
This trick even works for functions, because their arguments come *after* the equals sign. If you search for <code>mkDerivation\ =</code> for example, you will see that there is more than one definition of that symbol in <code>nixpkgs</code>, but at least all definitions are shown.
 
 
 
You will also notice that searching with <code>grep</code> takes quite a while on a large repository like <code>nixpkgs</code>. Tools like <code>ag</code> (The Silver Searcher) and <code>rg</code> (ripgrep) are orders of magnitudes faster (especially on modern SSDs).
 
 
 
If you don’t have a <code>nixpkgs</code> checkout at hand, you can use the repo search at [https://search.nix.gsc.io/ search.nix.gsc.io]. This even searches in all repositories of the [https://github.com/NixOS/ NixOS Github organization].
 
 
 
Another trick that only works for functions, is evaluating the function on the <code>nix repl</code>:
 
<pre>
 
nix-repl> pkgs.lib.strings.makeBinPath
 
«lambda @ /home/user/nixpkgs/lib/strings.nix:94:42»
 
</pre>
 
This doesn't work for non-functions or builtin functions, which show <code>«primop»</code>. It will always find the actual lambda, not an attribute that reexports a partial application, for example.
 
 
 
=== Convert a string to an (<code>import</code>-able) path ===
 
 
 
<syntaxHighlight lang=nix>
 
nix-repl> "/home/bernd/folder"
 
"/home/bernd/folder"
 
 
 
nix-repl> :t "/home/bernd/folder"
 
a string
 
 
 
nix-repl> builtins.toPath "/home/bernd/folder"
 
"/home/bernd/folder"
 
 
 
nix-repl> :t builtins.toPath "/home/bernd/folder"
 
a string
 
 
 
nix-repl> /. + builtins.toPath "/home/bernd/folder"
 
/home/bernd/folder
 
 
 
nix-repl> :t /. + builtins.toPath "/home/bernd/folder"
 
a path
 
</syntaxHighlight>
 
 
 
In contrast to what [https://nixos.org/nix/manual/#builtin-toPath <code>builtins.toPath</code>] suggests, it does not result in a path, but only checks whether the string is an absolute path, and normalizes it. The trick is to prepend the <code>/.</code> (“root”) path literal, which converts the result to a nix path (that will be copied to the store when used in a derivation).
 
 
 
Be careful not to confuse it with <code>./.</code>, which is the “directory of the current nix file” path literal, and will result in something like <code>/my/scripts/folder/home/bernd/folder</code> (provided you are in <code>/my/scripts/folder</code>).
 
 
 
This trick might be helpful in combination with [https://nixos.org/nix/manual/#builtin-getEnv <code>builtins.getEnv</code>], which returns a string (which might be a path). Be careful, depending on environment variables introduces heavy non-determinism and might lead to rebuilds!
 
 
 
If you need to '''build a path from a mix of paths and strings variables''', you can concatenate strings and paths, but you need to be careful of the evaluation order because Nix removes trailing  <code>/</code>.
 
 
 
For example if you need to concatenate <code>/data</code> with a variable call <code>my_var</code> you need to add parenthesis:
 
 
 
<syntaxHighlight lang=nix>
 
nix-repl> let my_var = "tmp"; in /data + "/" + my_var  # WRONG
 
/datatmp
 
 
 
nix-repl> let my_var = "tmp"; in /data + ("/" + my_var) # Better :)
 
/data/tmp
 
</syntaxHighlight>
 
 
 
=== Writing update scripts / Referencing a relative path as string ===
 
 
 
Nix has relative path syntax that describes files relative to the current nix file, for example
 
 
 
<syntaxHighlight lang=nix>
 
with import <nixpkgs> {};
 
let textdata = ../foo.txt;
 
in runCommand "alldata" {} ''
 
  echo "=this is a header=" >> $out
 
  cat ${textdata} >> $out
 
''
 
</syntaxHighlight>
 
 
 
If the file <code>../foo.txt</code> are needed by evaluation, it is copied to the nix store first, so the script in the resulting <code>drv</code> file looks like this:
 
 
 
<code>
 
"echo \"=this is a header=\" >> $out\ncat /nix/store/dcaph3ib0vq0c27bqzw2vhrakk272mga-foo.txt >> $out\n"
 
</code>
 
 
 
Notice the <code>/nix/store</code> path of <code>foo.txt</code>. When we build the file:
 
 
 
<pre>
 
$ nix-build code.nix
 
these derivations will be built:
 
  /nix/store/bfv13hxqlwll398y5vi3wn44raw48yva-alldata.drv
 
building '/nix/store/bfv13hxqlwll398y5vi3wn44raw48yva-alldata.drv'...
 
/nix/store/9fav4aw2fs8ybaj06gg6cjzz7bkqf461-alldata
 
 
 
$ cat /nix/store/9fav4aw2fs8ybaj06gg6cjzz7bkqf461-alldata
 
=this is a header=
 
this
 
is
 
some
 
data
 
</pre>
 
 
 
Now, what if we don’t want to import the data file into the store, but still reference the absolute path of that file? We use <code>toString</code>:
 
 
 
<syntaxHighlight lang=nix>
 
with import <nixpkgs> {};
 
let textdata = toString ../foo.txt;
 
in writeScript "update-foo.sh" ''
 
  echo "updating foo.txt!"
 
  cat "additional new data" >> ${lib.escapeShellArg textdata}
 
''
 
</syntaxHighlight>
 
 
 
In this example we use the actual absolute path of the file to write a script (notice the change from <code>runCommand</code> to <code>writeScript</code>, which are both helper functions from <code>nixpkgs</code>). This script can update the <code>foo.txt</code> file when it is run by bash:
 
 
 
<pre>
 
$ cat $(nix-build code.nix)
 
echo "updating foo.txt!"
 
echo "additional new data" >> '/home/philip/tmp/foo.txt'
 
 
 
$ bash $(nix-build code.nix)
 
updating foo.txt!
 
 
 
$ cat foo.txt
 
this
 
is
 
some
 
data
 
additional new data
 
</pre>
 
 
 
Bear in mind that this makes the absolute path vary between different systems. The users Bob and Alice are going to get different scripts, because the paths of their home folders differ: <code>/home/bob/foo.txt</code> and  <code>/home/alice/foo.txt</code>; so it’s not reproducible.
 
 
 
We can use this trick to update the sources of nix expressions (for example by generating a script which updates a json file with the software’s hashes).
 
 
 
[[Category:Discussion]]
 
 
[[Category:Nix Language]]
 
[[Category:Nix Language]]
[[Category:Incomplete]]
 

Latest revision as of 22:13, 4 March 2024

The Nix language is designed for conveniently creating and composing derivations – precise descriptions of how contents of existing files are used to derive new files.

Nix Reference Manual

Note: Check the Nix Reference Manual on the Nix Language for up-to-date documentation and Nix language basics for a gentle introduction.

See also