Update a package
Introduction
This is the easiest and more straightforward way I know to modify and expand nixpkgs. I hope this article will help future newbies so the experts can focus on other stuff.
This document is not intended as a formal approach to learn nix, nixpkgs, NixOS or systems administration. It is just a sort of tutorial for newbies to be able to easily fix and contribute trivial improvements. I believe this approach will help newbies such as myself to get (slowly and inefficiently) educated on the nix language, the nix package manager and NixOS.
The official nixpkgs sources are managed on a git repository at GitHub (https://github.com/NixOS/nixpkgs). And GitHub implements most of the tools for the nixpkgs development workflow.
This tutorial was designed for NixOS, and may not always apply on other systems. If you want reproduce the environment used in this tutorial exactly, you can git reset --hard 111c8db50038
in your local copy of Nixpkgs.
Cloning the "NixOS/nixpkgs" repository on GitHub
- I had to sign up at GitHub. This is free (as in beer).
- Then I visited the official nixpkgs repository and hit the fork button.
- This results in your own somewhat-independent repository: https://github.com/your_username/nixpkgs
Retrieving a working copy of your repository
- Now you can clone your repository somewhere on your system:
mkdir ~/devel/
cd ~/devel/
git clone https://github.com/your_username/nixpkgs.git \
--depth=1 # Prevents cloning the entire history, which is multiple gigabytes
cd nixpkgs/
git remote add upstream https://github.com/NixOS/nixpkgs.git
- When the clone is not recent or you suspect there have been changes upstream (in official nixpkgs):
cd ~/devel/nixpkgs/
git fetch upstream
git merge upstream/master
- You should get a git repository with the whole nixpkgs tree under ~/devel/nixpkgs/
- Unless explicitly stated, for the rest of the document we are working from this directory:
cd ~/devel/nixpkgs/
- Just to ensure everything is clear, on this directory there should be two main entries: the default.nix file and the pkgs/ directory.
- From now on, pkgs/ will refer to this nixpkgs/pkgs/ so if you keep the equivalent of ~/devel/nixpkgs/ as your CWD (current working directory), most commands and pathnames can be copied and pasted in real life.
- So for example, on this document pkgs/top-level/ refers to ~/devel/nixpkgs/pkgs/top-level/
- This looks perhaps too verbose, but I feel this extra context would have clarified many ambiguous explanations to me.
- Also, when talking about directories I will try to always include the trailing slash.
Following the white rabbit
- I will center this tutorial on the nix expression that packages the i3 window manager, as this expression is quite simple and it was contributing to this expression that I got to understand this whole workflow I am documenting.
- In ~/devel/nixpkgs/default.nix there is an import statement that reads import ./pkgs/top-level/all-packages.nix
- I have the feeling that most references between nix sources (nix expressions) in nixpkgs are stated as relative paths pointing up and down the nixpkgs/ directory hierarchy.
- Please have a look at this pkgs/top-level/all-packages.nix. There are many more or less complex definitions at the beginning, and then a lot of declarations such as:
i3 = callPackage ../applications/window-managers/i3 { };
i3lock = callPackage ../applications/window-managers/i3/lock.nix { };
i3status = callPackage ../applications/window-managers/i3/status.nix { };
and pango = callPackage ../development/libraries/pango/1.30.x.nix { };
- Now let's have a look at the contents of pkgs/applications/window-managers/i3/:
$ ls -1 pkgs/applications/window-managers/i3/
default.nix
lock.nix
status.nix
- The first assignment i3 = callPackage ... refers implicitly to default.nix on the pkgs/applications/window-managers/i3/ directory.
- The second and third assignments are explicitly referring to lock.nix and status.nix files on this pkgs/applications/window-managers/i3/ directory.
- Each of these three files contain somewhat equivalent nix expressions that the nix package manager will consider as packages thanks to the callPackage on pkgs/top-level/all-packages.nix.
The actual package "recipe"
- We will dive (not too deep, though) on pkgs/applications/window-managers/i3/default.nix which is the nix expression that corresponds to the package for the i3 window manager:
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
libXcursor, coreutils, perl, pango }:
stdenv.mkDerivation rec {
name = "i3-${version}";
version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
postPatch = ''
patchShebangs
'';
configurePhase = "makeFlags=PREFIX=$out";
meta = {
description = "i3 is a tiling window manager";
homepage = "http://i3wm.org";
maintainers = [ stdenv.lib.maintainers.garbas ];
license = stdenv.lib.licenses.bsd3;
};
}
- As an example of the trivial fixes and improvements I was talking about, you can have a look at two changesets (commits in the git terminology) I did:
- adding the pango dependency to enable antialiased fonts on i3
- and updating from upstream version 4.5 to 4.5.1
- As I anticipated above, I am not going to explain what's the purpose of the list of buildInputs, how a list enclosed by square brackets separated by spaces differs from the list at the beginning or the relevance of the colon that follows its closing curly brace. I just don't know so much yet.
Making changes
- On your local clone of your forked repository you can edit whatever you want.
- That is, in our example, under ~/devel/nixpkgs/
- As you can see on the commits cited above I edited pkgs/applications/window-managers/i3/default.nix. I did several changes at once:
- Added pango to both dependency lists. This pango refers to pango = callPackage ... in pkgs/top-level/all-packages.nix
- Changed the version string.
- Changed the checksum for the tarball on the sha256 string.
- Removed a line on the postPatch string. This line was a sed command which was erasing some lines from the file common.mk, which is part of the sources in the tarball, before building the application.
- And in an attempt to extract something positive from the issue, we are going to review an additional unfortunate change I made on pkgs/top-level/all-packages.nix
- To figure out the checksum for the tarball you have the nix-prefetch-url utility. In addition to returning the sha256 checksum it will also leave the downloaded blob on your nix store, thus avoiding additional downloads when you test the build process. You get the path to the downloaded blob and the sha256sum on the last two lines of the command's output:
$ nix-prefetch-url 'http://i3wm.org/downloads/i3-4.5.1.tar.bz2'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 858k 100 858k 0 0 866k 0 --:--:-- --:--:-- --:--:-- 867k
path is "/nix/store/clvs3jvyk24bskkn5pfd7fjpyaa1svsx-i3-4.5.1.tar.bz2"
1wi3p1s4z7r6kvzwxlg59fx6bfiqc2mlybhqmqddf8aaghf5zrds
- Almost out of scope here, but you can easily review your changes. Go get some git documentation if you need:
$ git diff
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
- libXcursor, coreutils, perl }:
+ libXcursor, coreutils, perl, pango }:
stdenv.mkDerivation rec {
name = "i3-${version}";
- version = "4.5";
+ version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
- sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+ sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
- libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+ libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
postPatch = ''
- sed -i -e '/^# Pango/,/^$/d' common.mk
patchShebangs .
'';
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0d3e4cc..a1c0d4b 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7224,7 +7224,9 @@ let
hydrogen = callPackage ../applications/audio/hydrogen { };
- i3 = callPackage ../applications/window-managers/i3 { };
+ i3 = callPackage ../applications/window-managers/i3 {
+ cairo = cairo.override { xcbSupport = true; };
+ };
i3lock = callPackage ../applications/window-managers/i3/lock.nix {
cairo = cairo.override { xcbSupport = true; };
Testing the changes
- The changes made to these nix expressions are affecting the build process and therefore the behavior of the resulting binaries, configuration and other support files (man pages, scripts, etc.). Additionally there might be other packages or system services that depend on the ones we are altering, so these changes should be carefully considered, reviewed and thoroughly tested before they can be submitted and accepted on the official nixpkgs repository.
- The worst case scenario is that a lot of people will get a broken package (or a broken system, even unable to boot) if the changes have unexpected side effects or if the changes are wrong and are reviewed too lightly. In special this happens more often for architectures other than 32 and 64-bit x86, because these are not so popular and thus less known and tested.
- If your current directory is, as we said, ~/devel/nixpkgs/, you can request nix to build the modified package (see man nix-build):
nix-build -A i3
- In this invocation the i3 in -A i3 refers to i3 = callPackage ... in pkgs/top-level/all-packages.nix. In the same way as the pango we talked about above, both names are attributes from the top-level Nix expression being evaluated. In this case ~/devel/nixpkgs/default.nix
- Here I would have detected the error I introduced on pkgs/top-level/all-packages.nix. Too bad I didn't still know how to test my changes:
$ nix-build -A i3
error: function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
(use `--show-trace' to show detailed location information)
- Let's follow the advice by adding --show-trace to the nix-build invocation:
$ nix-build --show-trace -A i3
error: while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:98:35':
while evaluating the function at `~/devel/nixpkgs/pkgs/lib/customisation.nix:59:24':
while evaluating the builtin function `isAttrs':
function at `~/devel/nixpkgs/pkgs/applications/window-managers/i3/default.nix:1:1' called with unexpected argument
- Not that I got any more clues from this extra step. But it turned out that there is no mention to cairo on pkgs/applications/window-managers/i3/default.nix so it was illegal to invoke the cairo.override ... thing inside the i3 = callPackage ... on pkgs/top-level/all-packages.nix:
i3 = callPackage ../applications/window-managers/i3 {
cairo = cairo.override { xcbSupport = true; }; # WRONG!
};
- So I would have reverted my changes to pkgs/top-level/all-packages.nix. Take care with this command as it is quite eager to undo your changes. But that's out of scope here. See some git documentation:
git checkout pkgs/top-level/all-packages.nix
- Now let's try again:
$ nix-build -A i3
/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1
- When the build succeeds, nix-build will leave the result linked as ./result on the current working directory:
$ stat -c %N ./result
File: "./result" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1"
- If convenient, you can review and fiddle with the resulting stuff:
$ find result/ -type f
result/include/i3/ipc.h
result/bin/i3-input
result/bin/i3-nagbar
result/bin/i3-msg
result/bin/i3-migrate-config-to-v4
result/bin/i3-sensible-terminal
result/bin/i3-sensible-pager
result/bin/i3-sensible-editor
result/bin/i3bar
result/bin/i3
result/bin/i3-config-wizard
result/bin/i3-dmenu-desktop
result/bin/i3-dump-log
result/etc/i3/config.keycodes
result/etc/i3/config
result/share/xsessions/i3.desktop
result/share/applications/i3.desktop
- And for some real-life testing we would invoke nix-env as explained above:
$ nix-env -f . -iA i3
replacing old `i3-4.5.1'
installing `i3-4.5.1'
building path(s) `/nix/store/49ag28h77sjmg3rygpsm83jxkhn5jzyn-user-environment'
created 2006 symlinks in user environment
- You might double-check that the binaries on $PATH are indeed the new ones:
$ stat `type -P i3` | head -n 1
File: "~/.nix-profile/bin/i3" -> "/nix/store/agld76p9rgvn3j610z8kgppz67f8kmg7-i3-4.5.1/bin/i3"
- It is quite annoying to test a window manager, i3 in special as it does not implement the --replace feature.
- And sometimes there are lots of other tricky possibilities. For example i3 invokes other binaries from the same package, such as i3bar. I lost a great deal of time trying to figure out why the i3 status bar was refusing to render pango fonts. That was because I was launching ~/devel/nixpkgs/result/bin/i3 instead of having the whole package replaced on my nix user environment. And the old i3bar from the nix store was on my $PATH. Obviously that was before I learned the nix-env invocation shown above.
Pushing the local changes to GitHub
Committing to the local repository
- The first step is to commit your local changes to the local repository.
- A quick overview of the affected stuff:
$ git status
# On branch example
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: pkgs/applications/window-managers/i3/default.nix
# modified: pkgs/top-level/all-packages.nix
#
no changes added to commit (use "git add" and/or "git commit -a")
- As we have just seen, you can review the pending changes with git diff
- You can limit the scope to review piece by piece:
$ git diff pkgs/top-level/all-packages.nix
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0d3e4cc..a1c0d4b 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7224,7 +7224,9 @@ let
hydrogen = callPackage ../applications/audio/hydrogen { };
- i3 = callPackage ../applications/window-managers/i3 { };
+ i3 = callPackage ../applications/window-managers/i3 {
+ cairo = cairo.override { xcbSupport = true; };
+ };
i3lock = callPackage ../applications/window-managers/i3/lock.nix {
cairo = cairo.override { xcbSupport = true; };
- This change, at the time, looked good to me. So I went ahead committing it:
$ git commit pkgs/top-level/all-packages.nix -m 'i3 window manager: cairo.override with xcbSupport enabled'
[example 731abb2] i3 window manager: cairo.override with xcbSupport enabled
1 file changed, 3 insertions(+), 1 deletion(-)
- Let's review what's left:
$ git diff pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
- libXcursor, coreutils, perl }:
+ libXcursor, coreutils, perl, pango }:
stdenv.mkDerivation rec {
name = "i3-${version}";
- version = "4.5";
+ version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
- sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+ sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
- libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+ libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
postPatch = ''
- sed -i -e '/^# Pango/,/^$/d' common.mk
patchShebangs .
'';
- These changes were made with two different purposes. I don't like commits mixing unrelated changes. It turns out that git can help here saving a lot of time and potential human error: git commit --patch pkgs/applications/window-managers/i3/default.nix.
- git will sequentially show "groups of changes" (it calls them hunks). For each hunk we are asked what we want to do; we can do several things than are displayed by typing a question mark. I will explain with my words the ones we are interested in right now. Obviously the explanation given by git prevails:
- type y to include the changes on this hunk in the commit
- type n to leave the changes on this hunk out of this commit
- type s to split the hunk if it consists of unrelated changes. You are then asked again about the smaller hunks.
- Note that this command is not destructive. Changes left out of the commit are not discarded.
- So I will do a first commit with all changes related to enabling pango support for i3. Please note how I explain git to s split the big hunk, y include the first small hunk on the commit, n skip the second hunk, n skip the third hunk, y and y to include the last two hunks:
$ git commit --patch pkgs/applications/window-managers/i3/default.nix -m 'i3 window manager: enable Pango support for anti-aliased fonts'
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index f8e4e16..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -1,21 +1,20 @@
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
- libXcursor, coreutils, perl }:
+ libXcursor, coreutils, perl, pango }:
stdenv.mkDerivation rec {
name = "i3-${version}";
- version = "4.5";
+ version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
- sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+ sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
- libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+ libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
postPatch = ''
- sed -i -e '/^# Pango/,/^$/d' common.mk
patchShebangs .
'';
Stage this hunk [y,n,q,a,d,/,s,e,?]? s
Split into 5 hunks.
@@ -1,6 +1,6 @@
{ fetchurl, stdenv, which, pkg-config, libxcb, xcbutilkeysyms, xcbutil,
xcbutilwm, libstartup_notification, libX11, pcre, libev, yajl,
- libXcursor, coreutils, perl }:
+ libXcursor, coreutils, perl, pango }:
stdenv.mkDerivation rec {
name = "i3-${version}";
Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y
@@ -4,7 +4,7 @@
stdenv.mkDerivation rec {
name = "i3-${version}";
- version = "4.5";
+ version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
@@ -8,7 +8,7 @@
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
- sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+ sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? n
@@ -12,6 +12,6 @@
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
- libstartup_notification libX11 pcre libev yajl libXcursor perl ];
+ libstartup_notification libX11 pcre libev yajl libXcursor perl pango ];
postPatch = ''
Stage this hunk [y,n,q,a,d,/,K,j,J,g,e,?]? y
@@ -16,6 +16,5 @@
postPatch = ''
- sed -i -e '/^# Pango/,/^$/d' common.mk
patchShebangs .
'';
Stage this hunk [y,n,q,a,d,/,K,g,e,?]? y
[example 729796b] i3 window manager: enable Pango support for anti-aliased fonts
1 file changed, 2 insertions(+), 3 deletions(-)
- As expected, the skipped hunks are still on the local file, uncommitted:
$ git diff pkgs/applications/window-managers/i3/default.nix
diff --git a/pkgs/applications/window-managers/i3/default.nix b/pkgs/applications/window-managers/i3/default.nix
index b977a6a..4173660 100644
--- a/pkgs/applications/window-managers/i3/default.nix
+++ b/pkgs/applications/window-managers/i3/default.nix
@@ -4,11 +4,11 @@
stdenv.mkDerivation rec {
name = "i3-${version}";
- version = "4.5";
+ version = "4.5.1";
src = fetchurl {
url = "http://i3wm.org/downloads/${name}.tar.bz2";
- sha256 = "1kiffcbvvjljqchw9ffgy9s8f9z06i8805jvjas58q5i2yxl5kcy";
+ sha256 = "bae55f1c7c4a21d71aae182e4fab6038ba65ba4be5d1ceff9e269f4f74b823f2";
};
buildInputs = [ which pkg-config libxcb xcbutilkeysyms xcbutil xcbutilwm
- As these changes are both related, we can issue a simple commit:
$ git commit pkgs/applications/window-managers/i3/default.nix -m "i3 window manager: version bump from 4.5 to 4.5.1"
[example 4c7daed] i3 window manager: version bump from 4.5 to 4.5.1
1 file changed, 2 insertions(+), 2 deletions(-)
- Let's check if we're done:
$ git status
# On branch example
# Your branch is ahead of 'origin/example' by 3 commit.
# (use "git push" to publish your local commits)
#
nothing to commit, working directory clean
Pushing to GitHub
- Now we want to push the local commits to our clone of NixOS/nixpkgs at GitHub:
$ git push
Counting objects: 27, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (19/19), 1.78 KiB, done.
Total 19 (delta 14), reused 14 (delta 12)
To https://github.com/your_username/nixpkgs.git
1c86db4..4c7daed example -> example
Sending a pull request to the official nixpkgs repository
step by step guide: pull request
- Now there are 3 commits on the example branch of out nixpkgs repository at GitHub.
- The idea is to propose the official NixOS/nixpkgs maintainers to include our changes on the official repository, so all the nixpkgs users can eventually benefit of our improvements.
- We should probably get in touch with the nixpkgs maintainers at #nixos to figure out the appropriate branch (e.g. master, x-updates or whatever) where the changes would be welcome.
- So now we have to formally request them to pull our changes, from our branch to the appropriate branch in NixOS/nixpkgs. Hence the name "pull request" which I don't feel so intuitive.
- There is a "Pull Request" button in GitHub. Push it. Go ahead.
- Now you will be asked to choose the origin repository and branch (branch example from our repository) and the destination repository and branch (e.g. branch master) on NixOS/nixpkgs).
- You should also enter a subject and a small abstract.
- And you have a last chance to review the commits that will be included on the pull request and the affected files.
- If you're ready to go, there is a green button that reads "send pull request".
- That's it :)
Waiting for your changes to appear on the nix channel
- If your pull request is accepted, your changes will be part of NixOS/nixpkgs.
- Or else you will have some kind of feedback, probably via GitHub and the nix-dev mailing list. You can follow up there or drop by #nixos to discuss whatever has to be discussed.
- Once the changes enter the official repository they are more likely to be seen and probably used by other users.
- In addition to that, if they enter the master branch, Hydra will eventually build the improved versions or new packages, and then the resulting binaries will be available on the NixOS channel.