Future of Nix and NixOS

From NixOS Wiki
Jump to: navigation, search

There are some hard-to-solve feature-requests for Nix and NixOS, which hopefully will be solved in the future.

Space efficiency of the Nix store

Possible solutions:

  • Store the Nix store on a filesystem with reflink support: Btrfs, XFS
  • Store the Nix store on a ZFS filesystem with block-level deduplication. This seems to give poor results [1].
  • Use a deduplication tool (TODO how do they work? could they be integrated into Nix?)

Network efficiency of system upgrades

Currently, fetching a new version of a package from a Binary Cache will fetch all files of the new version, without any deduplication, causing lots of network traffic. Ideally, fetching a new version would fetch only the difference between old version and new version.

Possible solutions:

See also:

Distributed Binary Cache

Currently, binary caches are centralized for better performance.

Memory usage of Nix evaluation

Possible solutions:

  • Swap
  • TODO remote evaluation?

Intellisense for the Nix language

Intellisense for Nix is part of the bigger goal Make Nix more user friendly.

Keywords: static analysis, dynamic analysis, incremental evaluation

See also:

Intellisense for embedded code fragments in Nix expressions

For example, the buildPhase of a stdenv.mkDerivation contains Bash code.

Integration with large package sets

The evaluation of nixpkgs must work offline, with a static set of inputs. Other package managers send dynamic queries to servers for dependency resolution. To integrate such package managers into nixpkgs, we need an offline version of this dependency resolution, to get the file URLs and checksums of all dependencies.

Number of packages

Number of modules per language, based on libraries.io and modulecounts.com

  • NPM: 4M packages
  • Maven: 500K packages
  • Go: 500K packages
  • PyPi: 500K packages
  • Cargo: 100K packages

Note: On average, every package has about 10 releases (10 different versions).

For reference, nixpkgs has about 70K packages per https://repology.org/

NodeJS

Machine-level store

Currently, NPM packages are installed by copying all files to a node_modules folder in the Nix store. Ideally, every NPM package would be stored only once in the Nix store, and symlinked into the node_modules folder, where it is used.

The problem is, nodejs would follow all symlinks, so it would escape the node_modules folder and break the resolution of sub-dependencies. Package managers like pnpm use hardlinks to solve this problem, but hardlinks cannot be stored in the Nix store.

Possible solutions:

  • nodejs-hide-symlinks, a LD_PRELOAD library, which stops the symlink-resolution before the Nix store, and makes symlinks appear as regular files to nodejs
  • file-deduplication in the Nix store

Python

Cargo

Cargo package (for the Rust language) are stored in the Github repository rust-lang/crates.io-index, so its trivial to fetch a snapshot of all Cargo packages.

Retrieved from "https://nixos.wiki/index.php?title=Future_of_Nix_and_NixOS&oldid=12874"