Posts tagged testing


Safer Nix installation

:: linux, nix, packaging, sandbox, shell, system, test, testing, tutorial

By: Maciej Barć

Nix is useful for quickly testing out software and providing a strict environment that can be shared between people.

Today I’m trying out Nix again, this time I want to do it my way.

Installation process

Nix store

I know Nix needs “Nix store” installation on / (the system root).

Create it manually to prevent the installation script from calling sudo. 1st I switch to the root account, and then I run:

1
2
mkdir -p -m 0755 /nix
chown -R xy:xy /nix

Running the install script

Download the Nix install script and examine the contents.

1
curl -L https://nixos.org/nix/install > nix_install.sh

Then, run it with --no-daemon to prevent it running as system service.

1
sh ./nix_install.sh --no-daemon
performing a single-user installation of Nix...
copying Nix to /nix/store...
installing 'nix-2.20.1'
building '/nix/store/1ahlg3bviy174d6ig1gn393c23sqlki6-user-environment.drv'...
unpacking channels...
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

. /home/xy/.nix-profile/etc/profile.d/nix.fish

in your shell.

Wait!

modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

That’s very rude!

Stopping Nix from making a mess

I need to prevent Nix from mess up with my environment when I do not want it to. Nix puts some code into the Bash, ZSH and Fish initialization files during installation to ease it’s use. I do not want that since I do not want Nix to meddle with my environment without me knowing it.

I keep my .bash_profile and .zshenv in a stow-managed git repo so I can just cd into my repo and do git reset --hard, but for you will have to revert those files to their old forms manually.

Playing with Nix

We do not have nix in PATH but we still can launch it. Nix executables are located inside ~/.nix-profile/bin/.

By invoking nix-shell one can create a ephemeral environment containing only packages specified after the -p flag. I always add -p nix to have the Nix tools available also inside the spawned environment.

I will test out chibi (small Scheme interpreter) + rlwrap (REPL support for software lacking it) inside a Nix ephemeral environment:

1
~/.nix-profile/bin/nix-shell -p nix chibi rlwrap

Inside the spawned shell:

1
rlwrap chibi-scheme

In the chibi REPL, let’s see the contents of the PATH environment variable:

1
(get-environment-variable "PATH")

And exit the Scheme REPL:

1
(exit)

After the playtime, run garbage collection:

1
~/.nix-profile/bin/nix-collect-garbage

Portage CI

:: continuous integration, gentoo, portage, quality assurance, testing

By: Maciej Barć

Potential benefits

Running tests

  • test BEFORE (src_test) and AFTER (pkg_postinst) installation
  • test if and how services break if they are not reloaded
  • test buildsystem configuration
  • sandbox enforces strict and consistent build rules
  • benchmarking with different compilation flags and libraries versions/releases

Configuration matrix

We can test across Cartesian product of different configuration settings, like:

  • USE flags
  • MAKEOPTS
  • CFLAGS, CXXFLAGS, CPPFLAGS, LDFAGS, RUSTFLAGS, etc.
  • arches (cross-compilation or run in qemu)
  • static linking
  • supported releases & versions of libraries (eg. glibc & musl)

Also, we could create diffs of installed files across different merges.

Reproducibility

  • mini overlay with ::gentoo or any other (eg. company's own) as master
  • record VCS (eg. git) hash of the dependent overlays

Binaries

  • grab dependencies from binhosts
  • distribute built binaries (maybe upload to a company's own artifacts server)
  • make AppImages

Getting there

How do we run this?

Do we want to write a proper tool, which we probably do or do we just run Portage + shells scripts?

Do we want to run under root, user, in eprefix, maybe all in docker?

Configuration files

The .portci directory contains the configuration.

Bug 799626

Link: bugs.gentoo.org/799626

Instead of using Ansible, Python, Yaml or Scheme we might use something similar to this for simple configuration, or if gets merged to upstream Portage the better.

Worth mentioning is the idea from Michał Górny who proposes to configure portage with toml files, like the example given in the bug report.

1
2
3
4
5
6
7
8
[package.unmask]
~virtual/libcrypt-2

[package.use.mask]
sys-libs/libxcrypt -system -split-usr

[package.use.force]
sys-libs/glibc -crypt

Also, package.x + Toml == a match made in heaven, it looks very nice!