SysStat is a amazing tool. In the age where telegraf and grafana are all the rage everybody forgot about the good old sysstat.

Selected command examples

• iostat -d -p nvme0n1 3 - disk I/O for a NVME drive (nvme0n1),
• sar -n DEV 3 - network throughput,
• sar -h -r 3 - memory usage,
• sar -P ALL 3 - CPU utlization
• sar -q 3 - system load levels,
• sar -A 3 - all the metrics.

Gathered info

qlist app-admin/sysstat | grep /usr/bin/

The app-admin/sysstat contains the following binaries and their respective statictic fields:

• sar - general utilization statistics,
• cifsiostat - CIFS,
• iostat - device input/output,
• mpstat - processors,
• pidstat - Linux tasks,
• tapestat - tape (yes, the real tape disks).

Installation

Gentoo

1
2
3

emerge --noreplace --verbose sys-process/cronie app-admin/sysstat
rc-service cronie start
rc-update add cronie default

Files

By default (on Gentoo): * sa (the collector) saves statistics to /var/log/sa, * /etc/sysstat is the configuration file * cron jobs are run via the *system* cronjob table.

Nix is useful for quickly testing out software and providing a strict environment that can be shared between people.

Today I’m trying out Nix again, this time I want to do it my way.

Installation process

Nix store

I know Nix needs “Nix store” installation on / (the system root).

Create it manually to prevent the installation script from calling sudo. 1st I switch to the root account, and then I run:

1
2

mkdir -p -m 0755 /nix
chown -R xy:xy /nix

Running the install script

Download the Nix install script and examine the contents.

1

curl -L https://nixos.org/nix/install > nix_install.sh

Then, run it with --no-daemon to prevent it running as system service.

1

sh ./nix_install.sh --no-daemon

performing a single-user installation of Nix...
copying Nix to /nix/store...
installing 'nix-2.20.1'
building '/nix/store/1ahlg3bviy174d6ig1gn393c23sqlki6-user-environment.drv'...
unpacking channels...
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

. /home/xy/.nix-profile/etc/profile.d/nix.fish

in your shell.

Wait!

modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

That’s very rude!

Stopping Nix from making a mess

I need to prevent Nix from mess up with my environment when I do not want it to. Nix puts some code into the Bash, ZSH and Fish initialization files during installation to ease it’s use. I do not want that since I do not want Nix to meddle with my environment without me knowing it.

I keep my .bash_profile and .zshenv in a stow-managed git repo so I can just cd into my repo and do git reset --hard, but for you will have to revert those files to their old forms manually.

Playing with Nix

We do not have nix in PATH but we still can launch it. Nix executables are located inside ~/.nix-profile/bin/.

By invoking nix-shell one can create a ephemeral environment containing only packages specified after the -p flag. I always add -p nix to have the Nix tools available also inside the spawned environment.

I will test out chibi (small Scheme interpreter) + rlwrap (REPL support for software lacking it) inside a Nix ephemeral environment:

1

~/.nix-profile/bin/nix-shell -p nix chibi rlwrap

Inside the spawned shell:

1

rlwrap chibi-scheme

In the chibi REPL, let’s see the contents of the PATH environment variable:

1

(get-environment-variable "PATH")

And exit the Scheme REPL:

1

(exit)

After the playtime, run garbage collection:

1

~/.nix-profile/bin/nix-collect-garbage

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

• Ebuild repository
• Creating an ebuild repository

Set up your repo with eselect-repository

Install the my-org repository:

1

eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1

emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1

echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1

emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1

0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1

0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1

0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8

#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1

0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3

pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

• pkg_postinst
• Phase Functions

Example Gentoo overlays

• xgqt/myov
• gentoo-racket/gentoo-racket-overlay
• gentoo-mirror/guru

Firefox began as the first open source browser to live through the browser wars, overcoming Microsoft’s Internet Explorer and continues to deliver competition-smashing technology to this day.

Chromium code

The only advantages of Chromium are that it was adopted by Electron and spread partially because of a more liberal license and Google’s own efforts.

Google will never be able to cope with the worst imaginable code base of Chromium.

Chromium is near-impossible to compile

On a 4cores/8threads Ryzen CPU Chromium compiles in ~12h and requires at least 20GB of disk space for build. At the same time Firefox compiles in ~1.5h and requires ~8GB for disk space.

Programming language adoption

Additionally Firefox team was able to rewrite a very large portion of Firefox codebase in Rust which improved the browser’s safety. There were attempts to add rust to Chromium but they all are in more of a addon-like fashion.

Porting to UNIXes

Because Chromium is extremely large it’s very hard to port and maintain for Linux and BSD based systems. There were numerous bugs with Chromium’s UI on Linux that cause crashes on pressing random controls. I believe Google has no Linux testers beside the “free software freeloaders” (wink, wink, IBM :P).

Anti-competition

This days Google has to result to dirty tactics where certain Google-owned websites would either refuse to work on Firefox or give a fake performance hit that is entirely caused by malicious JavaScript code.

Several popular FOSS-related sources have covered this news recently, check them out on the WWW.

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

1. Copy old config

   1 2	cd /usr/src cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/

2. Remove old kernel build directories

   1	rm -r linux-6.1.31-gentoo

3. Run initial preparation

   1	( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )

4. Call genkernel

   1 2 3 4 5 6 7 8 9 10 11 12

   genkernel \ --no-menuconfig \ --no-clean \ --no-clear-cachedir \ --no-cleanup \ --no-mrproper \ --lvm \ --luks \ --mdadm \ --nfs \ --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')" \ all

5. Rebuild the modules

   If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

   1	emerge -1 @module-rebuild

System preparation

Qemu

Emerge qemu with static-user USE enabled and your wanted architectures.

1
2
3
4
5
6
7
8

app-emulation/qemu      QEMU_SOFTMMU_TARGETS: aarch64 arm x86_64
app-emulation/qemu      QEMU_USER_TARGETS: aarch64 arm x86_64

app-emulation/qemu      static-user
dev-libs/glib           static-libs
sys-apps/attr           static-libs
sys-libs/zlib           static-libs
dev-libs/libpcre2       static-libs

OpenRC

Enable qemu-binfmt:

1

rc-update add qemu-binfmt default

Start qemu-binfmt:

1

rc-service qemu-binfmt start

Chrooting

• select chroot location (eg /chroots/gentoo-arm64-musl-stable)
• unpack the desired rootfs
• create needed directories
  • mkdir -p /chroots/gentoo-arm64-musl-stable/var/cache/distfiles
• execute bwrap
  • with last ro-bind mount the qemu emulator binary (eg qemu-aarch64)
  • execute the mounted emulator binary giving it a shell program (eg bash)

Chroot with bwrap:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

bwrap                                                       \
    --bind /chroots/gentoo-arm64-musl-stable /              \
    --dev /dev                                              \
    --proc /proc                                            \
    --perms 1777 --tmpfs /dev/shm                           \
    --tmpfs /run                                            \
    --ro-bind /etc/resolv.conf /etc/resolv.conf             \
    --bind /var/cache/distfiles /var/cache/distfiles        \
    --ro-bind /usr/bin/qemu-aarch64 /usr/bin/qemu-aarch64   \
    /usr/bin/qemu-aarch64 /bin/bash -l

User-mode

By default you would probably have something like this, the user-mode network:

1
2
3
4
5

<interface type="user">
  <mac address="00:00:00:00:00:00"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

Bridge

Bridges can be easily created using the NetworkManager’s TUI tool called nmtui.

Bridge XML configuration for Libvirt

1
2
3
4
5
6
7
8

<interface type="bridge">
  <mac address="00:00:00:00:00:00"/>
  <source bridge="br1"/>
  <target dev="vnet2"/>
  <model type="virtio"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
</interface>

Sysctl options

Be sure the following options are enabled (1):

• net.ipv4.ip_forward
• net.ipv4.conf.all.send_redirects

and the following options are disabled (0):

• net.bridge.bridge-nf-call-iptables