Posts tagged linux
Nix is useful for quickly testing out software and providing a strict environment that can be shared between people.
Today I’m trying out Nix again, this time I want to do it my way.
Installation process
Nix store
I know Nix needs “Nix store” installation on /
(the system root).
Create it manually to prevent the installation script from calling sudo
. 1st I switch to the root account, and then I run:
|
mkdir -p -m 0755 /nix
chown -R xy:xy /nix
|
Running the install script
Download the Nix install script and examine the contents.
|
curl -L https://nixos.org/nix/install > nix_install.sh
|
Then, run it with --no-daemon
to prevent it running as system service.
|
sh ./nix_install.sh --no-daemon
|
performing a single-user installation of Nix...
copying Nix to /nix/store...
installing 'nix-2.20.1'
building '/nix/store/1ahlg3bviy174d6ig1gn393c23sqlki6-user-environment.drv'...
unpacking channels...
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...
Installation finished! To ensure that the necessary environment
variables are set, either log in again, or type
. /home/xy/.nix-profile/etc/profile.d/nix.fish
in your shell.
Wait!
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...
That’s very rude!
Stopping Nix from making a mess
I need to prevent Nix from mess up with my environment when I do not want it to. Nix puts some code into the Bash, ZSH and Fish initialization files during installation to ease it’s use. I do not want that since I do not want Nix to meddle with my environment without me knowing it.
I keep my .bash_profile
and .zshenv
in a stow-managed git repo so I can just cd
into my repo and do git reset --hard
, but for you will have to revert those files to their old forms manually.
Playing with Nix
We do not have nix
in PATH
but we still can launch it. Nix executables are located inside ~/.nix-profile/bin/
.
By invoking nix-shell
one can create a ephemeral environment containing only packages specified after the -p
flag. I always add -p nix
to have the Nix tools available also inside the spawned environment.
I will test out chibi
(small Scheme interpreter) + rlwrap
(REPL support for software lacking it) inside a Nix ephemeral environment:
|
~/.nix-profile/bin/nix-shell -p nix chibi rlwrap
|
Inside the spawned shell:
In the chibi REPL, let’s see the contents of the PATH
environment variable:
|
(get-environment-variable "PATH")
|
And exit the Scheme REPL:
After the playtime, run garbage collection:
|
~/.nix-profile/bin/nix-collect-garbage
|
Portage as a CD system
This is a very simple way to use any system with Portage installed as a Continuous Delivery server.
I think for a testing environment this is a valid solution to consider.
Create a repository of software used in your organization
Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:
Set up your repo with eselect-repository
Install the my-org
repository:
|
eselect repository add my-org git https://git.my-org.local/portage/my-org.git
|
Sync my-org
:
Install live packages of a your software
First, enable live packages (keywordless) for your my-org
repo:
|
echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf
|
Install some packages from my-org
:
|
emerge -av "=mycategory/mysoftware-9999"
|
Install smart-live-rebuild
smart-live-rebuild
can automatically update live software packages that use git as their source URL.
Set up cron to run smart-live-rebuild
Refresh your my-org
repository every hour:
|
0 */1 * * * emerge --sync my-org
|
Refresh the main Gentoo tree every other 6th hour:
|
0 */6 * * * emerge --sync gentoo
|
Run smart-live-rebuild
every other 3rd hour:
|
0 */3 * * * smart-live-rebuild
|
Restarting services after update
All-in-one script
You can either restart all services after successful update:
File: /opt/update.sh
|
#!/bin/sh
set -e
smart-live-rebuild
systemctl restart my-service-1.service
systemctl restart my-service-2.service
|
Crontab:
|
0 */3 * * * /opt/update.sh
|
Via ebuilds pkg_ functions
File: my-service-1.ebuild
|
pkg_postinst() {
systemctl restart my-service-1.service
}
|
More about pkg_postinst
:
Example Gentoo overlays
Firefox began as the first open source browser to live through the browser wars, overcoming Microsoft’s Internet Explorer and continues to deliver competition-smashing technology to this day.
Chromium code
The only advantages of Chromium are that it was adopted by Electron and spread partially because of a more liberal license and Google’s own efforts.
Google will never be able to cope with the worst imaginable code base of Chromium.
Chromium is near-impossible to compile
On a 4cores/8threads Ryzen CPU Chromium compiles in ~12h and requires at least 20GB of disk space for build. At the same time Firefox compiles in ~1.5h and requires ~8GB for disk space.
Programming language adoption
Additionally Firefox team was able to rewrite a very large portion of Firefox codebase in Rust which improved the browser’s safety. There were attempts to add rust to Chromium but they all are in more of a addon-like fashion.
Porting to UNIXes
Because Chromium is extremely large it’s very hard to port and maintain for Linux and BSD based systems. There were numerous bugs with Chromium’s UI on Linux that cause crashes on pressing random controls. I believe Google has no Linux testers beside the “free software freeloaders” (wink, wink, IBM :P).
Anti-competition
This days Google has to result to dirty tactics where certain Google-owned websites would either refuse to work on Firefox or give a fake performance hit that is entirely caused by malicious JavaScript code.
Several popular FOSS-related sources have covered this news recently, check them out on the WWW.
I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.
And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.
I guess some short term solutions really become defined tools :P
So this is how I rebuild my kernel nowadays:
-
Copy old config
|
cd /usr/src
cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/
|
-
Remove old kernel build directories
|
rm -r linux-6.1.31-gentoo
|
-
Run initial preparation
|
( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )
|
-
Call genkernel
1
2
3
4
5
6
7
8
9
10
11
12 |
genkernel \
--no-menuconfig \
--no-clean \
--no-clear-cachedir \
--no-cleanup \
--no-mrproper \
--lvm \
--luks \
--mdadm \
--nfs \
--kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')" \
all
|
-
Rebuild the modules
If in your /etc/genkernel.conf
you have MODULEREBUILD
turned off, then also call emerge:
|
emerge -1 @module-rebuild
|
System preparation
Qemu
Emerge qemu
with static-user
USE enabled and your wanted architectures.
|
app-emulation/qemu QEMU_SOFTMMU_TARGETS: aarch64 arm x86_64
app-emulation/qemu QEMU_USER_TARGETS: aarch64 arm x86_64
app-emulation/qemu static-user
dev-libs/glib static-libs
sys-apps/attr static-libs
sys-libs/zlib static-libs
dev-libs/libpcre2 static-libs
|
OpenRC
Enable qemu-binfmt
:
|
rc-update add qemu-binfmt default
|
Start qemu-binfmt
:
|
rc-service qemu-binfmt start
|
Chrooting
- select chroot location (eg
/chroots/gentoo-arm64-musl-stable
)
- unpack the desired rootfs
- create needed directories
mkdir -p /chroots/gentoo-arm64-musl-stable/var/cache/distfiles
- execute
bwrap
- with last
ro-bind
mount the qemu emulator binary (eg qemu-aarch64
)
- execute the mounted emulator binary giving it a shell program (eg
bash
)
Chroot with bwrap
:
|
bwrap \
--bind /chroots/gentoo-arm64-musl-stable / \
--dev /dev \
--proc /proc \
--perms 1777 --tmpfs /dev/shm \
--tmpfs /run \
--ro-bind /etc/resolv.conf /etc/resolv.conf \
--bind /var/cache/distfiles /var/cache/distfiles \
--ro-bind /usr/bin/qemu-aarch64 /usr/bin/qemu-aarch64 \
/usr/bin/qemu-aarch64 /bin/bash -l
|
User-mode
By default you would probably have something like this, the user-mode network:
|
<interface type="user">
<mac address="00:00:00:00:00:00"/>
<model type="virtio"/>
<address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>
|
Bridge
Bridges can be easily created using the NetworkManager’s TUI tool called nmtui
.
Bridge XML configuration for Libvirt
|
<interface type="bridge">
<mac address="00:00:00:00:00:00"/>
<source bridge="br1"/>
<target dev="vnet2"/>
<model type="virtio"/>
<alias name="net0"/>
<address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
</interface>
|
Sysctl options
Be sure the following options are enabled (1
):
net.ipv4.ip_forward
net.ipv4.conf.all.send_redirects
and the following options are disabled (0
):
net.bridge.bridge-nf-call-iptables
Init basic functions
Init should:
- be the first started process - PID 1
- continue running until the system is shut down
Why systemd is more than init
Full system management
systemd needs to run as PID 1 to parenthood other services and… other parts of itself (listed below).
sytemd absorbed
- cron -> timers
- seat tracker -> systemd-logind
- service manager -> systemd-systemctl
- udev -> systemd-udevd
- system logger -> systemd-journald
- network manager -> systemd-networkd
- bootloader -> systemd-boot
- hostname -> systemd-hostnamed
- chroot -> systemd-nspawn
- resolv.conf generator -> systemd-resolved
Additionally systemd is also
- QR code generator
- http server (cockpit)
- home manager -> systemd-homed
- tmp manager -> systemd-tmpfiles
Why it's bad
systemd growth to cover more and more of other services capabilities is dangerous. It can mean that if we find a exploit in one part of systemd it will be easy to compromise the whole system. Actually, there is a even bigger problem - most of the time a user would like to run a standalone service covering one of the uses already covered by systemd - like cronie or rsyslog.
But what about cgroups
Cgroups are nothing new, they are available in other service managers too. Check out cgroups in OpenRC.
Alternatives
OpenRC
Primarily used by Gentoo based systems. Was adopted to Devuan and Artix. Alpine Linux uses it probably because it started out as a Gentoo-based system. Is also available in Debian - though that will still use some systemd services, notably systemd-udevd.
Runit
Avalible in Void Linux - the recommended init for that system.
Pure System V init
Most popular use of this is found in old CentOS, Slackware, Antix and MX Linux.
S6
Check out Devuan if you want to dive into this topic.
systemd + Emacs
One could think that because systemd and Emacs cover a lot of things - systemd - system management and emacs - user utilities - a fusion of two would be very good and will lack so-called bloat. There is one "problem" however - Emacs can be used also as a init! And that doesn't necessarily mean a lot more code being added. So i guess Emacs + Linux (kernel) is the way to go ;P
Was systemd a inside job?
Probably not - Microsoft made their own init for Azure because… systemd was to big… systemd is a history of what happens when everybody wants to be like the rest - adoption of systemd.
Is systemd the new busybox?
This is an interesting concept. There is one bad thing however - you can compile bysybox with any of its utilities and it will work - you can't do that with systemd, you will have to modify and fork it - like eudev and elogind projects
OpenRC prejudice
Many people think of OpenRC as a old SysVinit. I encourage you to check it out and see that most stuff you probably do with systemd you can get from OpenRC (service manager) + cronie (cron/timers) + htop/pstree (full description of system as opposed to "systemctl status")
Basic OpenRC vs systemd commands
OpenRC systemd ————————— ————————— rc-status -a systemctl status rc-service SERVICE start systemctl start SERVICE rc-service SERVICE stop systemctl stop SERVICE rc-update add SERVICE systemctl enable SERVICE rc-update del SERVICE systemctl disable SERVICE
Sources
Honorable Mentions