Binpkgs generated by user

The binary packages generated by user can have architecture-specific optimizations because they are generated after they were compiled by the host Portage installation.

In addition binpkgs are generated from ebuilds so if there is a USE flag incompatibility on the consumer system then the binpkg will not be installed on the host and Portage will fall back to from-source compilation.

Those binary packages can use two formats: XPAK and GPKG.

XPAK had many issues and is getting superseded by the GPKG format. Beware of upcoming GPKG transition and if you must use XPAKs then you should explicitly enable it in your system’s Portage configuration.

To host a binary package distribution server see the Binary package guide on the Gentoo wiki.

Bin packages in a repository

Binary packages in ::gentoo (the official Gentoo repository) have the -bin suffix.

Those packages might have USE flags but generally they are very limited in case of customizations or code optimizations because they were compiled either by a Gentoo developer or by a given package upstream maintainer (or their CI/CD system).

Those packages land in ::gentoo mostly because it is too hard (or even impossible) to compile them natively by Portage. Most of the time those packages use very complicated build systems or do not play nice with network sandbox like (e.g. Scala-based projects) or use very large frameworks/libraries like (e.g. Electron).

They can also be added to the repository because they are very desirable either by normal users (e.g. www-client/firefox-bin) or for (from-source) package bootstrapping purposes (e.g. dev-java/openjdk-bin). Such packages are sometimes generated from the regular source packages inside ::gentoo and later repackaged.

Template

File: /etc/systemd/system/APP.service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

[Unit]
Description=Run APP application

[Service]
Type=simple
ExecStart=/usr/bin/LANG APP_DIR/APP APP_ARGS
Restart=on-failure
User=root
WorkingDirectory=APP_DIR

[Install]
WantedBy=multi-user.target

Also, the application might need to reference a PID file, let systemD know abut it via PIDFile.

1

PIDFile=/tmp/APP.pid

Example

File: /etc/systemd/system/julia_dash_app.service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

[Unit]
Description=Run Julia Dash application

[Service]
Type=simple
ExecStart=/usr/bin/julia /root/julia_dash_app/main.jl
Restart=on-failure
User=root
WorkingDirectory=/root/julia_dash_app

[Install]
WantedBy=multi-user.target

Potential benefits

Running tests

• test BEFORE (src_test) and AFTER (pkg_postinst) installation
• test if and how services break if they are not reloaded
• test buildsystem configuration
• sandbox enforces strict and consistent build rules
• benchmarking with different compilation flags and libraries versions/releases

Configuration matrix

We can test across Cartesian product of different configuration settings, like:

• USE flags
• MAKEOPTS
• CFLAGS, CXXFLAGS, CPPFLAGS, LDFAGS, RUSTFLAGS, etc.
• arches (cross-compilation or run in qemu)
• static linking
• supported releases & versions of libraries (eg. glibc & musl)

Also, we could create diffs of installed files across different merges.

Reproducibility

• mini overlay with ::gentoo or any other (eg. company's own) as master
• record VCS (eg. git) hash of the dependent overlays

Binaries

• grab dependencies from binhosts
• distribute built binaries (maybe upload to a company's own artifacts server)
• make AppImages

Getting there

How do we run this?

Do we want to write a proper tool, which we probably do or do we just run Portage + shells scripts?

Do we want to run under root, user, in eprefix, maybe all in docker?

Configuration files

The .portci directory contains the configuration.

Bug 799626

Link: bugs.gentoo.org/799626

Instead of using Ansible, Python, Yaml or Scheme we might use something similar to this for simple configuration, or if gets merged to upstream Portage the better.

Worth mentioning is the idea from Michał Górny who proposes to configure portage with toml files, like the example given in the bug report.

1
2
3
4
5
6
7
8

[package.unmask]
~virtual/libcrypt-2

[package.use.mask]
sys-libs/libxcrypt -system -split-usr

[package.use.force]
sys-libs/glibc -crypt

Also, package.x + Toml == a match made in heaven, it looks very nice!

Switching from a git repository hosting

For nearly 2 years I have been synchronizing settings between my Gentoo machines via a git repository that is installed system-wide via portage itself.

This actually gets a little tiresome and slow as the package that installs the configuration has to be re-installed each time we want to update the settings.

So I thought it could be cool if I can just push files around between my machines with one command. Now: we can use rsync or scp but /etc/portage is owned by root so we either have to change the permissions to a user we use to ssh or a group that user is in.

Portage users

First change portage permissions of /etc/portage, now all users in the portage group will be able to modify the configuration

1
2

chown -R root:portage /etc/portage
chmod -R g+w /etc/portage

Synchronizing

We can do a rsync from different machine with:

1

rsync -r /etc/portage REMOTEHOST:/etc --exclude=".git" --exclude="make.profile" --copy-unsafe-links

Reminder that this can also be done as a cron job!

Portage snapshots with git

Probably the best thing I got out of genlica is that I engineered a script to automatically do a git commit of the changes to the portage configuration.

Emerge git

1

emerge --noreplace --verbose dev-vcs/git

Copy the file that does the snapshots

1
2

mkdir -p /etc/portage/postsync.d
curl -o /etc/portage/postsync.d/99-degitmerge.sh "https://gitlab.com/xgqt/genlica/-/raw/master/portage/postsync.d/99-degitmerge.sh"

Intro

Backing up using this method takes a lot less space - ~60MB (without distfiles) and can be restored on almost any system (running portage) and tweaked afterwards for, say, CPU architecture. I've created a a short script with similar method in here.

What we need

• ebuild repositories are installed with git
• distfiles (those might be gone when we want to replicate)

Backup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# System info
emerge --info > info.txt

# Portage tree
cp -Lr /etc/portage .

# Portage layout
tree -a -L 2 /etc/portage > layout.txt

# Packages in @world
cp /var/lib/portage/world .

# Installed sets
cp /var/lib/portage/world_sets .

# Installed packages (with versions)
qlist --installed --nocolor --umap > qlist-use.txt
qlist --installed --nocolor --verbose > qlist-ver.txt

# Distfiles
cp -rv "$(portageq envvar DISTDIR)" distfiles

# Ebuild database
cp -r /var/db/pkg pkgdb

Restoration

To faithfully restore the system perform those actions as root

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Copy the portage tree to /etc
rm -dr /etc/portage
cp -r portage /etc/portage

# Checkout the gentoo repo to a commit specified in info.txt
cd "$(portageq get_repo_path / gentoo)"
git checkout # <commit ID>

# Copy distfiles
cp -r distfiles/* "$(portageq envvar DISTDIR)"/

# Fake-install @world and sets
cp world /var/lib/portage/world
cp world_sets /var/lib/portage/world_sets

# Emerge the exact packages from qlist-ver.txt
emerge --keep-going=y -1Oav $(sed 's/^/=/' qlist-ver.txt)

Introduction

On some old routers, namely TP-Link's TL-WR840N version 2, there may not be a option to switch to access point mode. This is what you have to do to access that mode indirectly.

Steps

• Power off the router
• Unplug RJ cable from the WAN port
• Connect the router to a PC (using one of the LAN ports)
• Power on the router
• Log it to the web console entering your router's IP, you can find out the IP by executing ip --color a (on a Linux box) if the router's DHCP server is still active
• In LAN settings give your router a static client IP that fits into a network you want to connect it to; example: 192.168.100.100/24 (make sure no other hosts are associated with that IP in the target network)
• Restart the router
• Log in to the web console entering the IP you have set
• Turn off the DHCP server (DHCP -> DHCP Settings -> DHCP Server: Disable & Save)
• Power off the router
• Plug the cable from a network you want the router connected to to one of the LAN ports
• Power on the router
• Log in to the web console entering the IP you have set
• Turn on the wireless network and set it up (SSID, password, etc.)
• Done! :D

Sources

• https://www.tp-link.com/in/support/download/tl-wr840n/v2/
• https://www.youtube.com/watch?v=Cg_gGECGLiY

Init basic functions

Init should:

• be the first started process - PID 1
• continue running until the system is shut down

Why systemd is more than init

Full system management

systemd needs to run as PID 1 to parenthood other services and… other parts of itself (listed below).

systemd absorbed

• cron -> timers
• seat tracker -> systemd-logind
• service manager -> systemd-systemctl
• udev -> systemd-udevd
• system logger -> systemd-journald
• network manager -> systemd-networkd
• bootloader -> systemd-boot
• hostname -> systemd-hostnamed
• chroot -> systemd-nspawn
• resolv.conf generator -> systemd-resolved

Additionally systemd is also

• QR code generator
• http server (cockpit)
• home manager -> systemd-homed
• tmp manager -> systemd-tmpfiles

Why it's bad

systemd growth to cover more and more of other services capabilities is dangerous. It can mean that if we find a exploit in one part of systemd it will be easy to compromise the whole system. Actually, there is a even bigger problem - most of the time a user would like to run a standalone service covering one of the uses already covered by systemd - like cronie or rsyslog.

But what about cgroups

Cgroups are nothing new, they are available in other service managers too. Check out cgroups in OpenRC.

Alternatives

OpenRC

Primarily used by Gentoo based systems. Was adopted to Devuan and Artix. Alpine Linux uses it probably because it started out as a Gentoo-based system. Is also available in Debian - though that will still use some systemd services, notably systemd-udevd.

Runit

Available in Void Linux - the recommended init for that system.

Pure System V init

Most popular use of this is found in old CentOS, Slackware, Antix and MX Linux.

S6

Check out Devuan if you want to dive into this topic.

systemd + Emacs

One could think that because systemd and Emacs cover a lot of things - systemd - system management and emacs - user utilities - a fusion of two would be very good and will lack so-called bloat. There is one "problem" however - Emacs can be used also as a init! And that doesn't necessarily mean a lot more code being added. So i guess Emacs + Linux (kernel) is the way to go ;P

Was systemd a inside job?

Probably not - Microsoft made their own init for Azure because… systemd was to big… systemd is a history of what happens when everybody wants to be like the rest - adoption of systemd.

Is systemd the new busybox?

This is an interesting concept. There is one bad thing however - you can compile bysybox with any of its utilities and it will work - you can't do that with systemd, you will have to modify and fork it - like eudev and elogind projects

OpenRC prejudice

Many people think of OpenRC as a old SysVinit. I encourage you to check it out and see that most stuff you probably do with systemd you can get from OpenRC (service manager) + cronie (cron/timers) + htop/pstree (full description of system as opposed to "systemctl status")

Basic OpenRC vs systemd commands

OpenRC systemd ————————— ————————— rc-status -a systemctl status rc-service SERVICE start systemctl start SERVICE rc-service SERVICE stop systemctl stop SERVICE rc-update add SERVICE systemctl enable SERVICE rc-update del SERVICE systemctl disable SERVICE

Sources

• Ewontfix
• Wikipedia
• SystemE - Emacs as init

Honorable Mentions

• https://forums.gentoo.org/viewtopic-t-1105854.html
• https://unixsheikh.com/articles/the-real-motivation-behind-systemd.html