Posts tagged sysadmin


winget sucks. Why?

:: microsoft, packaging, sysadmin, windows, winget

By: Maciej Barć

I would have stopped myself from writing about winget but far too many times when I do a full package set upgrade some package does not install correctly.

Running too many install hooks

Sometimes package installation may fail because “same version of the package is installed”… Wait! What? How is the version I am updating already installed? Well I got this bizarre error (that probably indicates a more serious bug) when trying to update Docker Desktop via winget.

What often happens is that because winget runs the package install / update executable, that is a binary, we do not know exactly what it could do, want or expect. So, what you might get is that the software actually expects you to invoke the updater in a different way.

Not enough installer checks

This is a even more annoying bug that, after executing the installer the package in question package does not register itself. I have hit this bug with a package very crucial to me - GNU Emacs.

It is true that this should be fixed by the upstream project because the installer is just broken, but most of those bugs are known. So, is there no mechanism that we can use to register GNU Emacs ourselves? I just do not get why is there no check to see if the package is actually installed or uninstalled. Honestly quite insane!

Proper CI/CD

I think the winget issues can be mitigated by implementing a proper CI/CD for packages. Below I propose a system that will test packages in two clean runs:

  1. Installation checks.

    This step would consist of 3 actions to determine correct installation and removal of the package:

    • install the package and check if it installed correctly
    • install the package again, if the package registered correctly during the previous step, then no installation phase should run and this step would catch the mentioned beforehand GNU Emacs bug
    • uninstall the package to check if it is removed correctly
  2. Update test.

    Very simple - two steps: install older version of a package and hen attempt to update it. I guarantee lot of packages will not pass this with out-of-the-box configuration.

Portage Continuous Delivery

:: gentoo, linux, sysadmin, system

By: Maciej Barć

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

Set up your repo with eselect-repository

Install the my-org repository:

1
eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1
emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1
echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1
emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1
0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1
0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1
0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8
#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1
0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3
pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

Example Gentoo overlays

Genkernel in 2023

:: gentoo, kernel, linux, sysadmin, system, tutorial

By: Maciej Barć

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

  1. Copy old config

    1
    2
    cd /usr/src
    cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/
    
  2. Remove old kernel build directories

    1
    rm -r linux-6.1.31-gentoo
    
  3. Run initial preparation

    1
    ( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )
    
  4. Call genkernel

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    genkernel                                                       \
        --no-menuconfig                                             \
        --no-clean                                                  \
        --no-clear-cachedir                                         \
        --no-cleanup                                                \
        --no-mrproper                                               \
        --lvm                                                       \
        --luks                                                      \
        --mdadm                                                     \
        --nfs                                                       \
        --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')"    \
        all
    
  5. Rebuild the modules

    If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

    1
    emerge -1 @module-rebuild
    

Portage system replication

:: gentoo, portage, sysadmin, system

By: Maciej Barć

Intro

Backing up using this method takes a lot less space - ~60MB (without distfiles) and can be restored on almost any system (running portage) and tweaked afterwards for, say, CPU architecture. I've created a a short script with similar method in here.

What we need

  • ebuild repositories are installed with git
  • distfiles (those might be gone when we want to replicate)

Backup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# System info
emerge --info > info.txt

# Portage tree
cp -Lr /etc/portage .

# Portage layout
tree -a -L 2 /etc/portage > layout.txt

# Packages in @world
cp /var/lib/portage/world .

# Installed sets
cp /var/lib/portage/world_sets .

# Installed packages (with versions)
qlist --installed --nocolor --umap > qlist-use.txt
qlist --installed --nocolor --verbose > qlist-ver.txt

# Distfiles
cp -rv "$(portageq envvar DISTDIR)" distfiles

# Ebuild database
cp -r /var/db/pkg pkgdb

Restoration

To faithfully restore the system perform those actions as root

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Copy the portage tree to /etc
rm -dr /etc/portage
cp -r portage /etc/portage

# Checkout the gentoo repo to a commit specified in info.txt
cd "$(portageq get_repo_path / gentoo)"
git checkout # <commit ID>

# Copy distfiles
cp -r distfiles/* "$(portageq envvar DISTDIR)"/

# Fake-install @world and sets
cp world /var/lib/portage/world
cp world_sets /var/lib/portage/world_sets

# Emerge the exact packages from qlist-ver.txt
emerge --keep-going=y -1Oav $(sed 's/^/=/' qlist-ver.txt)