Portage CI

2022-03-08 :: continuous integration, gentoo, portage, quality assurance, testing

By: Maciej Barć

Potential benefits

Running tests

test BEFORE (src_test) and AFTER (pkg_postinst) installation
test if and how services break if they are not reloaded
test buildsystem configuration
sandbox enforces strict and consistent build rules
benchmarking with different compilation flags and libraries versions/releases

Configuration matrix

We can test across Cartesian product of different configuration settings, like:

USE flags
MAKEOPTS
CFLAGS, CXXFLAGS, CPPFLAGS, LDFAGS, RUSTFLAGS, etc.
arches (cross-compilation or run in qemu)
static linking
supported releases & versions of libraries (eg. glibc & musl)

Also, we could create diffs of installed files across different merges.

Reproducibility

mini overlay with ::gentoo or any other (eg. company's own) as master
record VCS (eg. git) hash of the dependent overlays

Binaries

grab dependencies from binhosts
distribute built binaries (maybe upload to a company's own artifacts server)
make AppImages

Getting there

How do we run this?

Do we want to write a proper tool, which we probably do or do we just run Portage + shells scripts?

Do we want to run under root, user, in eprefix, maybe all in docker?

Configuration files

The .portci directory contains the configuration.

Bug 799626

Link: bugs.gentoo.org/799626

Instead of using Ansible, Python, Yaml or Scheme we might use something similar to this for simple configuration, or if gets merged to upstream Portage the better.

Worth mentioning is the idea from Michał Górny who proposes to configure portage with toml files, like the example given in the bug report.

[package.unmask]
~virtual/libcrypt-2

[package.use.mask]
sys-libs/libxcrypt -system -split-usr

[package.use.force]
sys-libs/glibc -crypt

Also, package.x + Toml == a match made in heaven, it looks very nice!

Make gallery-dl work like youtube-dl

2021-09-26 :: customization

By: Maciej Barć

Make gallery-dl work like youtube-dl

A configuration file to download everything into current directory, like yotube-dl does.

File: ~/.config/gallery-dl/config.json

{
  "directory": [],
  "base-directory": "./"
}

Portage

2021-09-26 :: gentoo, portage

By: Maciej Barć

Switching from a git repository hosting

For nearly 2 years I have been synchronizing settings between my Gentoo machines via a git repository that is installed system-wide via portage itself.

This actually gets a little tiresome and slow as the package that installs the configuration has to be re-installed each time we want to update the settings.

So I thought it could be cool if I can just push files around between my machines with one command. Now: we can use rsync or scp but /etc/portage is owned by root so we either have to change the permissions to a user we use to ssh or a group that user is in.

Portage users

First change portage permissions of /etc/portage, now all users in the portage group will be able to modify the configuration

1 2	chown -R root:portage /etc/portage chmod -R g+w /etc/portage

Synchronizing

We can do a rsync from different machine with:

1	rsync -r /etc/portage REMOTEHOST:/etc --exclude=".git" --exclude="make.profile" --copy-unsafe-links

Reminder that this can also be done as a cron job!

Portage snapshots with git

Probably the best thing I got out of genlica is that I engineered a script to automatically do a git commit of the changes to the portage configuration.

Emerge git

1	emerge --noreplace --verbose dev-vcs/git

Copy the file that does the snapshots

1 2	mkdir -p /etc/portage/postsync.d curl -o /etc/portage/postsync.d/99-degitmerge.sh "https://gitlab.com/xgqt/genlica/-/raw/master/portage/postsync.d/99-degitmerge.sh"

HTML Renderers

2021-05-04 :: browser, emacs

By: Maciej Barć

How it should be?

I imagine that a web browser should have been a "window" to the Internet world that provides easy and efficient way to graphically access resources exposed by multiple protocols: HTTP(S), (S)FTP, Gopher, Gemini, etc…

How it is?

Only few protocols are supported!

Recently FTP support got "deprecated" in Firefox. It can still be enabled in by setting network.ftp.enabled to true.

FTP support is announced to be completely gone in FF 90.

Chrome dropped FTP some time ago.

BUT!

Luckily we have Emacs!

GNU Emacs has packages to support all kinds of protocols.

eww which is a simple HTTP(S) browser like w3m (also renders images)
tramp allows you to access files by SSH
elpher allows you to access Gemini and Gopher sites (graphically)
ange-ftp allows you to connect to FTP servers
and finally net-utils which wraps around system utilities to provide interactive mode for many protocols, ie.: gopher, irc, ntp, pop3, www

Awesome Racket language features

2021-05-03 :: racket, programming language

By: Maciej Barć

Also see: Fast-Racket at Racket's GitHub Wiki

Creating binaries

You can create portable binaries with Racket's raco command! Use raco exe and raco distribute.

More -> https://docs.racket-lang.org/raco/exe.html

Sample games

Racket provides a executable plt-games, when ran (from console) it opens a menu of miscellaneous games, among them: jewel, minesweeper, aces, spider, checkers. & more (20 games total).

Plots

You can plot data in 2d & 3d forms.

2D

Sample code:

#lang racket/base
(require racket/gui/base racket/math plot)

(plot-new-window? #true)

(plot (function sin (- pi) pi #:label "y = sin(x)"))

3D

Sample code:

#lang racket/base
(require racket/gui/base racket/math plot)

(plot-new-window? #true)

(plot3d
 (surface3d (lambda (x y) (* (cos x) (sin y)))
            (- pi) pi (- pi) pi)
 #:title "An R × R → R function"
 #:x-label "x" #:y-label "y" #:z-label "cos(x) sin(y)")

Browser

There's a included library to render web pages, just "(require browser)".

Sample code:

#lang racket
(require browser)

(open-url "https://xgqt.gitlab.io/")

FFI

You can use Racket's Foreign Function Interface to interact with non-Racket libraries to make use of very fast libraries written in (mainly) FORTRAN & C.

For example sci uses FFI for CBLAS & LAPACK.

Parallelism

For greater speed up with parallel execution there are futures, places and distributed places (for distributed programming).

Sage Math on Gentoo

2021-03-23 :: gentoo, tutorial, mathematics

By: Maciej Barć

Intro

Sage may be available on your distro but on Gentoo such frivolities for students are not there yet, so I had to install it the manual way.

User

I went to the Sage website, to the "download-source" link. The source mirror I picked was France.

Ok, so let's follow Sage Math build instructions and get it going

wget www-ftp.lip6.fr/pub/math/sagemath/src/sage-9.2.tar.gz
tar xvf sage-9.2.tar.gz
cd sage-9.2
./configure
make

And now, let's wait…

Portage

After a long, long, long time waiting I remembered that there existed a Gentoo overlay for Sage. And in the meantime I thought I'd try that solution instead since some good Gentoo people already did most of the effort.

Just a few files to edit…

File: /etc/portage/repos.conf/sage-on-gentoo.conf

# -*- conf -*-


[sage-on-gentoo]

auto-sync = yes
location = /var/db/repos/sage-on-gentoo
priority = 999
sync-git-clone-extra-opts = --depth=999999999 --no-shallow-submodules --verbose
sync-git-pull-extra-opts = --verbose
sync-type = git
sync-umask = 022
sync-uri = https://github.com/cschwan/sage-on-gentoo.git
sync-user = root:portage

File: /etc/portage/package.accept~keywords~/zz-sage

# -*- conf -*-


sci-mathematics/sage                    **

*/*::sage-on-gentoo

dev-python/cvxopt

media-gfx/tachyon

sci-libs/bliss
sci-libs/dsdp
sci-libs/fflas-ffpack
sci-libs/fplll
sci-libs/libhomfly
sci-libs/linbox
sci-libs/m4rie

sci-mathematics/glpk

File: /etc/portage/package.use/zz-sage

# -*- conf -*-


sci-mathematics/sage                    -doc-html -doc-html-bin
sci-mathematics/sage                    -doc-pdf -doc-pdf-bin
sci-mathematics/sage                    -jmol
sci-mathematics/sage                    X bliss meataxe

dev-python/pplpy                        doc

sci-libs/cddlib                         tools
sci-libs/pynac                          -giac

sci-mathematics/eclib                   flint
sci-mathematics/flint                   ntl
sci-mathematics/glpk                    gmp
sci-mathematics/gmp-ecm                 -openmp
sci-mathematics/lcalc                   pari
sci-mathematics/maxima                  ecls
sci-mathematics/pari                    gmp doc

Now - let's build Sage with Portage!

>>> Emerging (1 of 100) sci-mathematics/cliquer–1.21::gentoo

At least I know more or less how long I'm going to wait and know what exactly fails to build, if anything does.

Final

Build

$ qlop -tv sage

> 2021–03–23T20:49:12 >>> sci-mathematics/sage–9999: 55′04″

Git stats

Overlays

This info we can easily gather with executing:

1	emerge --info

Gentoo' HEAD: 33f2d770c28307b1e9a1199c681e1c543602c6d4

Sage-on-Gentoo's HEAD: f7eac5b7e1a844132164b7593dab85cd87918664

Sage

Sage repository's HEAD (because we are using the live (9999) ebuild):

1	cat /var/cache/distfiles/git3-src/sagemath_sage.git/refs/heads/develop

Which returns: 5cb72aade9b297c10bb0f1ae8529466e5b5eb41d

MTG alter-art collection

2021-01-15 :: art, card games, games, mtg

By: Maciej Barć

Story

In 2013–2017 I was very interested in Trading card game “Magic: the Gathering”. I started playing during the release of Theros and quit somewhere during Shadows over Innistrad. I still keep a small collection of alternative artwork that I have painted on the cards.