The official ::gentoo repository currently contains only GHC on version 9.2.8. To install newer GHC one has to either download/build themselves or use the ::haskell overlay (https://github.com/gentoo-haskell/gentoo-haskell).

Enable the ::haskell overlay

Enable:

1

eselect repository enable haskell

Sync:

1
2
3

emerge --sync haskell
egencache --update --repo haskell --jobs 12 --load 6
eix-update

Unmask needed packages

Add to /etc/portage/package.unmask/0000_hs.conf

1
2
3
4
5
6
7
8

<dev-lang/ghc-9.9

<dev-haskell/cabal-3.11
<dev-haskell/cabal-install-3.11
<dev-haskell/cabal-install-solver-3.11
<dev-haskell/cabal-syntax-3.11
<dev-haskell/text-2.2
<dev-haskell/parsec-3.1.18

Add to /etc/portage/package.accept_keywords/0000_hs.conf

1
2
3

app-admin/haskell-updater
dev-haskell/*
dev-lang/ghc

Install

1

emerge --ask --verbose ">=dev-lang/ghc-9.8" ">=dev-haskell/cabal-install-3.10"

Build of GHC 9.8 takes around ~2 hours on a 8-core laptop-grade CPU.

Bonus: masking packages from ::haskell

If you want to exclude a given version from the ::haskell overly from being installed/updated, then you can add a similar line(s) to /etc/portage/package.mask/0000_hs.conf:

1

app-emacs/haskell-mode::haskell

I made a mistake when splitting my Portage make.conf file, having it as one file instead of a directly with many small files is a lot easier to maintain.

Portage allows users to split all of files inside /etc/portage such as make.conf, package.use, package.mask and other into groups of files contained in directories of the same name. This is very helpful when using automation to add some wanted configuration. But in case of make.conf it becomes a “form over function” issue.

I would also recommend to keep make.conf as simple as possible, without useless overrides and variable reassignment.

See also:

• /etc/portage or Gentoo Wiki

Bonus: config

And of course, this is my current /etc/portage/make.conf of my main dev machine:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

BINPKG_FORMAT="gpkg"
CCACHE_DIR="/var/cache/ccache"
EMERGE_WARNING_DELAY="0"
LC_MESSAGES="C"

PORTAGE_NICENESS="10"
PORTAGE_WORKDIR_MODE="0775"

PORTAGE_LOGDIR="${EPREFIX}/var/log/portage"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_ELOG_SYSTEM="save"

QUICKPKG_DEFAULT_OPTS="--include-config=y --umask=0003"
MAKEOPTS="--jobs=7 --load-average=6"

COMMON_FLAGS="
  -march=znver1 -O2 -falign-functions=32
  -fstack-clash-protection -fstack-protector-strong
  -fdiagnostics-color=always -frecord-gcc-switches -pipe"

ADAFLAGS="${COMMON_FLAGS}"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

CARGO_TERM_VERBOSE="false"
RUSTFLAGS="-C opt-level=3 -C debuginfo=0"

LDFLAGS="${LDFLAGS} -Wl,--defsym=__gentoo_check_ldflags__=0"

L10N="en de pl"
VIDEO_CARDS="amdgpu radeon radeonsi"

CPU_FLAGS_X86="
  aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt
  sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"

EMERGE_DEFAULT_OPTS="
  --binpkg-changed-deps=y --binpkg-respect-use=y
  --nospinner --keep-going=y
  --jobs=3 --load-average=8"

GENTOO_MIRRORS="
  https://mirror.leaseweb.com/gentoo/
  https://gentoo.osuosl.org/
  https://distfiles.gentoo.org/"

FEATURES="
  userpriv usersandbox usersync
  downgrade-backup unmerge-backup binpkg-multi-instance buildsyspkg
  parallel-fetch parallel-install
  ccache
  -binpkg-logs -ebuild-locks"

USE="
  custom-cflags custom-optimization firmware initramfs vaapi vulkan
  -bindist -zeroconf"

.NET ecosystem in Gentoo in year 2023

The Gentoo Dotnet project introduced better support for building .NET-based software using the nuget, dotnet-pkg-base and dotnet-pkg eclasses. This opened new opportunities of bringing new packages depending on .NET ecosystem to the official Gentoo ebuild repository and helping developers that use dotnet-sdk on Gentoo.

New software requiring .NET is constantly being added to the main Gentoo tree, among others that is:

• PowerShell for Linux,
• Denaro — finance application,
• Ryujinx — NS emulator,
• OpenRA — RTS engine for Command & Conquer, Red Alert and Dune2k,
• Pinta — graphics program,
• Pablodraw — Ansi, Ascii and RIPscrip art editor,
• Dafny — verification-aware programming language
• many packages aimed straight at developing .NET projects.

Dotnet project is also looking for new maintainers and users who are willing to help out here and there. Current state of .NET in Gentoo is very good but we can still do a lot better.

Special thanks to people who helped out

• Anna Figueiredo Gomes
• Esteve Varela Colominas
• Andrew Udvare

Links to bugs and announcements

• Bugs
  • gentoo-dotnet–2023/1
  • 900597
• Github PRs
  • 21112
  • 21451
  • 29309
  • 32109
  • 33940
• Active Gentoo .NET projects
  • gentoo-dotnet-guide
  • gentoo-dotnet-maintainer-tools
  • csharp-gentoodotnetinfo
  • eselect-dotnet
  • eselect-pwsh

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

• Ebuild repository
• Creating an ebuild repository

Set up your repo with eselect-repository

Install the my-org repository:

1

eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1

emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1

echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1

emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1

0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1

0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1

0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8

#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1

0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3

pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

• pkg_postinst
• Phase Functions

Example Gentoo overlays

• xgqt/myov
• gentoo-racket/gentoo-racket-overlay
• gentoo-mirror/guru

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

1. Copy old config

   1 2	cd /usr/src cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/

2. Remove old kernel build directories

   1	rm -r linux-6.1.31-gentoo

3. Run initial preparation

   1	( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )

4. Call genkernel

   1 2 3 4 5 6 7 8 9 10 11 12

   genkernel \ --no-menuconfig \ --no-clean \ --no-clear-cachedir \ --no-cleanup \ --no-mrproper \ --lvm \ --luks \ --mdadm \ --nfs \ --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')" \ all

5. Rebuild the modules

   If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

   1	emerge -1 @module-rebuild

Check load path

Some Elisp package compilation failures are caused by not setting the loadpath correctly. It mostly happens when you compile source from a directory that is not the current working directory. For example:

1

elisp-compile elisp/*.el

In most cases you can cd or override the S variable to set it to location where ELisp source resides.

But in other cases you can append to load path the directory with source, see:

1

BYTECOMPFLAGS="${BYTECOMPFLAGS} -L elisp" elisp-compile elisp/*.el

Do not rename auto-generated autoload file

elisp-make-autoload-file allows to name the generated autoload file. For sake of easier debugging and writing Gentoo SITEFILEs, please do not rename the generated file.

The name of that file should always be ${PN}-autoloads.el.

Use new elisp-enable-tests function

elisp-enable-tests allows to set up IUSE, RESTRICT, BDEPEND and the test runner function for running tests with the specified test runner.

The 1st (test-runner) argument must be one of:

• buttercup — for buttercup provided via app-emacs/buttercup,
• ert-runner — for ert-runner provided via app-emacs/ert-runner,
• ert — for ERT, the built-in GNU Emacs test utility.

The 2nd argument is the directory where test are located, the leftover arguments are passed to the selected test runner.

Example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

EAPI=8

inherit elisp

# Other package settings ...

SITEFILE="50${PN}-gentoo.el"
DOCS=( README.md )

elisp-enable-tests buttercup test

Remove empty SITEFILEs

Recently a feature was added to elisp.eclass that will cause build process to generate the required SITEFILE with boilerplate code if it does not exist.

So if your SITEFILE looked like this:

1

(add-to-list 'load-path "@SITELISP@")

… then, you can just remove that file.

But remember to keep the SITEFILE variable inside your ebuild:

1

SITEFILE="50${PN}-gentoo.el"

Remove pkg.el files

The *-pkg.el files are useless to Gentoo distribution model of Emacs Lisp packages and should be removed. It is as simple as adding this line to a ebuild:

1

ELISP_REMOVE="${PN}-pkg.el"

Beware that some packages will try to find their ${PN}-pkg.el file, but in most cases this will show up in failing package tests.

Use official repository

It is tedious to repackage Elpa tarballs, so use the official upstream even if you have to snapshot a specific commit.

To snapshot GitHub repos you would generally use this code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# First check if we have the correct version to prevent
# autobumping package version without changing the commit.
[[ ${PV} == *_p20220325 ]] && COMMIT=65c496d3d1d1298345beb9845840067bffb2ffd8

# Use correct URL that supports snapshots.
SRC_URI="https://github.com/domtronn/${PN}/archive/${COMMIT}.tar.gz
    -> ${P}.tar.gz"

# Override the temporary build directory variable.
S="${WORKDIR}"/${PN}-${COMMIT}

Include live version support

We do not want to be worse than the Melpa unstable :D

So, why not allow the given package to be used live?

Even if you do not push the live package to the overlay, please include support for it.

1
2
3
4
5
6
7
8

if [[ ${PV} == *9999* ]] ; then
    inherit git-r3
    EGIT_REPO_URI="https://github.com/example/${PN}.git"
else
    SRC_URI="https://github.com/example/${PN}/archive/${PV}.tar.gz
        -> ${P}.tar.gz"
    KEYWORDS="~amd64 ~x86"
fi

Ask for tags

Git is good, git tags are good. In case if upstream does not tag their package or just forgets to, kindly ask them to create a git tag when bumping Emacs package versions.

Building PowerShell

As a part of my work of modernizing the way .NET SDK packages are distributed in Gentoo I delved into packaging a from-source build of PowerShell for Gentoo using the dotnet-pkg eclass.

Packaging pwsh was a little tricky but I got a lot of help from reading the Alpine Linux’s APKBUILD. I had to generate special C# code bindings with ResGen and repackage the PowerShell tarball. Other than this trick, restoring and building PowerShell was pretty straight forward with the NuGet package management support from the dotnet-pkg.eclass.

Alternatively if you do not want to build PowerShell you can install the binary package, I have in plans to keep that package around even after we get the non-binary app-shells/pwsh into the official Gentoo ebuild repository.

Why install modules via Portage?

But why stop on PowerShell when we can also package multiple PS modules?

Installing modules via Portage has many benefits:

• better version control,
• more control over global install,
• no need to enable PS Gallery,
• sandboxed builds,
• using system .NET runtime.

Merging the modules

PowerShell’s method of finding modules is at follows: check paths from the PSModulePath environment variable for directories containing valid .psd1 files which define the PS modules.

By default pwsh tries to find modules in paths:

• user’s modules directory — ~/.local/share/powershell/Modules
• system modules directory in /usr/local — /usr/local/share/powershell/Modules
• Modules directory inside the pwsh home — for example /usr/share/pwsh-7.3/Modules

Because we do not want to touch either /usr/local nor pwsh home, we embed a special environment variable inside the pwsh launcher script to extend the path where pwsh looks for PS modules. The new module directory is located at /usr/share/GentooPowerShell/Modules.

1
2

dotnet-pkg-utils_append_launchervar \
    'PSModulePath="${PSModulePath}:/usr/share/GentooPowerShell/Modules:"'

So every PowerShell module will install it’s files inside /usr/share/GentooPowerShell/Modules.

To follow PS module location convention we add to that path a segment for the real module name and a segment for module version. This also enables us to have proper multi-slotting because most of the time the modules will not block installing other versions.

Take a look at this example from the app-pwsh/posh-dotnet–1.2.3 ebuild:

1
2
3
4
5
6

src_install() {
    insinto /usr/share/GentooPowerShell/Modules/${PN}/${PV}
    doins ${PN}.psd1 ${PN}.psm1

    einstalldocs
}

And that is it. Some packages do not even need to be compiled, they just need files placed into specific location. But when compilation of C# code is needed we have dotnet-pkg to help.

Binpkgs generated by user

The binary packages generated by user can have architecture-specific optimizations because they are generated after they were compiled by the host Portage installation.

In addition binpkgs are generated from ebuilds so if there is a USE flag incompatibility on the consumer system then the binpkg will not be installed on the host and Portage will fall back to from-source compilation.

Those binary packages can use two formats: XPAK and GPKG.

XPAK had many issues and is getting superseded by the GPKG format. Beware of upcoming GPKG transition and if you must use XPAKs then you should explicitly enable it in your system’s Portage configuration.

To host a binary package distribution server see the Binary package guide on the Gentoo wiki.

Bin packages in a repository

Binary packages in ::gentoo (the official Gentoo repository) have the -bin suffix.

Those packages might have USE flags but generally they are very limited in case of customizations or code optimizations because they were compiled either by a Gentoo developer or by a given package upstream maintainer (or their CI/CD system).

Those packages land in ::gentoo mostly because it is too hard (or even impossible) to compile them natively by Portage. Most of the time those packages use very complicated build systems or do not play nice with network sandbox like (e.g. Scala-based projects) or use very large frameworks/libraries like (e.g. Electron).

They can also be added to the repository because they are very desirable either by normal users (e.g. www-client/firefox-bin) or for (from-source) package bootstrapping purposes (e.g. dev-java/openjdk-bin). Such packages are sometimes generated from the regular source packages inside ::gentoo and later repackaged.

Patching

The file lit.site.cfg has to be inspected for any incorrect calls to executables. For example see src_prepare function form dev-lang/boogie.

Eclasses

Because we will need to specify how many threads should lit run we need to inherit multiprocessing to detect how many parallel jobs the portage config sets.

1

inherit multiprocessing

Dependencies

Ensure that dev-python/lit is in BDEPEND, but also additional packages may be needed, for example dev-python/OutputCheck.

1
2
3
4
5
6
7

BDEPEND="
    ${RDEPEND}
    test? (
        dev-python/lit
        dev-python/OutputCheck
    )
"

Bad tests

To deal with bad test you can simply remove the files causing the failures.

1
2
3
4
5
6
7
8
9

local -a bad_tests=(
    civl/inductive-sequentialization/BroadcastConsensus.bpl
    civl/inductive-sequentialization/PingPong.bpl
    livevars/bla1.bpl
)
local bad_test
for bad_test in ${bad_tests[@]} ; do
    rm "${S}"/Test/${bad_test} || die
done

Test phase

--threads $(makeopts_jobs) specifies how many parallel tests to run.

--verbose option will show output of failed tests.

Last lit argument specifies where lit should look for lit.site.cfg and tests.

1
2
3

src_test() {
    lit --threads $(makeopts_jobs) --verbose "${S}"/Test || die
}

Prototype

Recently while browsing the Alpine git repo I noticed they have a function called snapshot, see: https://git.alpinelinux.org/aports/tree/testing/dart/APKBUILD#n45 I am not 100% sure about how that works but a wild guess is that the developers can run that function to fetch the sources and maybe later upload them to the Alpine repo or some sort of (cloud?) storage.

In Portage there exists a pkg_config function used to run miscellaneous configuration for packages. The only major difference between src_snapshot and that would of course be that users would never run snapshot.

Sandbox

Probably only the network sandbox would have to be lifted out… to fetch the sources of course.

But also a few (at least one?) special directories and variables would be useful.

Portage

Configure the following for Portage.

1

dev-util/pkgcheck emacs

Emerge

Emerge the following packages:

• app-emacs/company-ebuild
• dev-util/pkgcheck

Company-Ebuild should pull in app-emacs/ebuild-mode, if that does not happen, then report a bug ;-D

Standard

Add the following to your user's Emacs initialization file. The initialization file is either ~/.emacs.d/init.el or ~/.config/emacs/init.el for newer versions of GNU Emacs.

1
2
3
4
5
6
7
8

(require 'ebuild-mode)
(require 'company-ebuild)
(require 'flycheck)
(require 'flycheck-pkgcheck)

(add-hook 'ebuild-mode-hook 'company-ebuild-setup)
(add-hook 'ebuild-mode-hook 'flycheck-mode)
(add-hook 'ebuild-mode-hook 'flycheck-pkgcheck-setup)

Use-Package

We can also configure our environment using a use-package macro that simplifies the setup a little bit.

To use the below configuration the app-emacs/use-package package will have to be installed.

1
2
3
4
5
6
7
8
9

(require 'use-package)

(use-package ebuild-mode
  :defer t
  :mode "\\.\\(ebuild\\|eclass\\)\\'"
  :hook
  ((ebuild-mode . company-ebuild-setup)
   (ebuild-mode . flycheck-mode)
   (ebuild-mode . flycheck-pkgcheck-setup)))

The :defer t and :mode "..." enable deferred loading which theoretically speeds up GNU Emacs initialization time at the cost of running the whole use-package block of ebuild-mode configuration when the :mode condition is met.

News

Repository

With this commit first GNU Emacs integration was merged into the pkgcheck repository.

History

• https://github.com/pkgcore/pkgcheck/issues/417
• https://github.com/pkgcore/pkgcheck/pull/420
• https://github.com/gentoo/gentoo/pull/26700

Thanks

Huge thanks to Sam James and Arthur Zamarin for support and interest in getting this feature done.

Installation

Unmasking

The Flycheck integration is unreleased as of now, this will (hopefully) change in the future, but for now You need live versions of snakeoil, pkgcore and pkgcheck.

File: /etc/portage/package.accept_keywords/pkgcore.conf

1
2
3

dev-python/snakeoil  **
sys-apps/pkgcore     **
dev-util/pkgcheck    **

Also You will need to unmask app-emacs/flycheck and its dependencies.

File: /etc/portage/package.accept_keywords/emacs.conf

1
2
3

app-emacs/epl
app-emacs/pkg-info
app-emacs/flycheck

Emerging

Install pkgcheck with the emacs USE flag enabled.

File: /etc/portage/package.use/pkgcore.conf

1

dev-util/pkgcheck    emacs

Afterwards run:

1
2

emerge -1av dev-python/snakeoil sys-apps/pkgcore dev-util/pkgcheck
emerge -av --noreplace dev-util/pkgcheck

Configuration

Following is what I would suggest to put into your Emacs config file:

1
2
3
4
5
6
7
8

(require 'ebuild-mode)
(require 'flycheck)
(require 'flycheck-pkgcheck)

(setq flycheck-pkgcheck-enable t)

(add-hook 'ebuild-mode-hook 'flycheck-mode)
(add-hook 'ebuild-mode-hook 'flycheck-pkgcheck-setup)

If You are using use-package:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

(use-package flycheck
  :ensure nil)

(use-package ebuild-mode
  :ensure nil
  :hook ((ebuild-mode . flycheck-mode)))

(use-package flycheck-pkgcheck
  :ensure nil
  :custom ((flycheck-pkgcheck-enable t))
  :hook ((ebuild-mode . flycheck-pkgcheck-setup)))

The lines with :ensure nil are there to prevent use-package from trying to download the particular package from Elpa (because we use system packages for this configuration).