How it works

• There will be a FUSE OneDrive filesystem mounted on the machine Nextcloud runs on,
• You configure filesystem-local “External storage” and point to the mountpoint of a cloud drive (in this case OneDrive),
• users connected via Nextcloud Client will have a option to sync any chosen files from the External storage as if they were Nextcloud-owned files.

Benefits

There are some benefits from connecting OneDrive to Nextcloud:

• faster cached sync than by normal straight-to OneDrive connection - files will be pushed to Nextcloud and then uploaded to Nextcloud, this will take less time than uploading straight to OneDrive because of Microsoft OneDrive rate-limiting the uploads (especially in case of larger files),
• clients of Nextcloud do not have to configure any OneDrive connection,
• you will be able to have two-way sync of your OneDrive files (currently two-way sync on the RClone side is experimental, this will use Nextcloud’s two-way sync mechanism).

Set up RClone

First You will have to set up RClone. Connect to your cloud of choice and then copy the config to a location that will be readable by a service that mounts a given cloud drive.

For OneDrive I use /usr/local/share/rclone/config/rclone.conf that is accessible only to the apache user.

The config will look something like this:

1
2
3
4
5
6

[OneDrive]
type = onedrive
region = global
token = *REDACTED*
drive_id = *REDACTED*
drive_type = personal

Mounting OneDrive

I created this helper script (in /usr/local/bin/rclone-mount.sh):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/bin/sh

set -e
set -u
trap "exit 130" INT

conf_path="${1}"

local_user="${2}"
local_path="${3}"

cloud_name="${4}"
cloud_path="${5}"

log_dir="/var/log/rclone"
log_file="${log_dir}/rclone-mount-${local_user}-${cloud_name}.log"

mkdir -p "${log_dir}"
touch "${log_file}"

chmod a+r "${log_file}"
chown "${local_user}" "${log_file}"

exec rclone                                         \
     --default-permissions                          \
     --allow-other                                  \
     --verbose                                      \
     --vfs-cache-mode full                          \
     --config "${conf_path}"                        \
     mount                                          \
     "${cloud_name}:${cloud_path}" "${local_path}"  \
     >> "${log_file}" 2>&1

Then, I use it in a OpenRC service like this (/etc/init.d/mount-OneDrive):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/sbin/openrc-run

conf_path="/usr/local/share/rclone/config/rclone.conf"

cloud_name="OneDrive"
cloud_path="/"

local_user="apache"
local_path="/mnt/${cloud_name}"

command="/usr/local/bin/rclone-mount.sh"
command_args="${conf_path} ${local_user} ${local_path} ${cloud_name} ${cloud_path}"

command_background="false"
command_user="${local_user}:$(id -g -n ${local_user})"
supervisor="supervise-daemon"

depend() {
    need net
}

start_pre() {
    ebegin "Unmounting leftovers from ${local_path} before service start"
    umount "${local_path}"
    eend 0
}

stop_post() {
    ebegin "Unmounting leftovers from ${local_path} after service stop"
    umount "${local_path}"
    eend 0
}

Enabling the RClone service

Set up directories and correct permissions:

1
2
3
4
5

mkdir -p /usr/local/share/rclone/config
chown -R apache:apache /usr/local/share/rclone/config

mkdir -p /var/log/rclone
chown -R apache:apache /var/log/rclone

Do not forget to make the mount service script executable:

1

chmod +x /etc/init.d/mount-OneDrive

Enable and start this service on OpenRC:

1
2

rc-update add mount-OneDrive default
rc-service mount-OneDrive start

Drive permissions

RClone mounts cloud drives by using FUSE. To have the RClone option --allow-other available in order to allow root to access the drive you will have to modify the FUSE config file (/etc/fuse.conf) - add user_allow_other.

Nextcloud configuration

Download and enable the “External storage” app. Then, in “Administration” settings add a external storage:

• name: ExternalStorage_OneDrive
• type: Local
• authentication: None
• configuration: /mnt/OneDrive
• available for: YOUR USER

SysStat is a amazing tool. In the age where telegraf and grafana are all the rage everybody forgot about the good old sysstat.

Selected command examples

• iostat -d -p nvme0n1 3 - disk I/O for a NVME drive (nvme0n1),
• sar -n DEV 3 - network throughput,
• sar -h -r 3 - memory usage,
• sar -P ALL 3 - CPU utilization
• sar -q 3 - system load levels,
• sar -A 3 - all the metrics.

Gathered info

qlist app-admin/sysstat | grep /usr/bin/

The app-admin/sysstat contains the following binaries and their respective statistic fields:

• sar - general utilization statistics,
• cifsiostat - CIFS,
• iostat - device input/output,
• mpstat - processors,
• pidstat - Linux tasks,
• tapestat - tape (yes, the real tape disks).

Installation

Gentoo

1
2
3

emerge --noreplace --verbose sys-process/cronie app-admin/sysstat
rc-service cronie start
rc-update add cronie default

Files

By default (on Gentoo): * sa (the collector) saves statistics to /var/log/sa, * /etc/sysstat is the configuration file * cron jobs are run via the *system* cronjob table.

I made a mistake when splitting my Portage make.conf file, having it as one file instead of a directly with many small files is a lot easier to maintain.

Portage allows users to split all of files inside /etc/portage such as make.conf, package.use, package.mask and other into groups of files contained in directories of the same name. This is very helpful when using automation to add some wanted configuration. But in case of make.conf it becomes a “form over function” issue.

I would also recommend to keep make.conf as simple as possible, without useless overrides and variable reassignment.

See also:

• /etc/portage or Gentoo Wiki

Bonus: config

And of course, this is my current /etc/portage/make.conf of my main dev machine:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

BINPKG_FORMAT="gpkg"
CCACHE_DIR="/var/cache/ccache"
EMERGE_WARNING_DELAY="0"
LC_MESSAGES="C"

PORTAGE_NICENESS="10"
PORTAGE_WORKDIR_MODE="0775"

PORTAGE_LOGDIR="${EPREFIX}/var/log/portage"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_ELOG_SYSTEM="save"

QUICKPKG_DEFAULT_OPTS="--include-config=y --umask=0003"
MAKEOPTS="--jobs=7 --load-average=6"

COMMON_FLAGS="
  -march=znver1 -O2 -falign-functions=32
  -fstack-clash-protection -fstack-protector-strong
  -fdiagnostics-color=always -frecord-gcc-switches -pipe"

ADAFLAGS="${COMMON_FLAGS}"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"

CARGO_TERM_VERBOSE="false"
RUSTFLAGS="-C opt-level=3 -C debuginfo=0"

LDFLAGS="${LDFLAGS} -Wl,--defsym=__gentoo_check_ldflags__=0"

L10N="en de pl"
VIDEO_CARDS="amdgpu radeon radeonsi"

CPU_FLAGS_X86="
  aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt
  sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"

EMERGE_DEFAULT_OPTS="
  --binpkg-changed-deps=y --binpkg-respect-use=y
  --nospinner --keep-going=y
  --jobs=3 --load-average=8"

GENTOO_MIRRORS="
  https://mirror.leaseweb.com/gentoo/
  https://gentoo.osuosl.org/
  https://distfiles.gentoo.org/"

FEATURES="
  userpriv usersandbox usersync
  downgrade-backup unmerge-backup binpkg-multi-instance buildsyspkg
  parallel-fetch parallel-install
  ccache
  -binpkg-logs -ebuild-locks"

USE="
  custom-cflags custom-optimization firmware initramfs vaapi vulkan
  -bindist -zeroconf"

Add Flathub-Beta

Add the Flathub-Beta remote repository:

1

flatpak remote-add --user flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo

Install GIMP beta

Install org.gimp.GIMP form flathub-beta:

1

flatpak install --assumeyes --user flathub-beta org.gimp.GIMP

Run GIMP

If you have other GIMP versions installed you will have to specify the “beta” version with //beta.

1

flatpak run org.gimp.GIMP//beta

Otherwise you can just run:

1

flatpak run org.gimp.GIMP

Also, in the desktop menus (like KRunner) this version of GIMP will have (beta) in its name so there is no chance to miss it.

Nix is useful for quickly testing out software and providing a strict environment that can be shared between people.

Today I’m trying out Nix again, this time I want to do it my way.

Installation process

Nix store

I know Nix needs “Nix store” installation on / (the system root).

Create it manually to prevent the installation script from calling sudo. 1st I switch to the root account, and then I run:

1
2

mkdir -p -m 0755 /nix
chown -R xy:xy /nix

Running the install script

Download the Nix install script and examine the contents.

1

curl -L https://nixos.org/nix/install > nix_install.sh

Then, run it with --no-daemon to prevent it running as system service.

1

sh ./nix_install.sh --no-daemon

performing a single-user installation of Nix...
copying Nix to /nix/store...
installing 'nix-2.20.1'
building '/nix/store/1ahlg3bviy174d6ig1gn393c23sqlki6-user-environment.drv'...
unpacking channels...
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

. /home/xy/.nix-profile/etc/profile.d/nix.fish

in your shell.

Wait!

modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

That’s very rude!

Stopping Nix from making a mess

I need to prevent Nix from mess up with my environment when I do not want it to. Nix puts some code into the Bash, ZSH and Fish initialization files during installation to ease it’s use. I do not want that since I do not want Nix to meddle with my environment without me knowing it.

I keep my .bash_profile and .zshenv in a stow-managed git repo so I can just cd into my repo and do git reset --hard, but for you will have to revert those files to their old forms manually.

Playing with Nix

We do not have nix in PATH but we still can launch it. Nix executables are located inside ~/.nix-profile/bin/.

By invoking nix-shell one can create a ephemeral environment containing only packages specified after the -p flag. I always add -p nix to have the Nix tools available also inside the spawned environment.

I will test out chibi (small Scheme interpreter) + rlwrap (REPL support for software lacking it) inside a Nix ephemeral environment:

1

~/.nix-profile/bin/nix-shell -p nix chibi rlwrap

Inside the spawned shell:

1

rlwrap chibi-scheme

In the chibi REPL, let’s see the contents of the PATH environment variable:

1

(get-environment-variable "PATH")

And exit the Scheme REPL:

1

(exit)

After the playtime, run garbage collection:

1

~/.nix-profile/bin/nix-collect-garbage

.NET ecosystem in Gentoo in year 2023

The Gentoo Dotnet project introduced better support for building .NET-based software using the nuget, dotnet-pkg-base and dotnet-pkg eclasses. This opened new opportunities of bringing new packages depending on .NET ecosystem to the official Gentoo ebuild repository and helping developers that use dotnet-sdk on Gentoo.

New software requiring .NET is constantly being added to the main Gentoo tree, among others that is:

• PowerShell for Linux,
• Denaro — finance application,
• Ryujinx — NS emulator,
• OpenRA — RTS engine for Command & Conquer, Red Alert and Dune2k,
• Pinta — graphics program,
• Pablodraw — Ansi, Ascii and RIPscrip art editor,
• Dafny — verification-aware programming language
• many packages aimed straight at developing .NET projects.

Dotnet project is also looking for new maintainers and users who are willing to help out here and there. Current state of .NET in Gentoo is very good but we can still do a lot better.

Special thanks to people who helped out

• Anna Figueiredo Gomes
• Esteve Varela Colominas
• Andrew Udvare

Links to bugs and announcements

• Bugs
  • gentoo-dotnet–2023/1
  • 900597
• Github PRs
  • 21112
  • 21451
  • 29309
  • 32109
  • 33940
• Active Gentoo .NET projects
  • gentoo-dotnet-guide
  • gentoo-dotnet-maintainer-tools
  • csharp-gentoodotnetinfo
  • eselect-dotnet
  • eselect-pwsh

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

• Ebuild repository
• Creating an ebuild repository

Set up your repo with eselect-repository

Install the my-org repository:

1

eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1

emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1

echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1

emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1

0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1

0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1

0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8

#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1

0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3

pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

• pkg_postinst
• Phase Functions

Example Gentoo overlays

• xgqt/myov
• gentoo-racket/gentoo-racket-overlay
• gentoo-mirror/guru

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

1. Copy old config

   1 2	cd /usr/src cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/

2. Remove old kernel build directories

   1	rm -r linux-6.1.31-gentoo

3. Run initial preparation

   1	( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )

4. Call genkernel

   1 2 3 4 5 6 7 8 9 10 11 12

   genkernel \ --no-menuconfig \ --no-clean \ --no-clear-cachedir \ --no-cleanup \ --no-mrproper \ --lvm \ --luks \ --mdadm \ --nfs \ --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')" \ all

5. Rebuild the modules

   If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

   1	emerge -1 @module-rebuild

Config

First let’s prepare a suitable nginx configuration file.

This one is pretty bare but it works well for our case:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

worker_processes 1;
daemon off;
pid ./nginx/temp/nginx.pid;

error_log /dev/stdout info;

events {
    worker_connections 1024;
}

http {
    client_body_temp_path ./nginx/temp/client 1 2;
    proxy_temp_path ./nginx/temp/proxy;
    fastcgi_temp_path ./nginx/temp/fastcgi;
    uwsgi_temp_path ./nginx/temp/uwsgi;
    scgi_temp_path ./nginx/temp/scgi;

    server {
        listen 127.0.0.1:8080;
        server_name localhost;

        access_log /dev/stdout;
        error_log /dev/stdout info;

        root ./;

        location / {
            autoindex on;
        }
    }
}

Server config is set up for serving all static files from the current directory.

Startup

Preparation

Based on how you want to store _temp_path files it might be necessary to create (or clean up) additional directories, for example:

1
2

rm -r ./nginx/temp
mkdir -p ./nginx/temp

Run in current directory

1

nginx -c ./nginx.conf -p ./

BTW, you may want to replace ./ with "$(pwd)" and occurrences in the config with static paths.

Bonus: other simple servers

Some of no-dependency-except-itself http servers it’s good to know about:

Python http.server

1

python3 -m http.server -b 127.0.0.1 8080

Busybox

1

busybox httpd -f -p 127.0.0.1:8080 -v

You can read more about configuring busybox’s httpd on OpenWRT docs.

System preparation

Qemu

Emerge qemu with static-user USE enabled and your wanted architectures.

1
2
3
4
5
6
7
8

app-emulation/qemu      QEMU_SOFTMMU_TARGETS: aarch64 arm x86_64
app-emulation/qemu      QEMU_USER_TARGETS: aarch64 arm x86_64

app-emulation/qemu      static-user
dev-libs/glib           static-libs
sys-apps/attr           static-libs
sys-libs/zlib           static-libs
dev-libs/libpcre2       static-libs

OpenRC

Enable qemu-binfmt:

1

rc-update add qemu-binfmt default

Start qemu-binfmt:

1

rc-service qemu-binfmt start

Chrooting

• select chroot location (eg /chroots/gentoo-arm64-musl-stable)
• unpack the desired rootfs
• create needed directories
  • mkdir -p /chroots/gentoo-arm64-musl-stable/var/cache/distfiles
• execute bwrap
  • with last ro-bind mount the qemu emulator binary (eg qemu-aarch64)
  • execute the mounted emulator binary giving it a shell program (eg bash)

Chroot with bwrap:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

bwrap                                                       \
    --bind /chroots/gentoo-arm64-musl-stable /              \
    --dev /dev                                              \
    --proc /proc                                            \
    --perms 1777 --tmpfs /dev/shm                           \
    --tmpfs /run                                            \
    --ro-bind /etc/resolv.conf /etc/resolv.conf             \
    --bind /var/cache/distfiles /var/cache/distfiles        \
    --ro-bind /usr/bin/qemu-aarch64 /usr/bin/qemu-aarch64   \
    /usr/bin/qemu-aarch64 /bin/bash -l

User-mode

By default you would probably have something like this, the user-mode network:

1
2
3
4
5

<interface type="user">
  <mac address="00:00:00:00:00:00"/>
  <model type="virtio"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

Bridge

Bridges can be easily created using the NetworkManager’s TUI tool called nmtui.

Bridge XML configuration for Libvirt

1
2
3
4
5
6
7
8

<interface type="bridge">
  <mac address="00:00:00:00:00:00"/>
  <source bridge="br1"/>
  <target dev="vnet2"/>
  <model type="virtio"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
</interface>

Sysctl options

Be sure the following options are enabled (1):

• net.ipv4.ip_forward
• net.ipv4.conf.all.send_redirects

and the following options are disabled (0):

• net.bridge.bridge-nf-call-iptables

Building PowerShell

As a part of my work of modernizing the way .NET SDK packages are distributed in Gentoo I delved into packaging a from-source build of PowerShell for Gentoo using the dotnet-pkg eclass.

Packaging pwsh was a little tricky but I got a lot of help from reading the Alpine Linux’s APKBUILD. I had to generate special C# code bindings with ResGen and repackage the PowerShell tarball. Other than this trick, restoring and building PowerShell was pretty straight forward with the NuGet package management support from the dotnet-pkg.eclass.

Alternatively if you do not want to build PowerShell you can install the binary package, I have in plans to keep that package around even after we get the non-binary app-shells/pwsh into the official Gentoo ebuild repository.

Why install modules via Portage?

But why stop on PowerShell when we can also package multiple PS modules?

Installing modules via Portage has many benefits:

• better version control,
• more control over global install,
• no need to enable PS Gallery,
• sandboxed builds,
• using system .NET runtime.

Merging the modules

PowerShell’s method of finding modules is at follows: check paths from the PSModulePath environment variable for directories containing valid .psd1 files which define the PS modules.

By default pwsh tries to find modules in paths:

• user’s modules directory — ~/.local/share/powershell/Modules
• system modules directory in /usr/local — /usr/local/share/powershell/Modules
• Modules directory inside the pwsh home — for example /usr/share/pwsh-7.3/Modules

Because we do not want to touch either /usr/local nor pwsh home, we embed a special environment variable inside the pwsh launcher script to extend the path where pwsh looks for PS modules. The new module directory is located at /usr/share/GentooPowerShell/Modules.

1
2

dotnet-pkg-utils_append_launchervar \
    'PSModulePath="${PSModulePath}:/usr/share/GentooPowerShell/Modules:"'

So every PowerShell module will install it’s files inside /usr/share/GentooPowerShell/Modules.

To follow PS module location convention we add to that path a segment for the real module name and a segment for module version. This also enables us to have proper multi-slotting because most of the time the modules will not block installing other versions.

Take a look at this example from the app-pwsh/posh-dotnet–1.2.3 ebuild:

1
2
3
4
5
6

src_install() {
    insinto /usr/share/GentooPowerShell/Modules/${PN}/${PV}
    doins ${PN}.psd1 ${PN}.psm1

    einstalldocs
}

And that is it. Some packages do not even need to be compiled, they just need files placed into specific location. But when compilation of C# code is needed we have dotnet-pkg to help.