Posts tagged system

How it works

• There will be a FUSE OneDrive filesystem mounted on the machine Nextcloud runs on,
• You configure filesystem-local “External storage” and point to the mountpoint of a cloud drive (in this case OneDrive),
• users connected via Nextcloud Client will have a option to sync any chosen files from the External storage as if they were Nextcloud-owned files.

Benefits

There are some benefits from connecting OneDrive to Nextcloud:

• faster cached sync than by normal straight-to OneDrive connection - files will be pushed to Nextcloud and then uploaded to Nextcloud, this will take less time than uploading straight to OneDrive because of Microsoft OneDrive rate-limiting the uploads (especially in case of larger files),
• clients of Nextcloud do not have to configure any OneDrive connection,
• you will be able to have two-way sync of your OneDrive files (currently two-way sync on the RClone side is experimental, this will use Nextcloud’s two-way sync mechanism).

Set up RClone

First You will have to set up RClone. Connect to your cloud of choice and then copy the config to a location that will be readable by a service that mounts a given cloud drive.

For OneDrive I use /usr/local/share/rclone/config/rclone.conf that is accessible only to the apache user.

The config will look something like this:

1
2
3
4
5
6

[OneDrive]
type = onedrive
region = global
token = *REDACTED*
drive_id = *REDACTED*
drive_type = personal

Mounting OneDrive

I created this helper script (in /usr/local/bin/rclone-mount.sh):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/bin/sh

set -e
set -u
trap "exit 128" INT

conf_path="${1}"

local_user="${2}"
local_path="${3}"

cloud_name="${4}"
cloud_path="${5}"

log_dir="/var/log/rclone"
log_file="${log_dir}/rclone-mount-${local_user}-${cloud_name}.log"

mkdir -p "${log_dir}"
touch "${log_file}"

chmod a+r "${log_file}"
chown "${local_user}" "${log_file}"

exec rclone                                         \
     --default-permissions                          \
     --allow-other                                  \
     --verbose                                      \
     --vfs-cache-mode full                          \
     --config "${conf_path}"                        \
     mount                                          \
     "${cloud_name}:${cloud_path}" "${local_path}"  \
     >> "${log_file}" 2>&1

Then, I use it in a OpenRC service like this (/etc/init.d/mount-OneDrive):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/sbin/openrc-run

conf_path="/usr/local/share/rclone/config/rclone.conf"

cloud_name="OneDrive"
cloud_path="/"

local_user="apache"
local_path="/mnt/${cloud_name}"

command="/usr/local/bin/rclone-mount.sh"
command_args="${conf_path} ${local_user} ${local_path} ${cloud_name} ${cloud_path}"

command_background="false"
command_user="${local_user}:$(id -g -n ${local_user})"
supervisor="supervise-daemon"

depend() {
    need net
}

start_pre() {
    ebegin "Unmounting leftovers from ${local_path} before service start"
    umount "${local_path}"
    eend 0
}

stop_post() {
    ebegin "Unmounting leftovers from ${local_path} after service stop"
    umount "${local_path}"
    eend 0
}

Enabling the RClone service

Set up directories and correct permissions:

1
2
3
4
5

mkdir -p /usr/local/share/rclone/config
chown -R apache:apache /usr/local/share/rclone/config

mkdir -p /var/log/rclone
chown -R apache:apache /var/log/rclone

Do not forget to make the mount service script executable:

1

chmod +x /etc/init.d/mount-OneDrive

Enable and start this service on OpenRC:

1
2

rc-update add mount-OneDrive default
rc-service mount-OneDrive start

Drive permissions

RClone mounts cloud drives by using FUSE. To have the RClone option --allow-other available in order to allow root to access the drive you will have to modify the FUSE config file (/etc/fuse.conf) - add user_allow_other.

Nextcloud configuration

Download and enable the “External storage” app. Then, in “Administration” settings add a external storage:

• name: ExternalStorage_OneDrive
• type: Local
• authentication: None
• configuration: /mnt/OneDrive
• available for: YOUR USER

The official ::gentoo repository currently contains only GHC on version 9.2.8. To install newer GHC one has to either download/build themselves or use the ::haskell overlay (https://github.com/gentoo-haskell/gentoo-haskell).

Enable the ::haskell overlay

Enable:

1

eselect repository enable haskell

Sync:

1
2
3

emerge --sync haskell
egencache --update --repo haskell --jobs 12 --load 6
eix-update

Unmask needed packages

Add to /etc/portage/package.unmask/0000_hs.conf

1
2
3
4
5
6
7
8

<dev-lang/ghc-9.9

<dev-haskell/cabal-3.11
<dev-haskell/cabal-install-3.11
<dev-haskell/cabal-install-solver-3.11
<dev-haskell/cabal-syntax-3.11
<dev-haskell/text-2.2
<dev-haskell/parsec-3.1.18

Add to /etc/portage/package.accept_keywords/0000_hs.conf

1
2
3

app-admin/haskell-updater
dev-haskell/*
dev-lang/ghc

Install

1

emerge --ask --verbose ">=dev-lang/ghc-9.8" ">=dev-haskell/cabal-install-3.10"

Build of GHC 9.8 takes around ~2 hours on a 8-core laptop-grade CPU.

Bonus: masking packages from ::haskell

If you want to exclude a given version from the ::haskell overly from being installed/updated, then you can add a similar line(s) to /etc/portage/package.mask/0000_hs.conf:

1

app-emacs/haskell-mode::haskell

SysStat is a amazing tool. In the age where telegraf and grafana are all the rage everybody forgot about the good old sysstat.

Selected command examples

• iostat -d -p nvme0n1 3 - disk I/O for a NVME drive (nvme0n1),
• sar -n DEV 3 - network throughput,
• sar -h -r 3 - memory usage,
• sar -P ALL 3 - CPU utlization
• sar -q 3 - system load levels,
• sar -A 3 - all the metrics.

Gathered info

qlist app-admin/sysstat | grep /usr/bin/

The app-admin/sysstat contains the following binaries and their respective statictic fields:

• sar - general utilization statistics,
• cifsiostat - CIFS,
• iostat - device input/output,
• mpstat - processors,
• pidstat - Linux tasks,
• tapestat - tape (yes, the real tape disks).

Installation

Gentoo

1
2
3

emerge --noreplace --verbose sys-process/cronie app-admin/sysstat
rc-service cronie start
rc-update add cronie default

Files

By default (on Gentoo): * sa (the collector) saves statistics to /var/log/sa, * /etc/sysstat is the configuration file * cron jobs are run via the *system* cronjob table.

Add Flathub-Beta

Add the Flathub-Beta remote repository:

1

flatpak remote-add --user flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo

Install GIMP beta

Install org.gimp.GIMP form flathub-beta:

1

flatpak install --assumeyes --user flathub-beta org.gimp.GIMP

Run GIMP

If you have other GIMP versions installed you will have to specify the “beta” version with //beta.

1

flatpak run org.gimp.GIMP//beta

Otherwise you can just run:

1

flatpak run org.gimp.GIMP

Also, in the desktop menus (like KRunner) this version of GIMP will have (beta) in its name so there is no chance to miss it.

Nix is useful for quickly testing out software and providing a strict environment that can be shared between people.

Today I’m trying out Nix again, this time I want to do it my way.

Installation process

Nix store

I know Nix needs “Nix store” installation on / (the system root).

Create it manually to prevent the installation script from calling sudo. 1st I switch to the root account, and then I run:

1
2

mkdir -p -m 0755 /nix
chown -R xy:xy /nix

Running the install script

Download the Nix install script and examine the contents.

1

curl -L https://nixos.org/nix/install > nix_install.sh

Then, run it with --no-daemon to prevent it running as system service.

1

sh ./nix_install.sh --no-daemon

performing a single-user installation of Nix...
copying Nix to /nix/store...
installing 'nix-2.20.1'
building '/nix/store/1ahlg3bviy174d6ig1gn393c23sqlki6-user-environment.drv'...
unpacking channels...
modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

. /home/xy/.nix-profile/etc/profile.d/nix.fish

in your shell.

Wait!

modifying /home/xy/.bash_profile...
modifying /home/xy/.zshenv...
placing /home/xy/.config/fish/conf.d/nix.fish...

That’s very rude!

Stopping Nix from making a mess

I need to prevent Nix from mess up with my environment when I do not want it to. Nix puts some code into the Bash, ZSH and Fish initialization files during installation to ease it’s use. I do not want that since I do not want Nix to meddle with my environment without me knowing it.

I keep my .bash_profile and .zshenv in a stow-managed git repo so I can just cd into my repo and do git reset --hard, but for you will have to revert those files to their old forms manually.

Playing with Nix

We do not have nix in PATH but we still can launch it. Nix executables are located inside ~/.nix-profile/bin/.

By invoking nix-shell one can create a ephemeral environment containing only packages specified after the -p flag. I always add -p nix to have the Nix tools available also inside the spawned environment.

I will test out chibi (small Scheme interpreter) + rlwrap (REPL support for software lacking it) inside a Nix ephemeral environment:

1

~/.nix-profile/bin/nix-shell -p nix chibi rlwrap

Inside the spawned shell:

1

rlwrap chibi-scheme

In the chibi REPL, let’s see the contents of the PATH environment variable:

1

(get-environment-variable "PATH")

And exit the Scheme REPL:

1

(exit)

After the playtime, run garbage collection:

1

~/.nix-profile/bin/nix-collect-garbage

Portage as a CD system

This is a very simple way to use any system with Portage installed as a Continuous Delivery server.

I think for a testing environment this is a valid solution to consider.

Create a repository of software used in your organization

Those articles from the Gentoo Wiki describe how to create a custom ebuild repository (overlay) pretty well:

• Ebuild repository
• Creating an ebuild repository

Set up your repo with eselect-repository

Install the my-org repository:

1

eselect repository add my-org git https://git.my-org.local/portage/my-org.git

Sync my-org:

1

emerge --sync my-org

Install live packages of a your software

First, enable live packages (keywordless) for your my-org repo:

1

echo '*/*::my-org' >> /etc/portage/package.accept_keywords/0000_repo_my-org.conf

Install some packages from my-org:

1

emerge -av "=mycategory/mysoftware-9999"

Install smart-live-rebuild

smart-live-rebuild can automatically update live software packages that use git as their source URL.

Set up cron to run smart-live-rebuild

Refresh your my-org repository every hour:

1

0 */1 * * * emerge --sync my-org

Refresh the main Gentoo tree every other 6th hour:

1

0 */6 * * * emerge --sync gentoo

Run smart-live-rebuild every other 3rd hour:

1

0 */3 * * * smart-live-rebuild

Restarting services after update

All-in-one script

You can either restart all services after successful update:

File: /opt/update.sh

1
2
3
4
5
6
7
8

#!/bin/sh

set -e

smart-live-rebuild

systemctl restart my-service-1.service
systemctl restart my-service-2.service

Crontab:

1

0 */3 * * * /opt/update.sh

Via ebuilds pkg_ functions

File: my-service-1.ebuild

1
2
3

pkg_postinst() {
    systemctl restart my-service-1.service
}

More about pkg_postinst:

• pkg_postinst
• Phase Functions

Example Gentoo overlays

• xgqt/myov
• gentoo-racket/gentoo-racket-overlay
• gentoo-mirror/guru

I really wanted to look into the new kernel building solutions for Gentoo and maybe migrate to dracut, but last time I tried, ~1.5 years ago, the initreamfs was now working for me.

And now in 2023 I’m still running genkernel for my personal boxes as well as other servers running Gentoo.

I guess some short term solutions really become defined tools :P

So this is how I rebuild my kernel nowadays:

1. Copy old config

   1 2	cd /usr/src cp linux-6.1.38-gentoo/.config linux-6.1.41-gentoo/

2. Remove old kernel build directories

   1	rm -r linux-6.1.31-gentoo

3. Run initial preparation

   1	( eselect kernel set 1 && cd /usr/src/linux && make olddefconfig )

4. Call genkernel

   1 2 3 4 5 6 7 8 9 10 11 12

   genkernel \ --no-menuconfig \ --no-clean \ --no-clear-cachedir \ --no-cleanup \ --no-mrproper \ --lvm \ --luks \ --mdadm \ --nfs \ --kernel-localversion="-$(hostname)-$(date '+%Y.%m.%d')" \ all

5. Rebuild the modules

   If in your /etc/genkernel.conf you have MODULEREBUILD turned off, then also call emerge:

   1	emerge -1 @module-rebuild