Testing an Arch Linux package in Gitlab CI

I recently created a utility for backing up file system metadata called metadata-backup, which is my first project to make use of Gitlab's CI/CD offering. It was pretty easy to set up, but I was shocked to find that by default, all CI pipelines are run as root; this is because Gitlab uses docker for their CI, and docker containers default to running commands as root.

The first symptom of this was when my tests about how to handle permissions errors started failing, but I had a much bigger problem when I went to set up a CI pipeline for the corresponding Arch Linux package.

makepkg and root

Packages in Arch Linux are built with makepkg, which is a multi-use tool that does things like install and build packages from a PKGBUILD file. For various reasons, makepkg cannot be run as root; if you try to do so, you'll get this scary-looking message:

$ sudo makepkg
==> ERROR: Running makepkg as root is not allowed as it can cause
permanent, catastrophic damage to your system.

This makes sense for a couple of reasons. For one thing, PKGBUILD necessarily contains arbitrary code used to build, install and test your package — even if you trust a package source enough to install it, you may not want to trust it enough to run its code with root privileges. Additionally, processes running as root will blithely ignore file permissions, many of which serve to prevent you from accidentally damaging your system.

makepkg will likely need root privileges to install your software (or its dependencies or build-time dependencies), but since only certain actions need those privileges, makepkg will use sudo as necessary for the commands that require escalated privileges.

Using the nobody user

Since Gitlab CI runs all commands as root by default, and makepkg cannot be run as root, we need to use a non-root user account to run our commands. If you don't have any build-time dependencies, this is actually very simple, because Linux systems traditionally have a non-human role account called nobody which serves the opposite purpose as the root user: it has no special permissions on the system. For simple invocations of makepkg, you can simply run the commands as nobody by putting sudo -u nobody before the command. Here is an example of a basic CI configuration that does this: [1]

image: "archlinux"

test:makepkg:
  before_script:
    # Install the utilities needed to run makepkg
    - pacman -Sy binutils fakeroot sudo --noconfirm --needed

  script:
    # Test building a source package
    - sudo -u nobody makepkg -C -S --log --noconfirm
    # Test building and installing the package
    - sudo -u nobody makepkg -C -i --log --noconfirm

Building a custom user

In my case, the nobody user did not suffice because my package did require some special permissions in order to complete the build, specifically:

  1. My build uses cargo, which stores some configuration in the user's home directory, so I need a user account that had a home directory.
  2. My PKGBUILD has build-time dependencies specified in makedepends, so I need to be able to escalate to root privileges in order to install them.
  3. Because I am running this in CI, I cannot allow sudo to prompt me for a password, since anything that asks for user input causes the CI pipeline to immediately fail.

To solve these problems, I created a custom user named non_root in my before_script, and made a home directory for it. In order to allow sudo to work with no password for my user, I needed to append the following line to the /etc/sudoers file:

non_root ALL= NOPASSWD: ALL

This is a user specification, which has the following format

USER HOST=(RUN_AS_USER:RUN_AS_GROUP) COMMAND

This says that the USER may use sudo to execute the command COMMAND. Adding NOPASSWORD: as part of the command means that they will not be prompted for a password when they call sudo for this particular command.

The HOST argument specifies what hosts this rule is valid for — in this case I have chosen the special value ALL to mean that it's valid for all hosts. The (RUN_AS_USER:RUN_AS_GROUP) portion of the line can be used to allow the user to run commands as other users or groups with the -u and -g arguments, respectively. It is common to see ALL or (ALL:ALL) used to allow the user to act as anyone or any group, but in my case this is not necessary, so I have left it blank, which disables -u and -g. [2]

With this additional information, you can see that the line I added to /etc/sudoers says: "the user non_root may use sudo on any host to execute any command without being prompted for a password, but may not run the command as any user other than non_root", which solves criteria 2 and 3.

Putting it all together, here is the .gitlab-ci.yaml file: [3]

image: "archlinux"

test:makepkg:
  before_script:
    # Install the utilities needed to run makepkg
    - pacman -Sy binutils fakeroot sudo --noconfirm --needed

    # Create a non-root user in sudoers with a home directory
    - "echo 'non_root ALL=NOPASSWD: ALL' >> /etc/sudoers"
    - mkdir /home/non_root
    - chown -R non_root:non_root /home/non_root


  script:
    # Note: -s installs required dependencies. Use `-sr` to install the required
    # dependencies and remove the unnecessary ones afterwards.
    # Test building a source package
    - sudo -u non_root makepkg -C -sS --log --noconfirm
    # Test building and installing the package
    - sudo -u non_root makepkg -C -si --log --noconfirm

Eagle-eyed readers may notice that I needed to make one additional modification to the original YAML file: in order to tell makepkg to install my build-time dependencies, I added the -s flag to each of the commands.

Final thoughts

I will note that this solution is something of a hack that I've done to avoid building and maintaining a custom Docker image just to run this, because at the moment I've only got the one Arch Linux package. If you are maintaining many such packages, or you are more comfortable than me maintaining docker images, I believe you can achieve the same thing by simply setting up a docker container that runs all commands as your non_root user. The Dockerfile would look something like this:

FROM archlinux:latest

RUN pacman -Sy binutils fakeroot sudo --noconfirm --needed
RUN useradd non_root && mkdir /home/non_root && chown -R non_root:non_root /home/non_root
RUN echo 'non_root ALL=NOPASSWD: ALL' >> /etc/sudoers
USER non_root

Then your Gitlab CI file is significantly simplified:

image: "your-repository/your-image"

test:makepkg:
  script:
    # Note: -s installs required dependencies. Use `-sr` to install the required
    # dependencies and remove the unnecessary ones afterwards.
    # Test building a source package
    - makepkg -C -sS --log --noconfirm
    # Test building and installing the package
    - makepkg -C -si --log --noconfirm

This is almost certainly the right thing to do, but I imagine there are many people like me who have decided that configuring a custom docker image is something they will do "at some point" or "when I have free time", and will appreciate my quick-and-dirty way to solve the root problem using just Gitlab's CI.

Footnotes

[1]I will note that I have not spent a lot of time thinking about what sets of commands to makepkg make good tests, and in these examples I'm not even asserting that the build creates anything. If you have suggestions for improvements, please send me an e-mail, I'd be happy to update the post with better advice on testing these package builds.
[2]You may be naturally confused when you read that I'm disabling sudo -u in the sudoers file, and then immediately afterwards you see sudo -u in the .gitlab-ci.yml file. The reason for this is that these permissions are for my non_root user, who only needs sudo access to the extent that makepkg makes sudo calls, and makepkg does not use -u or -g; the sudo -u calls in the .gitlab-ci.yml file are made by root, who does have the ALL=(ALL:ALL) permission.
[3]I can never remember the canonical way to create a home directory for a user, so I've gone with mkdir and chown. Please feel free to correct me.