You are not logged in.

#1 2017-06-22 10:31:44

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 617

How to set up a build slave

Hi all,

I think, I've finally managed to make the set up of a build slave compact enought for "externals" to master:

A build slave is used to recompile the official packages for i686. Other architectures might follow in the future.

It is currently required to run on an arch x86_64 installation (it may run on i686 with a few modifications, though).

There is a package 'devtools32' in releng, which you need to install (alongside its dependencies 'pacman-mirrorlist32' and 'archlinux32-keyring').
This is a fork of the official devtools, which allows compiling i686 packages.
Note, that you may want to include the repository in your pacman.conf, so you will receive regular updates. The keys used for signing are the standard archlinux32 packaging keys from archlinux32-keyring.

Then you need to clone the builder scripts, which handle the interaction with the build master.
Feel free to override variables (which you understand) from 'conf/default.conf' by putting their replacement value in 'conf/local.conf'.

After configuration, source 'conf/default.conf' to generate a valid working tree, ssh-key and alike.

Then you need to give deep42tought your public ssh-key (In the standard configuration, it is found in 'work/.ssh/id_rsa.pub'), so he can grant you access to the build master.
Furthermore you need a gpg key for signing packages. If you do not have one yet, generate it now, get it signed by at least three devs and get it introduced in archlinux32-keyring (the last two points are basically the same).
Once your gpg key is in archlinux32-keyring and your ssh login to the build master works, you may fire up your build slave by running 'bin/build-packages' to your likings (preferably in a screen or tmux session in the background).

Note, that 'build-packages' comes with a few options to control how many packages are to be built and when to abort.

cheers,
deep42thought

Offline

#2 2017-06-22 18:53:11

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 86

Re: How to set up a build slave

How does the three-dev requirement work? How many dev keys are there currently in archlinux32-keyring? Is there a master key that has signed these?

btw, my public key (signature F295 5C3B 63EE 4E42 1F30 670A 3B08 EFC6 BA97 4CFC) is available on most keyservers, and is verifiable on https://keybase.io/tyzoid

Also, is this ssh key different from our user ssh key?

Offline

#3 2017-06-22 22:12:40

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 617

Re: How to set up a build slave

Currently, I (deep42thought), Polichronucci and City-Busz have a master key each. (The requirement of three dev-signatures is by gpg, not us.) These are the top of the chain: When installing archlinux32-keyring, pacman signs the master keys with your local pacman key and therefore trusts any key which has at least three dev signatures - or one signature by your local pacman key, for that matter.

The pgp key does not necessarily be your usual email key (in fact, the archlinux32-keyring script as forked from archlinuxarm-keyring failed with my email-key with multiple user ids - so it might be necessary to create a separate package signing key with only one user id).
The same holds true for the ssh key: It might just be your usual ssh key, but you may as well (let) generate a separate one just for interfacing the build master (this is the default). The latter is especially useful if you need full ssh access to the build master, too, because the access for build slaves is restricted to the commands they need.

Offline

#4 2017-06-22 22:33:23

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 86

Re: How to set up a build slave

Interesting. Is there a way to test whether my regular key would work?

Offline

#5 2017-06-22 22:48:38

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 617

Re: How to set up a build slave

You can clone archlinux32-keyring, insert your key in the file 'packager-keyids' and run 'update-keys'.

Offline

#6 2017-06-23 12:15:31

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 617

Re: How to set up a build slave

ah, sry I was wrong: the restriction of only one uid was for the master-key, so your regular gpg key should work

Offline

#7 2018-04-25 08:31:34

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 617

Re: How to set up a build slave

The packages required for building are now in [releng] which can be found on our mirror:
http://pool.mirror.archlinux32.org/x86_64/releng

Offline

#8 2018-05-24 14:37:32

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 86

Re: How to set up a build slave

I recently did a setup of the build slave on a new x86_64 container. Here's what I did:

  1. Add the archlinux32 x86_64/releng repo to pacman.conf:

    [releng]
    Server = https://${ANY_ARCHLINUX32_MIRROR}/$arch/$repo
  2. Update the repo-db/update packages (pacman -Syu)

  3. Install archlinux32-keyring-transition

  4. Install archlinux32-keyring (replaces archlinux32-keyring-transition)

  5. Install dependencies (base-devel wget moreutils devtools32)
    Note: devtools32 might not be up-to-date, use the git repo from https://git.archlinux32.org/archlinux32/devtools32.git directly and install manually

  6. Uncomment a mirror of your choice from /etc/pacman.d/mirrorlist32

    • Optional: update pacman.conf to point to mirrorlist32 instead of mirror directly

  7. update pacman*.conf files in /usr/share/devtools when building on x86_64 (set Architecture manually to i486, i686, not to auto)

  8. Create a user 'builder'

  9. visudo and add the build commands to builder's user:

    Cmnd_Alias BUILD_CMDS = /usr/bin/staging-with-build-support-i686-build, /usr/bin/staging-i686-build, /usr/bin/multilib-build, /usr/bin/extra-x86_64-build
    (Note: there might be many more, pentium4)
    builder ALL=(ALL) NOPASSWD: BUILD_CMDS
  10. su to builder

  11. create .gnupg directory, and add to ~/.gnupg/gpg.conf:

    keyserver-options auto-key-retrieve
  12. git clone the builder repo: https://git.archlinux32.org/archlinux32/builder.git

    • Make any changes of defaults (builder/conf/default.conf) in builder/conf/slave.conf (do not copy default.conf to slave.conf - only add changed values, see slave.conf.example for configuration variables.

    • Remember to set the key ID ('package_key'='0xXXXXX')

  13. run builder/bin/build-packages to clone upstream's pkgbuild files and verify everything is working. First time you start it you get asked about generating a private/public key pair. Leave the password blank.

    • Note: This will generate an error after cloning about not having permission to access buildmaster. Just hit Ctrl+C to quit once you hit this error.

  14. send the contents of 'builder/work/.ssh/id_rsa.pub' to deep42thought

  15. Import your build key (private key!) to the builder's gpg keyring - strip passphrase if one exists

  16. the git repositories in work/repos/ are packages, community and packages32, they are in bare format, so you have to clone them once, remove the bare once and rename the checkout out workspaces back. Also change the remote origin to point to the Gitea/SVNGit repos and not to the bare file system (git://git.archlinux.org/svntogit/packages.git, git://git.archlinux.org/svntogit/community.git and git@git.archlinux32.org:archlinux32/packages.git).

  17. Set PACKAGER in /etc/makepkg.conf on the host

  18. run builder/bin/build-packages (in a screen/tmux buffer) to start building!

    • I used the following unit file to run it as a systemd service:

      [Unit]
      Description=Archlinux32 Build Slave
      
      [Service]
      Type=simple
      Restart=always
      RestartSec=10
      User=builder
      ExecStart=/home/builder/builder/bin/build-packages
      
      [Install]
      WantedBy=multi-user.target

Offline

Board footer

Powered by FluxBB