You are not logged in.

#1 2017-06-22 09:31:44

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 288

How to set up a build slave

Hi all,

I think, I've finally managed to make the set up of a build slave compact enought for "externals" to master:

A build slave is used to recompile the official packages for i686. Other architectures might follow in the future.

It is currently required to run on an arch x86_64 installation (it may run on i686 with a few modifications, though).

There is a package 'devtools32' in releng, which you need to install (alongside its dependencies 'pacman-mirrorlist32' and 'archlinux32-keyring').
This is a fork of the official devtools, which allows compiling i686 packages.
Note, that you may want to include the repository in your pacman.conf, so you will receive regular updates. The keys used for signing are the standard archlinux32 packaging keys from archlinux32-keyring.

Then you need to clone the builder scripts, which handle the interaction with the build master.
Feel free to override variables (which you understand) from 'conf/default.conf' by putting their replacement value in 'conf/local.conf'.

After configuration, source 'conf/default.conf' to generate a valid working tree, ssh-key and alike.

Then you need to give deep42tought your public ssh-key (In the standard configuration, it is found in 'work/.ssh/id_rsa.pub'), so he can grant you access to the build master.
Furthermore you need a gpg key for signing packages. If you do not have one yet, generate it now, get it signed by at least three devs and get it introduced in archlinux32-keyring (the last two points are basically the same).
Once your gpg key is in archlinux32-keyring and your ssh login to the build master works, you may fire up your build slave by running 'bin/build-packages' to your likings (preferably in a screen or tmux session in the background).

Note, that 'build-packages' comes with a few options to control how many packages are to be built and when to abort.

cheers,
deep42thought

Offline

#2 2017-06-22 17:53:11

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 89

Re: How to set up a build slave

How does the three-dev requirement work? How many dev keys are there currently in archlinux32-keyring? Is there a master key that has signed these?

btw, my public key (signature F295 5C3B 63EE 4E42 1F30 670A 3B08 EFC6 BA97 4CFC) is available on most keyservers, and is verifiable on https://keybase.io/tyzoid

Also, is this ssh key different from our user ssh key?

Offline

#3 2017-06-22 21:12:40

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 288

Re: How to set up a build slave

Currently, I (deep42thought), Polichronucci and City-Busz have a master key each. (The requirement of three dev-signatures is by gpg, not us.) These are the top of the chain: When installing archlinux32-keyring, pacman signs the master keys with your local pacman key and therefore trusts any key which has at least three dev signatures - or one signature by your local pacman key, for that matter.

The pgp key does not necessarily be your usual email key (in fact, the archlinux32-keyring script as forked from archlinuxarm-keyring failed with my email-key with multiple user ids - so it might be necessary to create a separate package signing key with only one user id).
The same holds true for the ssh key: It might just be your usual ssh key, but you may as well (let) generate a separate one just for interfacing the build master (this is the default). The latter is especially useful if you need full ssh access to the build master, too, because the access for build slaves is restricted to the commands they need.

Offline

#4 2017-06-22 21:33:23

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 89

Re: How to set up a build slave

Interesting. Is there a way to test whether my regular key would work?

Offline

#5 2017-06-22 21:48:38

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 288

Re: How to set up a build slave

You can clone archlinux32-keyring, insert your key in the file 'packager-keyids' and run 'update-keys'.

Offline

#6 2017-06-23 11:15:31

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 288

Re: How to set up a build slave

ah, sry I was wrong: the restriction of only one uid was for the master-key, so your regular gpg key should work

Offline

#7 2018-04-25 07:31:34

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 288

Re: How to set up a build slave

The packages required for building are now in [releng] which can be found on our mirror:
http://pool.mirror.archlinux32.org/x86_64/releng

Offline

#8 2018-05-24 13:37:32

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 89

Re: How to set up a build slave

I recently did a setup of the build slave on a new x86_64 container. Here's what I did:

  1. Add the archlinux32 x86_64/releng repo to pacman.conf:

    [releng]
    Server = https://${ANY_ARCHLINUX32_MIRROR}/$arch/$repo
  2. Update the repo-db/update packages (pacman -Syu)

  3. Install archlinux32-keyring-transition

  4. Install archlinux32-keyring (replaces archlinux32-keyring-transition)

  5. Install dependencies (base-devel wget moreutils devtools32)

  6. Uncomment a mirror of your choice from /etc/pacman.d/mirrorlist32

    • Optional: update pacman.conf to point to mirrorlist32 instead of mirror directly

  7. Create a user 'builder'

  8. visudo and add the build commands to builder's user:

    Cmnd_Alias BUILD_CMDS = /usr/bin/staging-with-build-support-i686-build, /usr/bin/staging-i686-build, /usr/bin/multilib-build, /usr/bin/extra-x86_64-build
    builder ALL=(ALL) NOPASSWD: BUILD_CMDS
  9. su to builder

  10. create .gnupg directory, and add to ~/.gnupg/gpg.conf:

    keyserver-options auto-key-retrieve
  11. git clone the builder repo: https://github.com/archlinux32/builder.git

    • Make any changes of defaults (builder/conf/default.conf) in builder/conf/local.conf (do not copy default.conf to local.conf - only add changed values)

    • Remember to set the key ID

  12. run builder/bin/build-packages to clone upstream's pkgbuild files and verify everything is working.

    • Note: This will generate an error after cloning about not having permission to access buildmaster. Just hit Ctrl+C to quit once you hit this error.

  13. send the contents of 'builder/work/.ssh/id_rsa.pub' to deep42thought

  14. Import your build key (private key!) to the builder's gpg keyring - strip passphrase if one exists

  15. run builder/bin/build-packages (in a screen/tmux buffer) to start building!

    • I used the following unit file to run it as a systemd service:

      [Unit]
      Description=Archlinux32 Build Slave
      
      [Service]
      Type=simple
      Restart=always
      RestartSec=10
      User=builder
      ExecStart=/home/builder/builder/bin/build-packages
      
      [Install]
      WantedBy=multi-user.target

Offline

Board footer

Powered by FluxBB