You are not logged in.

#1 2017-10-30 12:47:22

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 275

automatic testing via manual installation(s)

Hi,

I would like to encourage more archlinux32 users to take part in testing packages.

There is a package (report-installed-packages) to send a correctly formatted email report about installed packages from testing/community-testing to the build master.
The build master will then mark these packages as tested and move them to their stable repositories if appropriate.

To set it up, you will need to:

  • activate the testing and community-testing repositories in your /etc/pacman.conf

  • install report-installed-packages

  • install hashcash

  • install sendmailadvanced

  • generate a gpg key, if you don't have one already

  • set up some "sendmail" command (for example msmtp)

  • let me know your gpg fingerprint, so I can put it in the build master's white list

Then you can simply run

report-installed-packages you@your.mail buildmaster@archlinux32.org

to report all installed packages (and their versions) from testing and community-testing to the build-master. (I do this before every invocation of "pacman -Syu")

Some sidenote:
You will send information about installed packages to a place you don't control (e.g. the buildmaster). However, the information should be encrypted on its way and the build master should only evaluate if you are allowed to mark packages as tested and, if so, mark the mentioned packages as tested. If you are paranoid enough (no offense), you can examine the relevant script on the buildmaster, as well as the scripts in report-installed-packages and sendmailadvanced.

cheers,
deep42thought

Offline

#2 2017-11-07 10:33:22

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 275

Re: automatic testing via manual installation(s)

The build master now checks our bug tracker for reports of bugs in packages in testing or community-testing before moving these packages to stable, so the report-installed-packages script is now safer to use: It will not move packages to stable if they work on one box but not another.

Offline

#3 2017-12-01 11:34:09

eugen-b
Member
Registered: 2017-07-09
Posts: 37

Re: automatic testing via manual installation(s)

Hi,
this is a great idea to automatize testing feedback!

It was however not easy to set up on my end as tester. Especially the part with "set up some "sendmail" command (for example msmtp)" took some effort. Finally the sending part seems to work, the report generation is not yet tested on my end. I will write down the steps I needed to set up the sending part of the process:
* Downloaded, checked and installed the packages

wget https://arch.eckner.net/archlinuxewe/os/i686/report-installed-packages-0.2-1-any.pkg.tar.xz
wget https://arch.eckner.net/archlinuxewe/os/i686/report-installed-packages-0.2-1-any.pkg.tar.xz.sig
gpg --search-keys 11D4A57809DCA3F25C63A2BD77BEA194A234FF8F
gpg --verify *.sig
rm *.sig
wget https://arch.eckner.net/archlinuxewe/os/i686/sendmailadvanced-1.3-1-any.pkg.tar.xz
wget https://arch.eckner.net/archlinuxewe/os/i686/sendmailadvanced-1.3-1-any.pkg.tar.xz.sig
gpg --verify *.sig
sudo pacman -U *pkg.tar.xz

* I had to import my public and private keys from another machine, I created them with the following commands (using the default parameters for algorithm and expiration, used a secure passphrase, a more precise approach is given in the  Arch Wiki):

gpg --gen-key
gpg --fingerprint eugen-b
gpg --keyserver pool.sks-keyservers.net --send-keys 'C4EC 81A0 F6E0 B1F9 3CB3  5EA5 E422 5CD9 EE14 90FE'

You don't need to publish your public key for this task here, you can send the finderprint to deep42thought.
Then I exported the keys to copy them over to the archlinux32 netbook:

gpg --export E4225CD9EE1490FE > public.key
gpg --export-secret-key E4225CD9EE1490FE > private.key

On the netbook then 'cd' to the folder with public.key and private.key and:

gpg --import public.key
gpg --import private.key

* I decided (as suggested) to use msmtp + msmtp-mta to set up the sendmail functionality and followed the Arch Wiki.
I copied the config file

cp /usr/share/doc/msmtp/msmtprc-user.example ~/.msmtprc
chmod 600 .msmtprc

I opted for email password managemnt with gpg, therefore created a secure folder in /tmp to write the my EMail account password in a textfile, encrypted it and copied to the home folder:

mkdir /tmp/pw
nano /tmp/pw/.msmtp-password
gpg --default-recipient-self --encrypt /tmp/pw/.msmtp-password1
cp /tmp/pw/.msmtp-password1.gpg ~/

* Then I had to set up my ~/.msmtprc file. For one email service provider I had to use untypical values like tls_fingerprint instead of the default certificates file, to get the finder print

msmtp --serverinfo --tls --tls-certcheck=off --host=webmail.mail1.org

The final file was then

defaults
auth           on
tls            on
logfile        ~/.msmtp.log

# mail1
account        mail1
host           webmail.mail1.org
port           465
tls_starttls   off
tls_fingerprint 54:C6:75:F3:CA:CC:4A:6A:5B:AE:AE:DD:51:F7:24:37:D7:6B:27:E1:B9:09:A9:DF:8F:59:63:BF:F5:58:CA:80
from           eugen@mail1.org
user           eugen@mail1.org
passwordeval   gpg2 --no-tty -q -d ~/.msmtp-password1.gpg

# mail2
account        mail2
host           smtp.mail2.net
port           587
tls_starttls   on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
from           eugen@mail2.net
user           eugen@mail2.net
passwordeval   gpg2 --no-tty -q -d ~/.msmtp-password2.gpg

# mail3
account        mail3
host           smtp.mail3.de
port           587
tls_starttls   on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
from           eugen@mail3.de
user           eugen@mail3.de
passwordeval   gpg2 --no-tty -q -d ~/.msmtp-password3.gpg

# Set a default account
account default : email1

* To test msmtp I used

echo "hello there username." | msmtp -a default eugen@mail1.org
cat testmail.txt | gpg -ea -r 'eugen-b' | msmtp -a default eugen@mail1.org

* Then I (found out that) I had to import the key of the buildmaster@archlinux32.org, I imported C963C94DD204443D

gpg --search-keys buildmaster@archlinux32.org

and set trust to ultimate (enter 'trust' and select ultimate)

gpg --edit-key C963C94DD204443D

* Then I tested sendmailadvanced, I created a textfile testmail-packages with

From: eugen@manjaro.org
To: buildmaster@archlinux32.org
Subject: Test-Betreff

stabilize: test-package-3.38.4-1

and edited /etc/sendmailadvanced.conf -> gpg_recipient="C963C94DD204443D" which is the buildserver. Then sent the test mail

sendmailadvanced -i testmail-packages

* You can check if your mail gets accepted at https://buildmaster.archlinux32.org/mail-log.html At current version if the mail is accepted there is no feedback which is not ideal.
* Now the functionality of

report-installed-packages eugen@mail1.org buildmaster@archlinux32.org

needs to be tested. I hope my guide "for Dummies" will help my fellow users to join the testing process!
* Edit: After pacman -Syyu (with [testing] and [community-testing] enabled) and some tests with my usual desktop workflow I ran the command

report-installed-packages eugen@mail1.org buildmaster@archlinux32.org

and got the reaction of the buildserver:

 Fri Dec 1 21:24:04 UTC 2017: Successfully marked 6 packages as tested.

This means that everything seems to work. Congratulations!

Last edited by eugen-b (2017-12-01 21:36:31)

Offline

#4 2017-12-03 20:36:52

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 275

Re: automatic testing via manual installation(s)

Thanks, eugen-b, for the guide smile
The buildmaster should now also emit "sucessfully stabilized 0 packages" messages, if no package was stabilized (but he was ordered to stabilize some).

Offline

#5 2018-03-06 19:49:56

jonathon
Member
Registered: 2017-07-19
Posts: 19

Re: automatic testing via manual installation(s)

Would/could this work for any manjaro32 systems? I'm trialling pulling packages from the testing repos into our "unstable" branch which is where testing happens our end, so if these systems could help with testing I can encourage that.

Offline

#6 2018-03-06 20:06:14

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 275

Re: automatic testing via manual installation(s)

it totally should work - the buildmaster will simply ignore any unknown packages - so as long as you actually test _our_ packages (and not recompiled ones), this would be really great smile

Offline

#7 2018-03-06 20:38:41

jonathon
Member
Registered: 2017-07-19
Posts: 19

Re: automatic testing via manual installation(s)

Excellent - I know there are a few packages which are in an "overlay" so excluding those would be handy... I might have to "fork" your script to do something like extract packager information to make sure you only get the right data. Sounds like a nice little programming project. smile

Offline

#8 2018-03-07 12:44:35

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 275

Re: automatic testing via manual installation(s)

or we could include sending the hash of the installed package - this would also avoid other errors ...

Offline

#9 2018-03-08 12:39:10

tyzoid
Administrator
From: Ann Arbor, MI
Registered: 2017-06-17
Posts: 89

Re: automatic testing via manual installation(s)

The hash would probably be nice, especially for AUR packages if they happen to have the same name.

Offline

Board footer

Powered by FluxBB