You are not logged in.

#1 2018-03-13 21:16:59

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

extra/firefox

Weirdly the FF59 PKGBUILD will build correctly under x86_64 without change, but under i686 it needs `valgrind` adding to makedepends otherwise pkg-config can't find it and configure bails.

makedepends+=('valgrind')

I don't understand why there's a difference between 32- and 64-bit here, but it might be a useful bit of info. smile

Last edited by jonathon (2018-03-13 21:18:13)

Offline

#2 2018-03-13 21:20:08

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

This was due to FS#57819 which should be fixed now.

Offline

#3 2018-03-13 21:23:32

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

Ah, cool. Carry on. smile

Offline

#4 2018-04-10 17:16:36

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

Apparently there are still people using systems with CPUs without SSE2 support (e.g. AthlonXP chips). Firefox ESR 52 works for them for the moment as upstream only switched to SSE2-only compiler options with FF53 (https://bugzilla.mozilla.org/show_bug.cgi?id=1274196). However, that isn't likely to be sustainable after August (ish) when Mozilla will bump to the Quantum-based ESR 60.

I've been doing some digging and the SSE2 change was introduced here: https://hg.mozilla.org/mozilla-central/ … ig.linux32

This led me to create the following patch for FF59 which I'm currently trying* to build:

+++ b/build/unix/mozconfig.linux32
@@ -3,16 +3,11 @@
 if [ -f /etc/redhat-release ]; then
 export PKG_CONFIG_LIBDIR=/usr/local/lib/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig
 fi
 
-export MOZ_LINUX_32_SSE2_STARTUP_ERROR=1
-
-CFLAGS="$CFLAGS -msse -msse2 -mfpmath=sse"
-CXXFLAGS="$CXXFLAGS -msse -msse2 -mfpmath=sse"
-
 if test `uname -m` = "x86_64"; then
-  CC="$CC -m32 -march=pentium-m"
-  CXX="$CXX -m32 -march=pentium-m"
+  CC="$CC -m32 -march=pentiumpro"
+  CXX="$CXX -m32 -march=pentiumpro"
   ac_add_options --target=i686-pc-linux
   ac_add_options --host=i686-pc-linux
   ac_add_options --x-libraries=/usr/lib
 fi

Assuming this works, I'm considering creating an AUR package and building for manjaro32 (possibly just a firefox-esr-sse, I don't think anyone running non-SSE2 CPUs would complain _too_ much about having to use the ESR version wink), or is this something you might want to include in the arch32 repos?

*  (apparently rustc "Cannot allocate memory" while building gkrust on machines with 16GB and 64GB RAM...)

Offline

#5 2018-04-11 04:16:09

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

Yes, we definitely want that in our repos (at some point[1]), but feel free, to create an AUR package first - that makes it a lot easier to import your changes smile

1] We're planning on introducing new architectures: i486 and pentium3, but I keep forgetting which should have which cpu features - "without sse2" will be either i486 or i686, I'm not sure which.

Offline

#6 2018-04-11 14:38:43

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

deep42thought wrote:

Yes, we definitely want that in our repos (at some point[1]), but feel free, to create an AUR package first - that makes it a lot easier to import your changes smile

1] We're planning on introducing new architectures: i486 and pentium3, but I keep forgetting which should have which cpu features - "without sse2" will be either i486 or i686, I'm not sure which.

Oh, sounds good!

I think technically i686 is non-SSE2 (Pentium Pro), whereas an "i786" would have SSE2 instructions? Not certain...

---

I eventually managed to get the package to build successfully. I've asked for some testing from people with pre-SSE2 CPUs to check the patch has actually had an effect.

The OOM errors I was having is a known issue (https://github.com/rust-lang/rust/issue … -380285099) with Rust and non-64-bit architectures, so if you hit the same thing there's a handy flag:

export RUSTFLAGS="-Cdebuginfo=0"

which vastly reduces memory use _and_ compilation time. Fedora use this on all 32-bit platforms.

Last edited by jonathon (2018-04-11 14:51:10)

Offline

#7 2018-05-20 22:36:38

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

Currently building 60.0.1 with GCC 8.1.0 fails due to the linker being unable to allocate memory. I eventually found a build configuration which works: disable gold and pass --no-keep-memory:

#ac_add_options --enable-gold
...
build() {
...
  LDFLAGS+=" -Wl,--no-keep-memory"
...

Last edited by jonathon (2018-05-21 08:47:49)

Offline

#8 2018-09-01 12:03:29

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

Linking should succeed with gold with

export LDFLAGS+=" -Wl,--no-keep-memory"

included outside of build() (i.e. as part of the normal "addition" PKGBUILD). I'll test again then open a PR.

Edit:

That may have been a fluke test result or it changed with FF62, but linking with ld.gold wouldn't succeed no matter the linker options. However, it will link without ld.gold and with the extra LDFLAGS.

Last edited by jonathon (2018-09-06 08:03:44)

Offline

#9 2018-09-17 09:28:14

maurer
Member
Registered: 2018-09-17
Posts: 1

Re: extra/firefox

Hi,

Does firefox now works on non-sse2 hardware?

Offline

#10 2018-09-17 09:30:21

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

maurer wrote:

Does firefox now works on non-sse2 hardware?

not yet - i486 is still on the rise.

Offline

#11 2018-09-18 18:26:11

Luffy
Member
Registered: 2018-05-23
Posts: 13

Re: extra/firefox

Palemoon SSE for Linux ... last update v27.9.4, (17 jul 2018)

https://forum.palemoon.org/viewtopic.ph … 80#p145717

Last edited by Luffy (2018-09-18 18:32:56)

Offline

#12 2019-01-21 00:50:07

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

Firefox 64.0.2 will build but it needs a switch from Clang to GCC. Something along the lines of:

--- PKGBUILD	2019-01-10 07:02:11.000000000 +0000
+++ PKGBUILD.i686	2019-01-21 00:41:45.234316208 +0000
@@ -54,13 +54,11 @@
 ac_add_options --enable-hardening
 ac_add_options --enable-optimize
 ac_add_options --enable-rust-simd
-ac_add_options --enable-lto
-export MOZ_PGO=1
-export CC=clang
-export CXX=clang++
-export AR=llvm-ar
-export NM=llvm-nm
-export RANLIB=llvm-ranlib
+export CC=gcc
+export CXX=g++
+export AR=gcc-ar
+export NM=gcc-nm
+export RANLIB=gcc-ranlib
 
 # Branding
 ac_add_options --enable-official-branding
@@ -101,11 +99,13 @@
   export MOZ_SOURCE_REPO="$_repo"
   export MOZ_NOSPAM=1
   export MOZBUILD_STATE_PATH="$srcdir/mozbuild"
+  export RUSTFLAGS="-Cdebuginfo=0 -Clto=off"
+  export LDFLAGS+=" -Wl,--no-keep-memory -Wl,--reduce-memory-overheads"
 
   # LTO needs more open files
   ulimit -n 4096
 
-  xvfb-run -a -n 97 -s "-screen 0 1600x1200x24" ./mach build
+  ./mach build
   ./mach buildsymbols
 }
 

I couldn't get LTO or PGO to work, the linking phase exceeds the 3GB memory limit. I also tried to get compilation working with Clang by adding `ac_add_option --enable=lld` to switch to ld.lld but that still takes too much RAM during linking. The `gkrust` crate is the only one which can't handle Rust's LTO, so disabling LTO within RUSTFLAGS is still necessary.

Interestingly, LTO with GCC might have worked but rather than running out of RAM the linker threw the error "/usr/sbin/ld: final link failed: No space left on device", so it might still work - or it might still run out of RAM after taking up a load of disk space...

---

Just for my own future reference:

ac_add_options --with-ccache=/usr/sbin/ccache
ac_add_options --enable-lld
-flto=thin -fuse-ld=lld

Last edited by jonathon (2019-01-21 01:28:21)

Offline

#13 2019-02-07 04:19:47

levi
Moderator
From: UK
Registered: 2018-06-16
Posts: 227

Re: extra/firefox

And as of a week ago last Tuesday we're even further behind current, with firefox 65.0 being released.  I can't see a build error for firefox, so I guess it's just not building automatically at the moment.  I may have to switch back to using one of the aur projects that pulls down mozilla's builds.

Offline

#14 2019-02-07 05:20:10

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

there are builds scheduled and failing
I didn't have a look into them yet, sry

regards,
deep42thought

Offline

#15 2019-02-07 07:17:57

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

Even installing the precompiled packages firefox-bin 65.0 and 64.0 are no help: they just result in an endless loop of starting itself over and over again.
So even if we manage to build the new versions, chances are, they will expose the same strange behaviour.

It's simply not acceptable that one company with Firefox and Rust libsrvg is creating 80% of the maintaince work for a Linux distribution IMHO.

Offline

#16 2019-02-07 16:51:11

levi
Moderator
From: UK
Registered: 2018-06-16
Posts: 227

Re: extra/firefox

deep42thought wrote:

there are builds scheduled and failing
I didn't have a look into them yet, sry

Well, thanks in advance for when you do.

FWIW, today I'm getting an HSTS key error because my browser claims the cert's only valid for archlinux32.org and www.the same when I try that link or go to packages.... manually.  It seemed to be working when I used it yesterday, so I'm not sure what's changed here.

I'll have to try out the prebuilt firefoxes on my machine and see if I can reproduce the reported errors

Offline

#17 2019-02-07 20:25:16

jonathon
Member
From: UK
Registered: 2017-07-19
Posts: 32
Website

Re: extra/firefox

The build failure just looks like memory exhaustion during linking (again)...

That's a bit odd because you're using the same set of flags which let me build successfully in my manjaro32 build container...

Could your build host be running out of RAM?

Offline

#18 2019-02-08 05:38:12

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

jonathon wrote:

Could your build host be running out of RAM?

That's possible for some, but not all build slaves.
E.g. nlopc46 and nlopc43 have 64 and 32GB memory respectively.
I can schedule firefox on one of those to make sure it was tried to build there.

Any other options I need to set on the host?

Offline

#19 2019-02-08 07:57:20

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

32-bit limits the amount of RAM per process to 3.x GB, so more memory might help, when building in parallel.
Which means for firefox: DON'T build in parallel. :-)
I still think, we had the chroot build method circumventing systemd-nspawn, maybe that helps?

Offline

#20 2019-02-10 09:10:31

deep42thought
Administrator
From: Jena, Germany
Registered: 2017-06-17
Posts: 365

Re: extra/firefox

I'll give it a try - but that machine does not have too much ram (it must be an i686 vm to allow for skipping systemd-nspawn) - andreas: do you have a slave running on i686 with 3-4GB RAM? Then you could try to build with "-s :without_systemd_nspawn:"

Offline

#21 2019-02-10 09:13:39

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

I'm trying that on the buildmaster machine. I'm building outside of our build system, if that works, then we may
find out, what exactly goes wrong. /tmp on tmpfs was a really bad idea, /tmp on a SSD with only 2 GB of disk space
left too. Now my 3rd run is using /tmp on the slow old platters. :-)

Offline

#22 2019-02-10 13:14:15

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

I don't think anything is working here: /tmp files are not used, LTO is still done,
and in parallel making matters worse.

So I cannot reproduce a successful build with the current patches in PKGBUILD:

252:05.90 toolkit/library/libxul.so

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                               
24627 abaumann  20   0 3699360 719468    660 D   2.0  23.4   0:02.11 lto1-wpa-stream
30886 abaumann  20   0 3699360 387472   1272 D   1.0  12.6   0:00.03 lto1-wpa-stream
20540 abaumann  20   0 3699360 379288      0 S   0.0  12.3   4:32.79 lto1-wpa

Offline

#23 2019-02-10 13:40:23

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

Trying a build now disabling also LTO in LDFLAGS and FLAGS, not just in Rust.

Offline

#24 2019-02-10 19:20:26

levi
Moderator
From: UK
Registered: 2018-06-16
Posts: 227

Re: extra/firefox

FWIW, I just tried installing the 65.0 binaries from mozilla using the non-Tim Song pkgbuild.  It seemed to have more shasums than I could really cope with (four different ones) so I just skipped them, but on successfully installing it and running it for a few hours, it just sat there allocating memory but never actually using it all the while spinning up one half of my processor (one hyperthreaded core) at pretty much 100%.  No logs even with verbose, and nothing in my logs either.  I'm not in the mood to debug it further, so I've switched back to 63.0 to post this.

Offline

#25 2019-02-11 06:50:32

andreas_baumann
Administrator
From: Zurich, Switzerland
Registered: 2017-08-10
Posts: 591
Website

Re: extra/firefox

Yes, if you strace the process then you see firefox-bin is restarting itself in an endless loop. This is for 0.64 and 0.65.
BTW: I managed to build a firefox by hand disabling more LTO stuff in mozconfig. I checked in and I hope it will hit
the build slaves soon.

Offline

Board footer

Powered by FluxBB