Stuck maxima builds on aarch64

[Jerry James]

I fixed the mass rebuild FTBFS errors for clisp and ecl yesterday, and
started a build of maxima due to changes in both clisp and ecl.
Something is afoot.

There are two previous builds for maxima that both claim to be in the
building state, but were started quite a long time ago:

19 Dec 2016: https://koji.fedoraproject.org/koji/buildinfo?buildID=826479
12 Jan 2017: https://koji.fedoraproject.org/koji/buildinfo?buildID=833415

Then there is my attempt from yesterday:

https://koji.fedoraproject.org/koji/buildinfo?buildID=861874

In all 3 cases, the build finished successfully on all architectures
except aarch64.  But I'm puzzled as to what is going on with my latest
build attempt.  The build log for aarch64 says, in its entirety:

Mock Version: 1.3.3
Mock Version: 1.3.3
ENTER ['do'](['bash', '--login', '-c',
'/usr/bin/rpmbuild -bs --target
aarch64 --nodeps /builddir/build/SPECS/maxima.spec'],
logger=<mockbuild.trace_decorator.getLog object at
0xffff7e8c3390>user='mockbuild'printOutput=Falseshell=Falseuid=1000env={'HOME':
'/builddir', 'PROMPT_COMMAND': 'printf
"\\033]0;<mock-chroot>\\007"',
'SHELL': '/bin/bash', 'PS1': '<mock-chroot> \\s-\\v\\$
', 'TERM':
'vt100', 'LANG': 'en_US.UTF-8', 'PATH':
'/usr/bin:/bin:/usr/sbin:/sbin', 'HOSTNAME':
'mock'}gid=425chrootPath='/var/lib/mock/f26-build-7778919-700214/root'timeout=172800)
Executing command: ['bash', '--login', '-c',
'/usr/bin/rpmbuild -bs
--target aarch64 --nodeps /builddir/build/SPECS/maxima.spec'] with env
{'HOME': '/builddir', 'PROMPT_COMMAND': 'printf
"\\033]0;<mock-chroot>\\007"', 'SHELL': '/bin/bash',
'PS1':
'<mock-chroot> \\s-\\v\\$ ', 'TERM': 'vt100',
'LANG': 'en_US.UTF-8',
'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'HOSTNAME':
'mock'} and shell
False
sh: ecl: command not found
Building target platforms: aarch64
Building for target aarch64
Wrote: /builddir/build/SRPMS/maxima-5.39.0-4.fc26.src.rpm
Child return code was: 0

So ... the source RPM was built successfully, even though ecl was not
found.  Then why wasn't the build launched?  Looking at root.log, it
looks like a dnf transaction has stalled.  The last lines are:

DEBUG util.py:435:  Total
7.7 MB/s |  32 MB     00:04
DEBUG util.py:435:  Running transaction check
DEBUG util.py:435:  Transaction check succeeded.
DEBUG util.py:435:  Running transaction test
DEBUG util.py:435:  Transaction test succeeded.
DEBUG util.py:435:  Running transaction

Except the transaction is not running.  I guess that "sh:ecl: command
not found" is due to this line in the spec file:

%global ecllib %(ecl -eval "(princ (SI:GET-LIBRARY-PATHNAME))" -eval
"(quit)")

but that doesn't seem to have been fatal for the SRPM building step,
and should evaluate just fine during the actual build.  So what's
going on?
-- 
Jerry James
http://www.jamezone.org/

[Jerry James]

On Sat, Feb 25, 2017 at 1:12 PM, Jerry James <loganjerry(a)gmail.com&gt; wrote:
>  I fixed the mass rebuild FTBFS errors for clisp and ecl yesterday,
> and
>  started a build of maxima due to changes in both clisp and ecl.
>  Something is afoot. 
Is there anybody here who can take a look at this?  The dnf
transaction is apparently still stuck on aarch64, 3 days later:

https://koji.fedoraproject.org/koji/taskinfo?taskID=18041410

Thank you,
-- 
Jerry James
http://www.jamezone.org/

[Kevin Fenzi]

On Mon, 27 Feb 2017 14:28:14 -0700
Jerry James <loganjerry(a)gmail.com&gt; wrote:

>  On Sat, Feb 25, 2017 at 1:12 PM, Jerry James
> <loganjerry(a)gmail.com&gt;
>  wrote:
>  > I fixed the mass rebuild FTBFS errors for clisp and ecl yesterday,
>  > and started a build of maxima due to changes in both clisp and ecl.
>  > Something is afoot.  
>  
>  Is there anybody here who can take a look at this?  The dnf
>  transaction is apparently still stuck on aarch64, 3 days later:
>  
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=18041410
>  
>  Thank you, 
Hey. I looked at this a few weeks ago... but I don't recall all the
details so I wanted to look more closely at it again before I replied. 

However, I have not had time since your orig email on saturday to do
so.

I'll try and look soon, but I'm afraid you will need some patience. 

kevin

[Jerry James]

On Mon, Feb 27, 2017 at 3:15 PM, Kevin Fenzi <kevin(a)scrye.com&gt; wrote:
>  Hey. I looked at this a few weeks ago... but I don't recall all
> the
>  details so I wanted to look more closely at it again before I replied.
>
>  However, I have not had time since your orig email on saturday to do
>  so.
>
>  I'll try and look soon, but I'm afraid you will need some patience. 
No worries.  I didn't realize the git branching was going on when I
sent that.  I will be patient.  Thanks, Kevin.
-- 
Jerry James
http://www.jamezone.org/

[Kevin Fenzi]

On Mon, 27 Feb 2017 21:02:40 -0700
Jerry James <loganjerry(a)gmail.com&gt; wrote:

>  On Mon, Feb 27, 2017 at 3:15 PM, Kevin Fenzi <kevin(a)scrye.com&gt;
> wrote:
>  > Hey. I looked at this a few weeks ago... but I don't recall all the
>  > details so I wanted to look more closely at it again before I
>  > replied.
>  >
>  > However, I have not had time since your orig email on saturday to do
>  > so.
>  >
>  > I'll try and look soon, but I'm afraid you will need some
>  > patience.  
>  
>  No worries.  I didn't realize the git branching was going on when I
>  sent that.  I will be patient.  Thanks, Kevin. 
Branching hasn't started yet... thats later today. ;) 

ok, on to the fun of maxima. 

So, on aarch64 (only) it never gets to the point of actually building. 
It gets stuck everytime in the dnf call that sets up the root: 

  ├─kojid,24165 /usr/sbin/kojid --fg --force-lock --verbose
  │   └─kojid,32174 /usr/sbin/kojid --fg --force-lock --verbose
  │       └─mock,32680 -tt /usr/libexec/mock/mock -r koji/f26-build-7778919-700214
--no-clean --target aarch64 ...
  │           └─dnf,32726 /usr/bin/dnf builddep --installroot
/var/lib/mock/f26-build-7778919-700214/root//var/lib/moc

in root.log we have a normal looking transaction: 

DEBUG util.py:435:  Transaction Summary
DEBUG util.py:435: 
================================================================================
DEBUG util.py:435:  Install  51 Packages
DEBUG util.py:435:  Total download size: 32 M
DEBUG util.py:435:  Installed size: 132 M
DEBUG util.py:435:  Downloading Packages:
DEBUG util.py:435: 
--------------------------------------------------------------------------------
DEBUG util.py:435:  Total                                           7.7 MB/s |  32 MB    
00:04     
DEBUG util.py:435:  Running transaction check
DEBUG util.py:435:  Transaction check succeeded.
DEBUG util.py:435:  Running transaction test
DEBUG util.py:435:  Transaction test succeeded.
DEBUG util.py:435:  Running transaction

In the dnf.rpm.log in the chroot we have: 

Feb 24 23:54:06 INFO Installed: ecl-16.1.3-1.fc26.aarch64
Feb 24 23:54:06 INFO Installed: desktop-file-utils-0.23-3.fc26.aarch64
Feb 24 23:54:08 INFO Installed: gcl-2.6.12-7.fc26.aarch64
...

Looking at the dnf process fd's: 

lrwx------. 1 root root 64 Feb 28 18:39 43 ->
/var/lib/mock/f26-build-7778919-700214/root/var/lib/dnf/history/history-2017-02-24.sqlite-journal
lrwx------. 1 root root 64 Feb 28 18:39 44 ->
/var/lib/mock/f26-build-7778919-700214/root/var/lib/rpm/.rpm.lock
lrwx------. 1 root root 64 Feb 28 18:39 45 ->
/var/lib/mock/f26-build-7778919-700214/root/var/lib/rpm/Supplementname
lrwx------. 1 root root 64 Feb 28 18:39 46 ->
/var/lib/mock/f26-build-7778919-700214/root/var/lib/rpm/Enhancename
lr-x------. 1 root root 64 Feb 28 18:39 47 ->
/var/lib/mock/f26-build-7778919-700214/root/var/cache/yum/build-ef45ecfc50ca7520/packages/time-1.7-52.fc26.aarch64.rpm
lr-x------. 1 root root 64 Feb 28 18:39 48 ->
/var/lib/mock/f26-build-7778919-700214/root/var/cache/yum/build-ef45ecfc50ca7520/packages/time-1.7-52.fc26.aarch64.rpm

So, something about the aarch64 'time' package? Nothing really leaps to
out at a quick glance, and wouldn't other packages hit this? and whats
pulling in the time package anyhow?

I'm not sure where to go from here, but thats the info I have... 

Happy to try and gather more. 

kevin

[Jerry James]

On Tue, Feb 28, 2017 at 11:48 AM, Kevin Fenzi <kevin(a)scrye.com&gt; wrote:
>  So, something about the aarch64 'time' package? Nothing
> really leaps to
>  out at a quick glance, and wouldn't other packages hit this? and whats
>  pulling in the time package anyhow? 
The maxima package BuildRequires: time, for what purpose I don't know.
It's not like the time package has unusual scriptlets, either:

postinstall scriptlet (using /bin/sh):
/sbin/install-info /usr/share/info/time.info.gz /usr/share/info/dir \
    --entry="* time: (time).        GNU time Utility" >/dev/null 2>&1
|| :
preuninstall scriptlet (using /bin/sh):
if [ "$1" = 0 ]; then
    /sbin/install-info --delete /usr/share/info/time.info.gz
/usr/share/info/dir \
    --entry="* time: (time).        GNU time Utility" >/dev/null 2>&1
|| :
fi

I'm really not sure what to check next.  If an aarch64 box for
packagers will be available in the not too distant future, I will try
to debug this.  Thanks for the information, Kevin.
-- 
Jerry James
http://www.jamezone.org/

[Jerry James]

On Tue, Feb 28, 2017 at 2:05 PM, Jerry James <loganjerry(a)gmail.com&gt; wrote:
>  I'm really not sure what to check next.  If an aarch64 box for
>  packagers will be available in the not too distant future, I will try
>  to debug this.  Thanks for the information, Kevin. 
This issue is now blocking a large update of sagemath and many of its
dependencies.  Sagemath invokes maxima during its build.  That
currently fails because the maxima in both F26 and Rawhide was built
with the pre-mass-rebuild version of ecl.  If this issue is not
resolved, we will ship a nonfunctional sagemath with F26.

I either need access to an aarch64 box so I can try to figure out the
problem, or I need somebody with access to the aarch64 koji builders
to figure it out.  I would file a bug, except I don't know which
component to file it against.  Probably rpm, but I don't know that for
sure.

Thank you,
-- 
Jerry James
http://www.jamezone.org/

[Jeremy Linton]

Hi,

On 03/21/2017 07:33 AM, Jerry James wrote:
>  On Tue, Feb 28, 2017 at 2:05 PM, Jerry James
> <loganjerry(a)gmail.com&gt; wrote:
> > I'm really not sure what to check next.  If an aarch64 box for
> > packagers will be available in the not too distant future, I will try
> > to debug this.  Thanks for the information, Kevin.
>
>  This issue is now blocking a large update of sagemath and many of its
>  dependencies.  Sagemath invokes maxima during its build.  That
>  currently fails because the maxima in both F26 and Rawhide was built
>  with the pre-mass-rebuild version of ecl.  If this issue is not
>  resolved, we will ship a nonfunctional sagemath with F26.
>
>  I either need access to an aarch64 box so I can try to figure out the
>  problem, or I need somebody with access to the aarch64 koji builders
>  to figure it out.  I would file a bug, except I don't know which
>  component to file it against.  Probably rpm, but I don't know that for
>  sure. 
I started a local build of this and it appears to be hanging like this:

make  check-TESTS
make[2]: Entering directory '/root/maxima/maxima-5.39.0/tests'
make[3]: Entering directory '/root/maxima/maxima-5.39.0/tests'
sed <"test.sh.in" >"gcl-test.sh"                                
         \
   -e 's#!LOCAL_MAXIMA!#/bin/sh ../maxima-local#'                       \
   -e 's#!LISPNAME!#gcl#'                                 \
   -e 's#!OUTPUT_LOG!#../tests/gcl.log#'
chmod a+x "gcl-test.sh"
FAIL: gcl-test.sh
sed <"test.sh.in" >"ecl-test.sh"                                
         \
   -e 's#!LOCAL_MAXIMA!#/bin/sh ../maxima-local#'                       \
   -e 's#!LISPNAME!#ecl#'                                 \
   -e 's#!OUTPUT_LOG!#../tests/ecl.log#'
chmod a+x "ecl-test.sh"


Breaking into maxima it seems to be stuck in:
(gdb) bt

#0  0x0000aaaaccdf2b3c in LC37testsuite (narg=0) at binary-ecl/mload.c:2800
#1  0x0000aaaaccdf2d8c in LC38__g239 (narg=0) at binary-ecl/mload.c:2967
#2  0x0000ffffb30532fc in L1do_time () from /lib64/libecl.so.16.1
#3  0x0000aaaaccdf2060 in L39run_testsuite (narg=<optimized out>) at 
binary-ecl/mload.c:2692
#4  0x0000ffffb3108f48 in cl_apply () from /lib64/libecl.so.16.1
#5  0x0000aaaaccdead24 in L44_run_testsuite (narg=<optimized out>) at 
binary-ecl/mload.c:3269
#6  0x0000ffffb3108f48 in cl_apply () from /lib64/libecl.so.16.1
#7  0x0000aaaaccc5c42c in L13meval1 (v1form=<optimized out>) at 
binary-ecl/mlisp.c:1146
#8  0x0000aaaaccc41b1c in L10meval (v1form=0xaaaaed4c6991) at 
binary-ecl/mlisp.c:631
#9  0x0000aaaaccdfc7d4 in L1meval_ (v1expr=<optimized out>) at 
binary-ecl/suprv1.c:30
#10 0x0000aaaaccde4ec8 in L7toplevel_macsyma_eval (v1x=<optimized out>) 
at binary-ecl/macsys.c:179
#11 0x0000aaaaccde85d4 in L9continue (narg=<optimized out>) at 
binary-ecl/macsys.c:369
#12 0x0000aaaaccde75b8 in L24macsyma_top_level (narg=<optimized out>) at 
binary-ecl/macsys.c:1154
#13 0x0000aaaacd3044c4 in L39common_lisp_user__run () at 
binary-ecl/init-cl.c:1355
#14 0x0000ffffb310b840 in ecl_interpret () from /lib64/libecl.so.16.1
#15 0x0000ffffb310f82c in eval_form () from /lib64/libecl.so.16.1
#16 0x0000ffffb310f8e0 in execute_each_form () from /lib64/libecl.so.16.1
#17 0x0000ffffb310ec04 in compile_form () from /lib64/libecl.so.16.1
#18 0x0000ffffb310f4f8 in compile_with_load_time_forms () from 
/lib64/libecl.so.16.1
#19 0x0000ffffb310f7d8 in eval_form () from /lib64/libecl.so.16.1
#20 0x0000ffffb31130d0 in si_eval_with_env () from /lib64/libecl.so.16.1
#21 0x0000ffffb30f1120 in si_safe_eval () from /lib64/libecl.so.16.1
#22 0x0000aaaaccbb9130 in main (argc=<optimized out>, argv=<optimized 
out>) at /tmp/eclinitVioCS0.c:1843

So, I'm not sure how helpful that is. I will spend a little more time 
looking at it.


In the mean time, linaro has instances for loan ( 
https://www.linaro.org/leg/servercluster/ ) That said, an aarch64 target 
in fedora/qemu on x86 isn't a great solution, but on a high end machine 
is actually fairly reasonable when compared with some of the low end 
SOCs (particularly with regard to IO).  So its possible to install 
fedora/aarch64 and do basic builds. I wouldn't try building kernels on a 
regular basis but it is doable.

[Kevin Fenzi]

On 03/21/2017 10:35 AM, Jeremy Linton wrote:
>  Hi,
>  
>  On 03/21/2017 07:33 AM, Jerry James wrote:
> > On Tue, Feb 28, 2017 at 2:05 PM, Jerry James <loganjerry(a)gmail.com&gt;
> > wrote:
> >> I'm really not sure what to check next.  If an aarch64 box for
> >> packagers will be available in the not too distant future, I will try
> >> to debug this.  Thanks for the information, Kevin.
> >
> > This issue is now blocking a large update of sagemath and many of its
> > dependencies.  Sagemath invokes maxima during its build.  That
> > currently fails because the maxima in both F26 and Rawhide was built
> > with the pre-mass-rebuild version of ecl.  If this issue is not
> > resolved, we will ship a nonfunctional sagemath with F26.
> >
> > I either need access to an aarch64 box so I can try to figure out the
> > problem, or I need somebody with access to the aarch64 koji builders
> > to figure it out.  I would file a bug, except I don't know which
> > component to file it against.  Probably rpm, but I don't know that for
> > sure.
>  
>  I started a local build of this and it appears to be hanging like this:
>  
>  make  check-TESTS 
...

Note that what I was seeing is that setting up the mock chroot to build
the package is where it was hanging.

So, if someone could try on a fedora 25 host to setup a rawhide/f26 mock
chroot and try and build maxima in it, and see if it hangs installing
the buildsys packages (in particular the 'time' package) that would be
great.

kevin

[Rex Dieter]

Kevin Fenzi wrote:

>  On 03/21/2017 10:35 AM, Jeremy Linton wrote:
> > Hi,
> > 
> > On 03/21/2017 07:33 AM, Jerry James wrote:
> >> On Tue, Feb 28, 2017 at 2:05 PM, Jerry James <loganjerry(a)gmail.com&gt;
> >> wrote:
> >>> I'm really not sure what to check next.  If an aarch64 box for
> >>> packagers will be available in the not too distant future, I will try
> >>> to debug this.  Thanks for the information, Kevin.
> >>
> >> This issue is now blocking a large update of sagemath and many of its
> >> dependencies.  Sagemath invokes maxima during its build.  That
> >> currently fails because the maxima in both F26 and Rawhide was built
> >> with the pre-mass-rebuild version of ecl.  If this issue is not
> >> resolved, we will ship a nonfunctional sagemath with F26.
> >>
> >> I either need access to an aarch64 box so I can try to figure out the
> >> problem, or I need somebody with access to the aarch64 koji builders
> >> to figure it out.  I would file a bug, except I don't know which
> >> component to file it against.  Probably rpm, but I don't know that for
> >> sure.
> > 
> > I started a local build of this and it appears to be hanging like this:
> > 
> > make  check-TESTS
>  ...
>  
>  Note that what I was seeing is that setting up the mock chroot to build
>  the package is where it was hanging.
>  
>  So, if someone could try on a fedora 25 host to setup a rawhide/f26 mock
>  chroot and try and build maxima in it, and see if it hangs installing
>  the buildsys packages (in particular the 'time' package) that would be
>  great. 
Taking a closer look, it appears that 'time' is no longer needed or used as 
far as I can tell.  I'll try removing it (as well as other possible 
tweaking) with some scratch builds, and report back if I find anything 
useful.

-- Rex

[Rex Dieter]

Rex Dieter wrote:

>  Kevin Fenzi wrote:
>  
> > On 03/21/2017 10:35 AM, Jeremy Linton wrote:
> >> Hi,
> >> 
> >> On 03/21/2017 07:33 AM, Jerry James wrote:
> >>> On Tue, Feb 28, 2017 at 2:05 PM, Jerry James <loganjerry(a)gmail.com&gt;
> >>> wrote:
> >>>> I'm really not sure what to check next.  If an aarch64 box for
> >>>> packagers will be available in the not too distant future, I will try
> >>>> to debug this.  Thanks for the information, Kevin.
> >>>
> >>> This issue is now blocking a large update of sagemath and many of its
> >>> dependencies.  Sagemath invokes maxima during its build.  That
> >>> currently fails because the maxima in both F26 and Rawhide was built
> >>> with the pre-mass-rebuild version of ecl.  If this issue is not
> >>> resolved, we will ship a nonfunctional sagemath with F26.
> >>>
> >>> I either need access to an aarch64 box so I can try to figure out the
> >>> problem, or I need somebody with access to the aarch64 koji builders
> >>> to figure it out.  I would file a bug, except I don't know which
> >>> component to file it against.  Probably rpm, but I don't know that for
> >>> sure.
> >> 
> >> I started a local build of this and it appears to be hanging like this:
> >> 
> >> make  check-TESTS
> > ...
> > 
> > Note that what I was seeing is that setting up the mock chroot to build
> > the package is where it was hanging.
> > 
> > So, if someone could try on a fedora 25 host to setup a rawhide/f26 mock
> > chroot and try and build maxima in it, and see if it hangs installing
> > the buildsys packages (in particular the 'time' package) that would be
> > great.
>  
>  Taking a closer look, it appears that 'time' is no longer needed or used
>  as
>  far as I can tell.  I'll try removing it (as well as other possible
>  tweaking) with some scratch builds, and report back if I find anything
>  useful. 
OK, I dropped both ecl and time from the buildroot, and it is still hanging 
populating the buildroot, filed bug for tracking purposes:

https://bugzilla.redhat.com/show_bug.cgi?id=1435395

I'm out of ideas, may end up having to drop aarch64 support (at least 
temporarily) if this isn't resolved soon.

-- Rex

[Jeffrey Bastian]

On Wed, Mar 22, 2017 at 01:10:22PM -0600, Kevin Fenzi wrote:
>  So, if someone could try on a fedora 25 host to setup a rawhide/f26
> mock
>  chroot and try and build maxima in it, and see if it hangs installing
>  the buildsys packages (in particular the 'time' package) that would be
>  great. 

I tried this: I installed Fedora 25 + updates (kernel
4.9.14-200.fc25.aarch64) on an AMD Seattle system, then used fedpkg to
build a src rpm for maxima, and finally tried to build it with mock.
But, mock is failing on rpm gpg keys:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
$ mock -r fedora-26-aarch64 --resultdir=/home/rpmbuild/mock \
       fedpkg/maxima/maxima-5.39.0-4.fc26.src.rpm
...
warning:
/var/lib/mock/fedora-26-aarch64/root/var/cache/dnf/fedora-f88a95d51c1e4379/packages/info-6.3-2.fc26.aarch64.rpm:
Header V3 RSA/SHA256 Signature, key ID 64dab85d: NOKEY
Importing GPG key 0x3B921D09:
 Userid     : "Fedora 26 Secondary (26)
<fedora-26-secondary(a)fedoraproject.org&gt;&quot;
 Fingerprint: 19AA 0442 2491 9109 8B3D 8035 4560 FD4D 3B92 1D09
 From       : /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-26-secondary
Key imported successfully
Import of key(s) didn't help, wrong key(s)?
Error: 


Public key for info-6.3-2.fc26.aarch64.rpm is not installedFailing package is:
info-6.3-2.fc26.aarch64
 GPG Keys are configured as:
file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-26-secondary
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If I try it again, it fails the same way with public keys, but for a
different package each time (info, bash, tar, etc.)

'mock -r fedora-26-aarch64 --init' fails the same way.

So I switched from F26 to rawhide
  mock -r fedora-rawhide-aarch64 ...

And now I can reproduce the hang right after it installs the last of the
build requirements.  strace on the dnf child process of mock shows it's
just waiting on a futex() call:
  ~]# strace -f -p 11494
  strace: Process 11494 attached
  futex(0xffff8c0836c0, FUTEX_WAIT, 2, NULL

It seems mock has issues with both F26 and rawhide!


Finally, I noticed that my system has mock-1.3.4 installed, but Koji
builders are running mock-1.3.3.  Can the Koji builders get the newer
mock installed?  Maybe that would help?

-- 
Jeff Bastian

[Jeffrey Bastian]

On Tue, Mar 21, 2017 at 11:35:55AM -0500, Jeremy Linton wrote:
>  Hi,
>  
>  On 03/21/2017 07:33 AM, Jerry James wrote:
>  > On Tue, Feb 28, 2017 at 2:05 PM, Jerry James <loganjerry(a)gmail.com&gt; wrote:
>  > > I'm really not sure what to check next.  If an aarch64 box for
>  > > packagers will be available in the not too distant future, I will try
>  > > to debug this.  Thanks for the information, Kevin.
>  > 
>  > This issue is now blocking a large update of sagemath and many of its
>  > dependencies.  Sagemath invokes maxima during its build.  That
>  > currently fails because the maxima in both F26 and Rawhide was built
>  > with the pre-mass-rebuild version of ecl.  If this issue is not
>  > resolved, we will ship a nonfunctional sagemath with F26.
>  > 
>  > I either need access to an aarch64 box so I can try to figure out the
>  > problem, or I need somebody with access to the aarch64 koji builders
>  > to figure it out.  I would file a bug, except I don't know which
>  > component to file it against.  Probably rpm, but I don't know that for
>  > sure.
>  
>  I started a local build of this and it appears to be hanging like this:
>  
>  make  check-TESTS
>  make[2]: Entering directory '/root/maxima/maxima-5.39.0/tests'
>  make[3]: Entering directory '/root/maxima/maxima-5.39.0/tests'
>  sed <"test.sh.in" >"gcl-test.sh"                               
>           \
>    -e 's#!LOCAL_MAXIMA!#/bin/sh ../maxima-local#'                       \
>    -e 's#!LISPNAME!#gcl#'                                 \
>    -e 's#!OUTPUT_LOG!#../tests/gcl.log#'
>  chmod a+x "gcl-test.sh"
>  FAIL: gcl-test.sh
>  sed <"test.sh.in" >"ecl-test.sh"                               
>           \
>    -e 's#!LOCAL_MAXIMA!#/bin/sh ../maxima-local#'                       \
>    -e 's#!LISPNAME!#ecl#'                                 \
>    -e 's#!OUTPUT_LOG!#../tests/ecl.log#'
>  chmod a+x "ecl-test.sh" 
I got to this point with a plain rpmbuild (i.e., without mock) and
noticed an SELinux AVC:

time->Thu Mar 23 17:06:04 2017 type=AVC msg=audit(1490303164.568:291): avc:  denied  {
execheap } for pid=25435 comm="maxima"
scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=process
permissive=0

Rebuilding the src rpm with SELinux in permissive mode (setenforce 0)
fixes this problem and allows rpmbuild to finish.

~]$ rpmbuild --rebuild maxima-5.39.0-4.fc26.src.rpm
...
Wrote: /home/rpmbuild/rpmbuild/RPMS/aarch64/maxima-5.39.0-4.fc25.aarch64.rpm
Wrote: /home/rpmbuild/rpmbuild/RPMS/aarch64/maxima-gui-5.39.0-4.fc25.aarch64.rpm
Wrote: /home/rpmbuild/rpmbuild/RPMS/aarch64/maxima-src-5.39.0-4.fc25.aarch64.rpm
Wrote: /home/rpmbuild/rpmbuild/RPMS/aarch64/maxima-runtime-gcl-5.39.0-4.fc25.aarch64.rpm
Wrote: /home/rpmbuild/rpmbuild/RPMS/aarch64/maxima-runtime-ecl-5.39.0-4.fc25.aarch64.rpm
...
+ exit 0

Note: it took a *long* time running the self-tests, but it did finish
eventually.  (Maybe it was never hanging in the first place and we were
just impatient?  It's possible the AVC is a red-herring.)


In any case, changing SELinux to permissive does not fix the original
problem where mock hangs setting up the buildroot environment...

-- 
Jeff Bastian