HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3)

[Joe Todaro]

Hi, 
I'm having a problem *not* being able to build 'i686' packages (i.e. 
optional_arches=i686) anymore *after* having upgraded our plague 
server/builder (Opteron x86_64) a couple of weeks ago from plague-0.4.3 to 
*plague-0.5.0*. The problem happens only with i686 (optional_arches=i686) 
-- not with i386 (base_arches=i386) which continues to work flawlessly.

PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION:
Here's what the Arches section looks like in all the 
/etc/plague/server/targets/*.cfg files:
  [Arches]
  base_arches=i386
  optional_arches=i686 noarch

PLAGUE-0.4.3 SERVER LOG: 
Here's an example of how things used to work (snipped from 
/var/log/plague-server.log) whenever an i686 package-build request was 
submitted to our PLAGUE-0.4.3 server:
  Request to enqueue 'e1000' tag 
'/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for 
target 'ao100' (user '[e-mail]')
  503 (e1000): Starting tag 
'/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on 
target 'lnxaddons-100-install'
  503 (e1000): Requesting depsolve...
  503 (e1000): Starting depsolve for arches: ['i686'].
  503 (e1000): Finished depsolve (successful), requesting archjobs.
  503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is 
d90078ec928db631ae8f590e6d5491d514cfe4a8
  503 (e1000/i686): Build result files - [ 'mockconfig.log', 'build.log', 
'root.log', 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 
'job.log', 'e1000-7.0.38-1_rhel4.src.rpm', 
'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ]
  Repo 'lnxaddons-100-install': updating repository metadata...
  503 (e1000): Job finished.

PLAGUE-0.5.0 SERVER LOG: 
But here's what happens (snipped from /var/log/plague-server.log) now, 
whenever the above i686 package-build request gets submitted to our 
"upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely 
nothing!)
  Request to enqueue 'e1000' tag 
'/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for 
target 'ao100' (user '[e-mail]')
  508 (e1000): Starting tag 
'/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on 
target 'lnxaddons-100-install'
  508 (e1000): Job finished.

OBSERVATIONS:
o The "last" function executed in the 'PackageJob.py' module (before it 
returned to 'BuildMaster.py') was 'arch_handling(self, ba, exclusive, 
exclude)'.
o Adding the following section to /etc/plague/server/targets/*.cfg 
(including server/builder restart, request resubmit) did *not* help 
'PackageJob.py' to progress any further than the 'arch_handling(self, ba, 
exclusive, exclude)' function.
   [Additional Package Arches]
   kernel=i686
o Moving 'i686' from the 'optional_arches' line up to the 'base_arches' 
line (including server/builder restart, request resubmit) *did* in fact 
cause 'i686' to be recognized by 'PackageJob.py' (but only as a "base 
arch" -- not as an "optional arch" like we need it to be)

MY QUESTIONS: 
1. Why is the *optional_arches* tag -- in the [Arches] section of our 
/etc/plague/server/*.cfg files -- *no longer* recognized *after* upgrading 
to plague-0.5.0? 
2. What are some things I can try, that might help resolve the above (i.e. 
getting *plague-0.5.0* to recognize 'i686' as an *optional arch*?)

Any help will be much appreciated! .. I have run out of ideas, and things 
to try... ;-(

Thanks,
--Joe

[Dan Williams]

On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> Hi,  
> I'm having a problem *not* being able to build 'i686' packages (i.e.
> optional_arches=i686) anymore *after* having upgraded our plague
> server/builder (Opteron x86_64) a couple of weeks ago from
> plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> (optional_arches=i686) -- not with i386 (base_arches=i386) which
> continues to work flawlessly.

Sounds like a bug; do the builder logs show anything, i.e. does the
server farm the job out to builders?  Note that as of Jun 21st, I
committed a fix to the builders that was causing stuff that needed
setarch to not work correctly, but that doesn't sound like this problem.

If the job doesn't go to the builder, there are some known issues where
plague will just not farm the job out to anyone if the builder config
files aren't just right.  The builder doesn't think it supports a
particular target and so of course the server won't try to build on that
target.  Try double-checking your plague-builder targets config files
and make _sure_ they support every architecture you want to build on
(i386, i686, i586, etc).

I'll see if I can look a bit more over the weekend.

Dan
> PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION: 
> Here's what the Arches section looks like in all
> the /etc/plague/server/targets/*.cfg files: 
>   [Arches] 
>   base_arches=i386 
>   optional_arches=i686 noarch
>
> PLAGUE-0.4.3 SERVER LOG:  
> Here's an example of how things used to work (snipped
> from /var/log/plague-server.log) whenever an i686 package-build
> request was submitted to our PLAGUE-0.4.3 server: 
>   Request to enqueue 'e1000' tag
> '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for
> target 'ao100' (user '[e-mail]') 
>   503 (e1000): Starting tag
> '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on
> target 'lnxaddons-100-install' 
>   503 (e1000): Requesting depsolve... 
>   503 (e1000): Starting depsolve for arches: ['i686']. 
>   503 (e1000): Finished depsolve (successful), requesting archjobs. 
>   503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is
> d90078ec928db631ae8f590e6d5491d514cfe4a8 
>   503 (e1000/i686): Build result files - [ 'mockconfig.log',
> 'build.log', 'root.log',
> 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 'job.log',
> 'e1000-7.0.38-1_rhel4.src.rpm',
> 'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ] 
>   Repo 'lnxaddons-100-install': updating repository metadata... 
>   503 (e1000): Job finished.
>
> PLAGUE-0.5.0 SERVER LOG:  
> But here's what happens (snipped from /var/log/plague-server.log) now,
> whenever the above i686 package-build request gets submitted to our
> "upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely
> nothing!) 
>   Request to enqueue 'e1000' tag
> '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for
> target 'ao100' (user '[e-mail]') 
>   508 (e1000): Starting tag
> '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on
> target 'lnxaddons-100-install' 
>   508 (e1000): Job finished.
>
> OBSERVATIONS: 
> o The "last" function executed in the 'PackageJob.py' module (before
> it returned to 'BuildMaster.py') was 'arch_handling(self, ba,
> exclusive, exclude)'. 
> o Adding the following section to /etc/plague/server/targets/*.cfg
> (including server/builder restart, request resubmit) did *not* help
> 'PackageJob.py' to progress any further than the 'arch_handling(self,
> ba, exclusive, exclude)' function. 
>    [Additional Package Arches] 
>    kernel=i686 
> o Moving 'i686' from the 'optional_arches' line up to the
> 'base_arches' line (including server/builder restart, request
> resubmit) *did* in fact cause 'i686' to be recognized by
> 'PackageJob.py' (but only as a "base arch" -- not as an "optional
> arch" like we need it to be)
>
> MY QUESTIONS:
> Why is the *optional_arches* tag -- in the [Arches] section of
>
> our /etc/plague/server/*.cfg files -- *no longer* recognized *after*
> upgrading to plague-0.5.0?  
> 2. What are some things I can try, that might help resolve the above
> (i.e. getting *plague-0.5.0* to recognize 'i686' as an *optional
> arch*?)
>
> Any help will be much appreciated! .. I have run out of ideas, and
> things to try... ;-(
>
> Thanks,
> --Joe
>
> Fedora-buildsys-list mailing list
> [e-mail]
> https://www.redhat.com/mailman/listinfo/fedora-buildsys-list

[Joe Todaro]

Thank You, Dan.  My answers are below...
> Dan Williams  wrote on 07/06/2006 09:59:33 AM:
> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > Hi, 
> > I'm having a problem *not* being able to build 'i686' packages (i.e.
> > optional_arches=i686) anymore *after* having upgraded our plague
> > server/builder (Opteron x86_64) a couple of weeks ago from
> > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > continues to work flawlessly.
>
> Sounds like a bug; do the builder logs show anything, i.e. does the
> server farm the job out to builders?

The builder log shows nothing--the job is definitely *not* farmed out to 
the builder.
> Note that as of Jun 21st, I
> committed a fix to the builders that was causing stuff that needed
> setarch to not work correctly, but that doesn't sound like this problem.

I know--I installed your builder/BuilderMock.py updates on June 28th 
hoping that might help.

Also, a few days ago I upgraded Mock to the latest version, mock-0.6-1, 
hoping that might help.

I also forgot to mention sooner that we only use Passive Buliders -- not 
Active Builders.
> If the job doesn't go to the builder, there are some known issues where
> plague will just not farm the job out to anyone if the builder config
> files aren't just right.  The builder doesn't think it supports a
> particular target and so of course the server won't try to build on that
> target.  Try double-checking your plague-builder targets config files
> and make _sure_ they support every architecture you want to build on
> (i386, i686, i586, etc).

Hopefully these next five sections will help shed more light... 
- - - - -
(1 of 5) Here's what our /etc/plague/builder/plague-builder.cfg file looks 
like (with hostname/server changes to protect the innocent--that's me;):
- - - - -
[Active]
fileserver_port = 8889
xmlrpc_port = 8888

[Passive]
fileserver_port = 8889
xmlrpc_port = 8888

[Directories]
target_configs_dir = /etc/plague/builder/targets
builder_work_dir = /build/builder_work

[SSL]
builder_key_and_cert_dir = /etc/plague/builder/certs
use_ssl = yes
ca_cert = /etc/plague/builder/certs/ca_cert.pem

[General]
builder_cmd = /usr/bin/mock
builder_user = plague-builder
hostname = lnxbuild1.abc.com
server = lnxbuild1.abc.com
debug = yes
comm_type = passive

- - - - -
(2 of 5) Here's what our 
/etc/plague/builder/targets/lnxaddons-100-i386-install.cfg file looks like 
(with distro change to protect the innocent):
- - - - -
[General]
distro=lnxaddons
target=100
basearch=i386
repo=install
mock_config=lnxaddons-100-i386-install

- - - - -
(3 of 5) Here's what our /etc/mock/lnxaddons-100-i386-install.cfg file 
looks like (with baseurl changes to protect the innocent):
- - - - -
#!/usr/bin/python -tt

import os

config_opts['root'] = 'lnxaddons-100-i386'
config_opts['basedir'] = '/var/lib/mock/'
config_opts['chroot'] = '/usr/sbin/mock-helper chroot'
config_opts['mount'] = '/usr/sbin/mock-helper mount'
config_opts['umount'] = '/usr/sbin/mock-helper umount'
config_opts['rm'] = '/usr/sbin/mock-helper rm'
config_opts['mknod'] = '/usr/sbin/mock-helper mknod'
config_opts['yum'] = '/usr/sbin/mock-helper yum'
config_opts['runuser'] = '/sbin/runuser'
config_opts['buildgroup'] = 'build'
config_opts['chrootuser'] = 'aobuild'
config_opts['chrootgroup'] = 'aobuild'
config_opts['chrootuid'] = os.geteuid()
config_opts['chrootgid'] = os.getegid()
config_opts['chroothome'] = '/builddir'
config_opts['clean'] = True
config_opts['target_arch'] = 'i386'

config_opts['yum.conf'] = """
[main]
cachedir=/var/cache/yum
debuglevel=1
logfile=/var/log/yum.log
reposdir=/dev/null
retries=20
obsoletes=1
gpgcheck=0
assumeyes=1

# repos

[install]
name=install
baseurl=http://.../rhel4/i386/install/

[updates]
name=updates
baseurl=http://.../rhel4/updates/

[updates-rhel]
name=updates-rhel
baseurl=http://.../rhel4/updates/

[groups]
name=groups
baseurl=file:///build/buildgroups/i386/

[plague]
name=plague
baseurl=file:///build/yum/lnxaddons-100-install/i386/

"""
- - - - -
(4 of 5) Here's the e-mail that plague sends to my inbox each time I try 
to build the i686 (optional arch) e1000 package:
- - - - -
Subject: Prep Error (Job 618): 
/afs/.../SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm on lnxaddons-100-install
Package /afs/.../SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm does not build 
on any architectures this build system supports.
    Package: ['i686']
    Build System: ['i386', 'i386']

- - - - -
(5 of 5) Here are all the arches our builders think they support--i got 
this info by clicking on the [Builder Status] button of the web interface:

- - - - -

lnxbuild1.abc.com  (0/1)
x86_64, amd64, ia32e, noarch, i386, i486, i586, i686, athlon

lnxppc.abc.com  (0/8)
ppc64, noarch
> I'll see if I can look a bit more over the weekend.
>
> Dan
> > PLAGUE-0.4.3 / PLAGUE 0.5.0 ARCHES CONFIGURATION: 
> > Here's what the Arches section looks like in all
> > the /etc/plague/server/targets/*.cfg files: 
> >   [Arches] 
> >   base_arches=i386 
> >   optional_arches=i686 noarch
> >
> > PLAGUE-0.4.3 SERVER LOG: 
> > Here's an example of how things used to work (snipped
> > from /var/log/plague-server.log) whenever an i686 package-build
> > request was submitted to our PLAGUE-0.4.3 server: 
> >   Request to enqueue 'e1000' tag
> > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for
> > target 'ao100' (user '[e-mail]') 
> >   503 (e1000): Starting tag
> > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on
> > target 'lnxaddons-100-install' 
> >   503 (e1000): Requesting depsolve... 
> >   503 (e1000): Starting depsolve for arches: ['i686']. 
> >   503 (e1000): Finished depsolve (successful), requesting archjobs. 
> >   503 (e1000/i686): https://lnxbuild1.abc.com:8888 - UID is
> > d90078ec928db631ae8f590e6d5491d514cfe4a8 
> >   503 (e1000/i686): Build result files - [ 'mockconfig.log',
> > 'build.log', 'root.log',
> > 'kernel-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm', 'job.log',
> > 'e1000-7.0.38-1_rhel4.src.rpm',
> > 'kernel-smp-module-e1000-7.0.38-1.6.9_34.EL_2_rhel4.i686.rpm' ] 
> >   Repo 'lnxaddons-100-install': updating repository metadata... 
> >   503 (e1000): Job finished.
> >
> > PLAGUE-0.5.0 SERVER LOG: 
> > But here's what happens (snipped from /var/log/plague-server.log) now,
> > whenever the above i686 package-build request gets submitted to our
> > "upgraded" plague server/builder running PLAGUE-0.5.0 (absolutely
> > nothing!) 
> >   Request to enqueue 'e1000' tag
> > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' for
> > target 'ao100' (user '[e-mail]') 
> >   508 (e1000): Starting tag
> > '/afs/pok/projects/devel/SRPMS/e1000/e1000-7.0.38-1_rhel4.src.rpm' on
> > target 'lnxaddons-100-install' 
> >   508 (e1000): Job finished.
> >
> > OBSERVATIONS: 
> > o The "last" function executed in the 'PackageJob.py' module (before
> > it returned to 'BuildMaster.py') was 'arch_handling(self, ba,
> > exclusive, exclude)'. 
> > o Adding the following section to /etc/plague/server/targets/*.cfg
> > (including server/builder restart, request resubmit) did *not* help
> > 'PackageJob.py' to progress any further than the 'arch_handling(self,
> > ba, exclusive, exclude)' function. 
> >    [Additional Package Arches] 
> >    kernel=i686 
> > o Moving 'i686' from the 'optional_arches' line up to the
> > 'base_arches' line (including server/builder restart, request
> > resubmit) *did* in fact cause 'i686' to be recognized by
> > 'PackageJob.py' (but only as a "base arch" -- not as an "optional
> > arch" like we need it to be)
> >
> > MY QUESTIONS:
> > Why is the *optional_arches* tag -- in the [Arches] section of
> >
> > our /etc/plague/server/*.cfg files -- *no longer* recognized *after*
> > upgrading to plague-0.5.0? 
> > 2. What are some things I can try, that might help resolve the above
> > (i.e. getting *plague-0.5.0* to recognize 'i686' as an *optional
> > arch*?)
> >
> > Any help will be much appreciated! .. I have run out of ideas, and
> > things to try... ;-(
> >
> > Thanks,
> > --Joe
> >
> > Fedora-buildsys-list mailing list
> > [e-mail]
> > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list

-Joe

[Dan Williams]

On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> Hi,  
> I'm having a problem *not* being able to build 'i686' packages (i.e.
> optional_arches=i686) anymore *after* having upgraded our plague
> server/builder (Opteron x86_64) a couple of weeks ago from
> plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> (optional_arches=i686) -- not with i386 (base_arches=i386) which
> continues to work flawlessly.

Found and fixed in CVS HEAD.  Patch attached that should apply against
an installed copy of plague-server as well.  Thanks for the report, and
sorry for the lag.

Cheers,
Dan

[Joe Todaro]

Thanks Dan.

So I ran a quick regression test, just to make sure that *i386* is still 
working, but, it *failed* ... (also note that i686 gave basically the same 
errors..)

SEE BELOW for *regression* test results, both BEFORE and AFTER applying 
patch 'plague-opt-arches-fix.patch' to server/Config.py.  I repeated that 
test sequence several times, by the way.  Also, I was surprised to see 
that 'WhoAmI' was trying to build as an 'i686' rpm rather than a 'noarch' 
which is what it really is. So, since I can't be exactly sure at this 
point how the "optional_arches" design is supposed to tie-in with the 
"base_arches" design, it doesn't make much sense for me (with my limited 
Python skills) to try and dig in any further..

Also, could you please explain/elaborate when you say: "Patch attached 
that should apply against an installed copy of plague-server as well." 
Um, your patch is clearly intended to update file server/Config.py file, 
so I really don't understand how I can also use *it* to update file 
*plague-server* (/usr/bin/plague-server?) as well, which appears to be a 
start-up script -- am I missing something here?

For the good news (for me anyway), I'm also trying to learn Python (Diving 
Into Python) on my own time, so that hopefully one day I might be able 
help squash some of these bugs.. However, in the meantime, I/we very much 
appreciate your help and expertise..

REGRESSION TEST RESULTS ARE BELOW...

Dan Williams  wrote on 07/15/2006 01:20:54 PM:
> On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > Hi, 
> > I'm having a problem *not* being able to build 'i686' packages (i.e.
> > optional_arches=i686) anymore *after* having upgraded our plague
> > server/builder (Opteron x86_64) a couple of weeks ago from
> > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > continues to work flawlessly.
>
> Found and fixed in CVS HEAD.  Patch attached that should apply against
> an installed copy of plague-server as well.  Thanks for the report, and
> sorry for the lag.
>
> Cheers,
> Dan
>
> [attachment "plague-opt-arches-fix.patch" deleted by Joe 
> Todaro/Poughkeepsie/IBM]

REGRESSION TEST RESULTS:

===========================================================================================
*WITHOUT* THE OPTIONAL-ARCH FIX TO CONFIG.PY:
===========================================================================================
[root@lnxbuild1 ]$ cat /var/log/plague-server.log
Using database engine mysql.

Authorized Builders:
------------------------------------------------------------------------------------------
  https://lnxbuild1.abc.com:8888 unavailable
  https://lnxppc.abc.com:8888 unavailable

Build Server accepting requests on lnxbuild1.abc.com:8887.

Re-activating builder 'https://lnxbuild1.abc.com:8888'.
Re-activating builder 'https://lnxppc.abc.com:8888'.
Request to enqueue 'WhoAmI' tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for 
target 'oc100' (user '[e-mail]')
625 (WhoAmI): Starting tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 
'lnxaddons-100-install'
625 (WhoAmI): Requesting depsolve...
625 (WhoAmI): Starting depsolve for arches: ['i386'].
625 (WhoAmI): Finished depsolve (successful), requesting archjobs.
625 (WhoAmI/noarch): https://lnxbuild1.abc.com:8888 - UID is 
f2e2220b50c7707b7f224f18455f39fa088c5a22
625 (WhoAmI/noarch): Build result files - [ 
'WhoAmI-4.00-9_rhel4.noarch.rpm', 'mockconfig.log', 'build.log', 
'root.log', 'WhoAmI-4.00-9_rhel4.src.rpm', 'job.log' ]
Repo 'lnxaddons-100-install': updating repository metadata...
Repo 'lnxaddons-100-install': Done updating.
625 (WhoAmI): Job finished.
Received SIGTERM, quitting...
Shutting down...
Repo (lnxaddons-100-install): shut down.
Repo (lnxaddons-350-install): shut down.
Repo (lnxaddons-100-unsupported): shut down.
Done.

===========================================================================================
*WITH* THE OPTIONAL-ARCH FIX TO CONFIG.PY:
===========================================================================================
[root@lnxbuild1 ]$ cat /var/log/plague-server.log
Using database engine mysql.

Authorized Builders:
------------------------------------------------------------------------------------------
  https://lnxbuild1.abc.com:8888 unavailable
  https://lnxppc.abc.com:8888 unavailable

Build Server accepting requests on lnxbuild1.abc.com:8887.

Re-activating builder 'https://lnxbuild1.abc.com:8888'.
Re-activating builder 'https://lnxppc.abc.com:8888'.
Request to enqueue 'WhoAmI' tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for 
target 'oc100' (user '[e-mail]')
626 (WhoAmI): Starting tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 
'lnxaddons-100-install'
626 (WhoAmI): Requesting depsolve...
626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686', 'noarch'].
626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch i686.
Exception in thread PackageJob: 626/WhoAmI:
Traceback (most recent call last):
  File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap
    self.run()
  File "/usr/share/plague/server/PackageJob.py", line 86, in run
    self._pkg_job.process()
  File "/usr/share/plague/server/PackageJob.py", line 719, in process
    if func():
  File "/usr/share/plague/server/PackageJob.py", line 587, in 
_stage_depsolve
    if self._arch_deps_solved(arch) == False:
  File "/usr/share/plague/server/PackageJob.py", line 540, in 
_arch_deps_solved
    if base:
UnboundLocalError: local variable 'base' referenced before assignment

Thanks again,
-Joe

[Dan Williams]

On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote:
> Thanks Dan.
>
> So I ran a quick regression test, just to make sure that *i386* is
> still working, but, it *failed* ... (also note that i686 gave
> basically the same errors..)
>
> SEE BELOW for *regression* test results, both BEFORE and AFTER
> applying patch 'plague-opt-arches-fix.patch' to server/Config.py.  I
> repeated that test sequence several times, by the way.  Also, I was
> surprised to see that 'WhoAmI' was trying to build as an 'i686' rpm
> rather than a 'noarch' which is what it really is. So, since I can't

Hmm... Building an i386 package worked for me when I tested it with
current CVS HEAD, as did an i686 package.
> be exactly sure at this point how the "optional_arches" design is
> supposed to tie-in with the  "base_arches" design, it doesn't make
> much sense for me (with my limited Python skills) to try and dig in
> any further..

Ok; most distributions have a "base" architecture for any given
architecture family.  For RHEL & FC, that's 'i386' for x86 processors,
and for POWER, that's 'ppc'.  We only build specific RPMs as i686 since
not every RPM requires it, and building many RPMs for i686 can actually
_hurt_ performance.  In the same vein, we don't build both 'em64t' and
x86-64 packages for everything either, because the difference between
EM64T and x86-64 only matters for stuff like the kernel.

Given a set of packages, you _don't_ want to build every package on
every architecture the buildsystem supports, unless the package
explicitly asks for it.  Given a specfile with _no_ ExcludeArch or
IncludeArch, plague will build that package only for the "base"
architecture, in this case i386.

So what happens when you want a package to explicitly build on i686 or
i586 or em64t?  In this case, the specfile for the package must
explicitly request to be built as such, using ExclusiveArch.

Coming back to the original question, optional_arches gives buildsystem
administrators a filter on arches to build.  For example, Fedora Extras
doesn't necessarily want to allow packages to be built for i486, even if
the RPM requests it, because we don't support i486.  So if you add i686
to 'optional_arches', then packages that request to be built for i686
with ExclusiveArch will be allowed to build for i686, otherwise not.
> Also, could you please explain/elaborate when you say: "Patch attached
> that should apply against an installed copy of plague-server as
> well."    
> Um, your patch is clearly intended to update file server/Config.py
> file, so I really don't understand how I can also use *it* to update
> file *plague-server* (/usr/bin/plague-server?) as well, which appears
> to be a start-up script -- am I missing something here?

I had meant to change the paths such that you could apply the patch
from / using patch -p0 or such; I did that and then re-diffed and likely
forgot to update the patch bits again.  Since the 'cvs diff' is rooted
at a different directory than the installed plague server directory, I
was trying to make life simpler for you.  My bad.
> For the good news (for me anyway), I'm also trying to learn Python
> (Diving Into Python) on my own time, so that hopefully one day I might
> be able help squash some of these bugs.. However, in the meantime,
> I/we very much appreciate your help and expertise..
>
> REGRESSION TEST RESULTS ARE BELOW...
>
> Dan Williams  wrote on 07/15/2006 01:20:54 PM:
> > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > > Hi,  
> > > I'm having a problem *not* being able to build 'i686' packages
>
> (i.e.
> > > optional_arches=i686) anymore *after* having upgraded our plague
> > > server/builder (Opteron x86_64) a couple of weeks ago from
> > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > > continues to work flawlessly.
> >
> > Found and fixed in CVS HEAD.  Patch attached that should apply
>
> against
> > an installed copy of plague-server as well.  Thanks for the report,
>
> and
> > sorry for the lag.
> >
> > Cheers,
> > Dan
> >
> > [attachment "plague-opt-arches-fix.patch" deleted by Joe 
> > Todaro/Poughkeepsie/IBM]
>
> 626 (WhoAmI): Requesting depsolve... 
> 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686',
> 'noarch'].

Ok; what do you have in the 'optional_arches' for this target?

What do the top bits of the specfile for WhoAmI look like?  i.e., are
there any of the following specfile tags, and what are their values?

BuildArch
ExclusiveArch
ExcludeArch
> 626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch
> i686. 
> Exception in thread PackageJob: 626/WhoAmI: 
> Traceback (most recent call last): 
>   File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap 
>     self.run() 
>   File "/usr/share/plague/server/PackageJob.py", line 86, in run 
>     self._pkg_job.process() 
>   File "/usr/share/plague/server/PackageJob.py", line 719, in process 
>     if func(): 
>   File "/usr/share/plague/server/PackageJob.py", line 587, in
> _stage_depsolve 
>     if self._arch_deps_solved(arch) == False: 
>   File "/usr/share/plague/server/PackageJob.py", line 540, in
> _arch_deps_solved 
>     if base: 
> UnboundLocalError: local variable 'base' referenced before assignment

To get rid of this error, remove the marked line around line 490 of
PackageJob.py:

base = yum.YumBase()
            yum_config = self._write_yum_conf(arch)
            if not yum_config:
-               del base
                raise DepError("WARNING: bad yum config for arch %s." % arch)

depsolve_root = os.path.dirname(yum_config) + '/'
            base.doConfigSetup(fn=yum_config, root=depsolve_root)

That's a bug.  Hope we can get this fixed.

Dan
> Thanks again, 
> -Joe

[Joe Todaro]

Dan Williams  wrote on 07/16/2006 10:41:55 AM:
> On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote:
> > Thanks Dan.
> >
> > So I ran a quick regression test, just to make sure that *i386* is
> > still working, but, it *failed* ... (also note that i686 gave
> > basically the same errors..)
> >
> > SEE BELOW for *regression* test results, both BEFORE and AFTER
> > applying patch 'plague-opt-arches-fix.patch' to server/Config.py.  I
> > repeated that test sequence several times, by the way.  Also, I was
> > surprised to see that 'WhoAmI' was trying to build as an 'i686' rpm
> > rather than a 'noarch' which is what it really is. So, since I can't
>
> Hmm... Building an i386 package worked for me when I tested it with
> current CVS HEAD, as did an i686 package.

Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm -ivh) 
might help? 
If so, are there any "gotchas" I need to be aware of?
But hopefully this won't be necessary.
> > be exactly sure at this point how the "optional_arches" design is
> > supposed to tie-in with the  "base_arches" design, it doesn't make
> > much sense for me (with my limited Python skills) to try and dig in
> > any further..
>
> Ok; most distributions have a "base" architecture for any given
> architecture family.  For RHEL & FC, that's 'i386' for x86 processors,
> and for POWER, that's 'ppc'.  We only build specific RPMs as i686 since
> not every RPM requires it, and building many RPMs for i686 can actually
> _hurt_ performance.  In the same vein, we don't build both 'em64t' and
> x86-64 packages for everything either, because the difference between
> EM64T and x86-64 only matters for stuff like the kernel.

That was my understanding.
> Given a set of packages, you _don't_ want to build every package on
> every architecture the buildsystem supports, unless the package
> explicitly asks for it.  Given a specfile with _no_ ExcludeArch or
> IncludeArch, plague will build that package only for the "base"
> architecture, in this case i386.

That was my understanding too.
> So what happens when you want a package to explicitly build on i686 or
> i586 or em64t?  In this case, the specfile for the package must
> explicitly request to be built as such, using ExclusiveArch.

Ok, now I'm clear on this.

But also I have a question: Since i686 is a "subarch" of i386, do I need
to create any target config files specicially for i686, such as in the
following directories:
- /etc/plague/server/targets
- /etc/plague/builder/targets
- /etc/mock 
?
> Coming back to the original question, optional_arches gives buildsystem
> administrators a filter on arches to build.  For example, Fedora Extras
> doesn't necessarily want to allow packages to be built for i486, even if
> the RPM requests it, because we don't support i486.  So if you add i686
> to 'optional_arches', then packages that request to be built for i686
> with ExclusiveArch will be allowed to build for i686, otherwise not.

I'm also really clear on this, too, now.  Well, at least this proves, to 
me anyway, that I've had a pretty decent understanding of how things work
all along -- thanks to the great description you gave above, of course.
> > Also, could you please explain/elaborate when you say: "Patch attached
> > that should apply against an installed copy of plague-server as
> > well." 
> > Um, your patch is clearly intended to update file server/Config.py
> > file, so I really don't understand how I can also use *it* to update
> > file *plague-server* (/usr/bin/plague-server?) as well, which appears
> > to be a start-up script -- am I missing something here?
>
> I had meant to change the paths such that you could apply the patch
> from / using patch -p0 or such; I did that and then re-diffed and likely
> forgot to update the patch bits again.  Since the 'cvs diff' is rooted
> at a different directory than the installed plague server directory, I
> was trying to make life simpler for you.  My bad.

Oh, cool, I've never done that before - thanks.

Regardless, though, I still don't see any variable in 
/usr/bin/plague-server
that has *anything* to with "optional arches" or "base arches", which is 
what the patch was for right?

Anyway, here's a snippet from /usr/bin/plague-server script, which is the 
one I've been assuming that you're referring to - is this the one?

===================================
Excerpt from /usr/bin/plague-server
===================================

------------<snip>------------ 
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, 
USA.
#
# Copyright 2005 Dan Williams  and Red Hat, Inc.

import sys
import os
import socket
import signal
import time
from plague import AuthedXMLRPCServer
from plague import HTTPServer
from plague import daemonize
from plague import DebugUtils
import SimpleXMLRPCServer
from optparse import OptionParser
import threading

sys.path.append('/usr/share/plague/server')

import User
import BuildMaster
import BuilderManager
import DBManager
import Config
from UserInterface import UserInterfaceSSLAuth
from UserInterface import UserInterfaceNoAuth
-----------</snip>------------
> > For the good news (for me anyway), I'm also trying to learn Python
> > (Diving Into Python) on my own time, so that hopefully one day I might
> > be able help squash some of these bugs.. However, in the meantime,
> > I/we very much appreciate your help and expertise..
> >
> > REGRESSION TEST RESULTS ARE BELOW...
> >
> > Dan Williams  wrote on 07/15/2006 01:20:54 PM:
> > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > > > Hi, 
> > > > I'm having a problem *not* being able to build 'i686' packages
> >
> > (i.e.
> > > > optional_arches=i686) anymore *after* having upgraded our plague
> > > > server/builder (Opteron x86_64) a couple of weeks ago from
> > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > > > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > > > continues to work flawlessly.
> > >
> > > Found and fixed in CVS HEAD.  Patch attached that should apply
> >
> > against
> > > an installed copy of plague-server as well.  Thanks for the report,
> >
> > and
> > > sorry for the lag.
> > >
> > > Cheers,
> > > Dan
> > >
> > > [attachment "plague-opt-arches-fix.patch" deleted by Joe 
> > > Todaro/Poughkeepsie/IBM]
> >
> > 626 (WhoAmI): Requesting depsolve... 
> > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686',
> > 'noarch'].
>
> Ok; what do you have in the 'optional_arches' for this target?

======================================
Excerpt from lnxaddons-100-install.cfg
======================================

-----------<snip>------------ 
[Arches]
base_arches=i386
optional_arches=i686 noarch
----------</snip>------------
> What do the top bits of the specfile for WhoAmI look like?  i.e., are
> there any of the following specfile tags, and what are their values?
>
> BuildArch
> ExclusiveArch
> ExcludeArch

========================
Excerpt from WhoAmI.spec
========================

-----------<snip>------------ 
%define rversion 4.00
%define rel      9
%define pkgname  WhoAmI

Summary         : %{__distribution_long} system information
Name            : %{pkgname}
Version         : %{rversion}
Release         : %{rel}_%{__build_distribution}%{__build_release}
License         : %__spec_internal_license
Group           : System Environment/Base
Url             : %__repository_url

Packager        : %{packager}

Source0         : %{pkgname}.tar.gz

%if %__build_rhel
Requires        : coreutils, bash, python, grep, rpm
%endif

%if %__build_suse
Requires        : coreutils, bash, aaa_base, python, grep, rpm
%endif

BuildArch       : noarch
BuildRoot       : %{_tmppath}/%{name}-%{version}-root
AutoReqProv     : no

%Description
%{__distribution_long} system information

%prep
[ "%{buildroot}" != "/" ] && %{__rm} -rf %{buildroot}
-----------</snip>------------
> > 626 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch
> > i686. 
> > Exception in thread PackageJob: 626/WhoAmI: 
> > Traceback (most recent call last): 
> >   File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap 
> >     self.run() 
> >   File "/usr/share/plague/server/PackageJob.py", line 86, in run 
> >     self._pkg_job.process() 
> >   File "/usr/share/plague/server/PackageJob.py", line 719, in process 
> >     if func(): 
> >   File "/usr/share/plague/server/PackageJob.py", line 587, in
> > _stage_depsolve 
> >     if self._arch_deps_solved(arch) == False: 
> >   File "/usr/share/plague/server/PackageJob.py", line 540, in
> > _arch_deps_solved 
> >     if base: 
> > UnboundLocalError: local variable 'base' referenced before assignment
>
> To get rid of this error, remove the marked line around line 490 of
> PackageJob.py:
>         base = yum.YumBase()
>         yum_config = self._write_yum_conf(arch)
>         if not yum_config:
>           del base
>           raise DepError("WARNING: bad yum config for arch %s." % 

arch)
>         depsolve_root = os.path.dirname(yum_config) + '/'
>         base.doConfigSetup(fn=yum_config, root=depsolve_root)
>
> That's a bug.  Hope we can get this fixed.

Me too.. your help is really appreciated.
> Dan
> > Thanks again, 
> > -Joe

Joe

[Dan Williams]

On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote:
> Dan Williams  wrote on 07/16/2006 10:41:55 AM:
> > On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote:
> > > Thanks Dan.
> > >
> > > So I ran a quick regression test, just to make sure that *i386* is
> > > still working, but, it *failed* ... (also note that i686 gave
> > > basically the same errors..)
> > >
> > > SEE BELOW for *regression* test results, both BEFORE and AFTER
> > > applying patch 'plague-opt-arches-fix.patch' to server/Config.py.
>
> I
> > > repeated that test sequence several times, by the way.  Also, I
>
> was
> > > surprised to see that 'WhoAmI' was trying to build as an 'i686'
>
> rpm
> > > rather than a 'noarch' which is what it really is. So, since I
>
> can't
> > Hmm... Building an i386 package worked for me when I tested it with
> > current CVS HEAD, as did an i686 package.
>
> Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm
> -ivh) might help?  
> If so, are there any "gotchas" I need to be aware of? 
> But hopefully this won't be necessary.

Hmm, this likely wouldn't change much.
> But also I have a question: Since i686 is a "subarch" of i386, do I
> need 
> to create any target config files specicially for i686, such as in
> the 
> following directories:
> /etc/plague/server/targets
> /etc/plague/builder/targets
> /etc/mock

No, you shouldn't need to create target configs for that.  Only for
'base' architectures like i386, ppc, ppc64, x86_64, sparc, sparc64, etc.
Sub-architectures like i686, em64t, sparcv9, etc, don't need their own
target configs.

For the builder, you need one target config for every "repository" that
builder will be able to support.  For example, if the machine were an
x86-64-class machine able to build Fedora Core and Fedora Extras, you'd
have in /etc/plague/builder/targets:

fedora-devel-i386-core.cfg
fedora-5-i386-core.cfg
fedora-4-i386-core.cfg
fedora-devel-i386-extras.cfg
fedora-5-i386-extras.cfg
fedora-4-i386-extras.cfg
fedora-devel-x86_64-core.cfg
fedora-5-x86_64-core.cfg
...

and on the server, which has _no_ architecture at all, you'd have
in /etc/plague/server/targets:

fedora-devel-core.cfg
fedora-5-core.cfg
fedora-4-core.cfg
fedora-devel-extras.cfg
fedora-5-extras.cfg
fedora-4-extras.cfg

and on the server, if you care about depsolving (ie if you have the
"depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) then
you actually do need mock target configs for everything too, because the
server will use those to depsolve.
> > Coming back to the original question, optional_arches gives
>
> buildsystem
> > administrators a filter on arches to build.  For example, Fedora
>
> Extras
> > doesn't necessarily want to allow packages to be built for i486,
>
> even if
> > the RPM requests it, because we don't support i486.  So if you add
>
> i686
> > to 'optional_arches', then packages that request to be built for
>
> i686
> > with ExclusiveArch will be allowed to build for i686, otherwise not.
>
> I'm also really clear on this, too, now.  Well, at least this proves,
> to  
> me anyway, that I've had a pretty decent understanding of how things
> work 
> all along -- thanks to the great description you gave above, of
> course.

No problem.  There's so many options to stuff like RPM and apt/dpkg that
it's hard to keep straight.
> > > Also, could you please explain/elaborate when you say: "Patch
>
> attached
> > > that should apply against an installed copy of plague-server as
> > > well."    
> > > Um, your patch is clearly intended to update file server/Config.py
> > > file, so I really don't understand how I can also use *it* to
>
> update
> > > file *plague-server* (/usr/bin/plague-server?) as well, which
>
> appears
> > > to be a start-up script -- am I missing something here?
> >
> > I had meant to change the paths such that you could apply the patch
> > from / using patch -p0 or such; I did that and then re-diffed and
>
> likely
> > forgot to update the patch bits again.  Since the 'cvs diff' is
>
> rooted
> > at a different directory than the installed plague server directory,
>
> I
> > was trying to make life simpler for you.  My bad.
>
> Oh, cool, I've never done that before - thanks.
>
> Regardless, though, I still don't see any variable
> in /usr/bin/plague-server 
> that has *anything* to with "optional arches" or "base arches", which
> is  
> what the patch was for right?

The patch would actually have been
against /usr/share/plague/server/Config.py.  That's where it gets
somewhat confusing, since in the build directory everything is together,
but during RPM creation and install the server's main.py file gets
renamed to /usr/bin/plague-server.  What I've said 'plague-server', I've
generally been referring to the server process as a whole, not specific
files.  Sorry for that.
> > > For the good news (for me anyway), I'm also trying to learn Python
> > > (Diving Into Python) on my own time, so that hopefully one day I
>
> might
> > > be able help squash some of these bugs.. However, in the meantime,
> > > I/we very much appreciate your help and expertise..
> > >
> > > REGRESSION TEST RESULTS ARE BELOW...
> > >
> > > Dan Williams  wrote on 07/15/2006 01:20:54 PM:
> > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > > > > Hi,  
> > > > > I'm having a problem *not* being able to build 'i686' packages
> > >
> > > (i.e.
> > > > > optional_arches=i686) anymore *after* having upgraded our
>
> plague
> > > > > server/builder (Opteron x86_64) a couple of weeks ago from
> > > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with
>
> i686
> > > > > (optional_arches=i686) -- not with i386 (base_arches=i386)
>
> which
> > > > > continues to work flawlessly.
> > > >
> > > > Found and fixed in CVS HEAD.  Patch attached that should apply
> > >
> > > against
> > > > an installed copy of plague-server as well.  Thanks for the
>
> report,
> > > and
> > > > sorry for the lag.
> > > >
> > > > Cheers,
> > > > Dan
> > > >
> > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe 
> > > > Todaro/Poughkeepsie/IBM]
> > >
> > > 626 (WhoAmI): Requesting depsolve... 
> > > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686',
> > > 'noarch'].
> >
> > Ok; what do you have in the 'optional_arches' for this target?
>
> ====================================== 
> Excerpt from lnxaddons-100-install.cfg 
> ======================================
>
> -----------<snip>------------   
> [Arches] 
> base_arches=i386 
> optional_arches=i686 noarch

Ok; I think this is the cause of the noarch problem.  Since 'noarch' is
a base architecture in itself, you don't need it in optional_arches.  If
you remove it, I think things will work as you expect.  Every builder
can build a 'noarch' package, so that architecture is essentially added
by default and you don't need to do anything explicit for it.  Yay for
documentation (or complete lack of it).

However, this does expose a bug.  If an architecture is listed in
optional_arches, it shouldn't actually be used by a package build unless
the package was going to build for noarch already.
> ----------</snip>------------
> > What do the top bits of the specfile for WhoAmI look like?  i.e.,
>
> are
> > there any of the following specfile tags, and what are their values?
> >
> > BuildArch
> > ExclusiveArch
> > ExcludeArch
>
> ======================== 
> Excerpt from WhoAmI.spec 
> ========================
>
> -----------<snip>------------   
> %define rversion 4.00 
> %define rel      9 
> %define pkgname  WhoAmI
>
> Summary         : %{__distribution_long} system information 
> Name            : %{pkgname} 
> Version         : %{rversion} 
> Release         : %{rel}_%{__build_distribution}%{__build_release} 
> License         : %__spec_internal_license 
> Group           : System Environment/Base 
> Url             : %__repository_url
>
> Packager        : %{packager}
>
> Source0         : %{pkgname}.tar.gz
>
> %if %__build_rhel 
> Requires        : coreutils, bash, python, grep, rpm 
> %endif
>
> %if %__build_suse 
> Requires        : coreutils, bash, aaa_base, python, grep, rpm 
> %endif
>
> BuildArch       : noarch

I'm getting slightly confused here.  Plague 0.4.3 was building this
package correctly as an i686 package?  Do you really _want_ this to
build as an i686 package?  If so, I think the "BuildArch: noarch" is
wrong then.  Typically, if you want a package to build noarch, you
specify "BuildArch: noarch" and it will then only build on noarch.  What
arch is this package supposed to be for, exactly?

Cheers,
Dan

[Joe Todaro]

DW> Dan Williams  wrote on 07/17/2006 06:11:34 AM:

JT> > On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote:

JT> > Regardless, though, I still don't see any variable
JT> > in /usr/bin/plague-server that has *anything* to 
JT> > with "optional arches" or "base arches", which
JT> > is what the patch was for right? 
DW> 
DW> The patch would actually have been
DW> against /usr/share/plague/server/Config.py.

And indeed was successfully applied against 
/usr/share/plague/server/Config.py on July 15; 
one line got updated.

DW> That's where it gets
DW> somewhat confusing, since in the build directory everything is 
together,
DW> but during RPM creation and install the server's main.py file gets
DW> renamed to /usr/bin/plague-server.  What I've said 'plague-server', 
I've
DW> generally been referring to the server process as a whole, not 
specific
DW> files.  Sorry for that.
DW>

No apology necessary - my misunderstanding. Also, thanks for the 
excellent clarification/insight.

DW> and on the server, which has _no_ architecture at all, you'd have
DW> in /etc/plague/server/targets:
DW> 
DW> fedora-devel-core.cfg
DW> fedora-5-core.cfg
DW> fedora-4-core.cfg
DW> fedora-devel-extras.cfg
DW> fedora-5-extras.cfg
DW> fedora-4-extras.cfg
DW> 
DW> and on the server, if you care about depsolving (ie if you have the
DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) 
then
DW> you actually do need mock target configs for everything too,
DW> because the server will use those to depsolve.

Absolutely, we do care about depsolving..

So, when you say "everything", does that also include (for example) 
'i686' now, too, regardless of the fact that i686 is still only a 
sub-architecture?

(Hmm, I wouldn't think so..but I just need to be clear on this.)

JT> > ======================== 
JT> > Excerpt from WhoAmI.spec 
JT> > ======================== 
JT> > 
JT> > -----------<snip>------------ 
JT> > %define rversion 4.00 
JT> > %define rel      9 
JT> > %define pkgname  WhoAmI 
JT> > 
JT> > Summary         : %{__distribution_long} system information 
JT> > Name            : %{pkgname} 
JT> > Version         : %{rversion} 
JT> > Release         : %{rel}_%{__build_distribution}%{__build_release} 
JT> > License         : %__spec_internal_license 
JT> > Group           : System Environment/Base 
JT> > Url             : %__repository_url 
JT> > 
JT> > Packager        : %{packager} 
JT> > 
JT> > Source0         : %{pkgname}.tar.gz 
JT> > 
JT> > %if %__build_rhel 
JT> > Requires        : coreutils, bash, python, grep, rpm 
JT> > %endif 
JT> > 
JT> > %if %__build_suse 
JT> > Requires        : coreutils, bash, aaa_base, python, grep, rpm 
JT> > %endif 
JT> > 
JT> > BuildArch       : noarch 
JT> >

DW> I'm getting slightly confused here.

Sorry for that - my fault.

DW> Plague 0.4.3 was building this
DW> package correctly as an i686 package?

No, plague 0.4.3 was building this package correctly as 
a 'noarch'; it never tried building as an 'i686' package
until *after* I applied the one-liner patch to 
/usr/share/plague/server/Config.py on July 15;  Moreover,
it started doing so (building i686) automatically, because
PackageJob.py is now setting 
"archlist = ['i386', 'i686']" rather than
just "archlist = ['i386'] which it has always done in 
the past using plague-0.4.

DW> Do you really _want_ this to
DW> build as an i686 package?

No, of course not - this is a noarch only package, and always
has been.

DW> If so, I think the "BuildArch: noarch" is
DW> wrong then.  Typically, if you want a package to build noarch, you
DW> specify "BuildArch: noarch" and it will then only build on noarch. 
What
DW> arch is this package supposed to be for, exactly?

'noarch' as is specified in the specfile.

JT> > ====================================== 
JT> > Excerpt from lnxaddons-100-install.cfg 
JT> > ====================================== 
JT> > 
JT> > -----------<snip>------------ 
JT> > [Arches] 
JT> > base_arches=i386 
JT> > optional_arches=i686 noarch 
JT> >

GW> Ok; I think this is the cause of the noarch problem.  Since 'noarch' 
is
GW> a base architecture in itself, you don't need it in optional_arches. 
If
GW> you remove it, I think things will work as you expect.

Nope - no such luck; I have removed 'noarch' from 'optional_arches', 
re-tested, and am still seeing that same error..

(Jeez, I feel like we're soo close..I can taste it.)

And now I have a design-related question if I may: 
Why is the PackageJob.py module appending 'optional_arches'
onto the 'archlist' list (lines 574-576), and later trying 
to process those "optional" arches as is they were "base" 
arches?

Hmm, *I* don't get it.. just when I thought 
things were starting to make sense too. (:

CW> Every builder
GW> can build a 'noarch' package, so that architecture is essentially 
added
GW> by default and you don't need to do anything explicit for it.  Yay for
GW> documentation (or complete lack of it).
GW> 
GW> However, this does expose a bug.  If an architecture is listed in
GW> optional_arches, it shouldn't actually be used by a package build 
unless
GW> the package was going to build for noarch already.
GW>

I think we're *real* close..

Regards,
Joe
====================== COMPLETE THREAD =============================

Dan Williams  wrote on 07/17/2006 06:11:34 AM:
> On Sun, 2006-07-16 at 12:18 -0400, Joe Todaro wrote:
> > Dan Williams  wrote on 07/16/2006 10:41:55 AM:
> > > On Sun, 2006-07-16 at 09:48 -0400, Joe Todaro wrote:
> > > > Thanks Dan.
> > > >
> > > > So I ran a quick regression test, just to make sure that *i386* is
> > > > still working, but, it *failed* ... (also note that i686 gave
> > > > basically the same errors..)
> > > >
> > > > SEE BELOW for *regression* test results, both BEFORE and AFTER
> > > > applying patch 'plague-opt-arches-fix.patch' to server/Config.py.
> >
> > I
> > > > repeated that test sequence several times, by the way.  Also, I
> >
> > was
> > > > surprised to see that 'WhoAmI' was trying to build as an 'i686'
> >
> > rpm
> > > > rather than a 'noarch' which is what it really is. So, since I
> >
> > can't
> > > Hmm... Building an i386 package worked for me when I tested it with
> > > current CVS HEAD, as did an i686 package.
> >
> > Do you think that re-installing plague-0.5 from scratch (rpm -e/rpm
> > -ivh) might help? 
> > If so, are there any "gotchas" I need to be aware of? 
> > But hopefully this won't be necessary.
>
> Hmm, this likely wouldn't change much.
> > But also I have a question: Since i686 is a "subarch" of i386, do I
> > need 
> > to create any target config files specicially for i686, such as in
> > the 
> > following directories:
> > /etc/plague/server/targets
> > /etc/plague/builder/targets
> > /etc/mock
>
> No, you shouldn't need to create target configs for that.  Only for
> 'base' architectures like i386, ppc, ppc64, x86_64, sparc, sparc64, etc.
> Sub-architectures like i686, em64t, sparcv9, etc, don't need their own
> target configs.
>
> For the builder, you need one target config for every "repository" that
> builder will be able to support.  For example, if the machine were an
> x86-64-class machine able to build Fedora Core and Fedora Extras, you'd
> have in /etc/plague/builder/targets:
>
> fedora-devel-i386-core.cfg
> fedora-5-i386-core.cfg
> fedora-4-i386-core.cfg
> fedora-devel-i386-extras.cfg
> fedora-5-i386-extras.cfg
> fedora-4-i386-extras.cfg
> fedora-devel-x86_64-core.cfg
> fedora-5-x86_64-core.cfg
> ...
>
> and on the server, which has _no_ architecture at all, you'd have
> in /etc/plague/server/targets:
>
> fedora-devel-core.cfg
> fedora-5-core.cfg
> fedora-4-core.cfg
> fedora-devel-extras.cfg
> fedora-5-extras.cfg
> fedora-4-extras.cfg
>
> and on the server, if you care about depsolving (ie if you have the
> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg) then
> you actually do need mock target configs for everything too, because the
> server will use those to depsolve.
> > > Coming back to the original question, optional_arches gives
> >
> > buildsystem
> > > administrators a filter on arches to build.  For example, Fedora
> >
> > Extras
> > > doesn't necessarily want to allow packages to be built for i486,
> >
> > even if
> > > the RPM requests it, because we don't support i486.  So if you add
> >
> > i686
> > > to 'optional_arches', then packages that request to be built for
> >
> > i686
> > > with ExclusiveArch will be allowed to build for i686, otherwise not.
> >
> > I'm also really clear on this, too, now.  Well, at least this proves,
> > to 
> > me anyway, that I've had a pretty decent understanding of how things
> > work 
> > all along -- thanks to the great description you gave above, of
> > course.
>
> No problem.  There's so many options to stuff like RPM and apt/dpkg that
> it's hard to keep straight.
> > > > Also, could you please explain/elaborate when you say: "Patch
> >
> > attached
> > > > that should apply against an installed copy of plague-server as
> > > > well." 
> > > > Um, your patch is clearly intended to update file server/Config.py
> > > > file, so I really don't understand how I can also use *it* to
> >
> > update
> > > > file *plague-server* (/usr/bin/plague-server?) as well, which
> >
> > appears
> > > > to be a start-up script -- am I missing something here?
> > >
> > > I had meant to change the paths such that you could apply the patch
> > > from / using patch -p0 or such; I did that and then re-diffed and
> >
> > likely
> > > forgot to update the patch bits again.  Since the 'cvs diff' is
> >
> > rooted
> > > at a different directory than the installed plague server directory,
> >
> > I
> > > was trying to make life simpler for you.  My bad.
> >
> > Oh, cool, I've never done that before - thanks.
> >
> > Regardless, though, I still don't see any variable
> > in /usr/bin/plague-server 
> > that has *anything* to with "optional arches" or "base arches", which
> > is 
> > what the patch was for right?
>
> The patch would actually have been
> against /usr/share/plague/server/Config.py.  That's where it gets
> somewhat confusing, since in the build directory everything is together,
> but during RPM creation and install the server's main.py file gets
> renamed to /usr/bin/plague-server.  What I've said 'plague-server', I've
> generally been referring to the server process as a whole, not specific
> files.  Sorry for that.
> > > > For the good news (for me anyway), I'm also trying to learn Python
> > > > (Diving Into Python) on my own time, so that hopefully one day I
> >
> > might
> > > > be able help squash some of these bugs.. However, in the meantime,
> > > > I/we very much appreciate your help and expertise..
> > > >
> > > > REGRESSION TEST RESULTS ARE BELOW...
> > > >
> > > > Dan Williams  wrote on 07/15/2006 01:20:54 PM:
> > > > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > > > > > Hi, 
> > > > > > I'm having a problem *not* being able to build 'i686' packages
> > > >
> > > > (i.e.
> > > > > > optional_arches=i686) anymore *after* having upgraded our
> >
> > plague
> > > > > > server/builder (Opteron x86_64) a couple of weeks ago from
> > > > > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with
> >
> > i686
> > > > > > (optional_arches=i686) -- not with i386 (base_arches=i386)
> >
> > which
> > > > > > continues to work flawlessly.
> > > > >
> > > > > Found and fixed in CVS HEAD.  Patch attached that should apply
> > > >
> > > > against
> > > > > an installed copy of plague-server as well.  Thanks for the
> >
> > report,
> > > > and
> > > > > sorry for the lag.
> > > > >
> > > > > Cheers,
> > > > > Dan
> > > > >
> > > > > [attachment "plague-opt-arches-fix.patch" deleted by Joe 
> > > > > Todaro/Poughkeepsie/IBM]
> > > >
> > > > 626 (WhoAmI): Requesting depsolve... 
> > > > 626 (WhoAmI): Starting depsolve for arches: ['i386', 'i686',
> > > > 'noarch'].
> > >
> > > Ok; what do you have in the 'optional_arches' for this target?
> >
> > ====================================== 
> > Excerpt from lnxaddons-100-install.cfg 
> > ======================================
> >
> > -----------<snip>------------ 
> > [Arches] 
> > base_arches=i386 
> > optional_arches=i686 noarch
>
> Ok; I think this is the cause of the noarch problem.  Since 'noarch' is
> a base architecture in itself, you don't need it in optional_arches.  If
> you remove it, I think things will work as you expect.  Every builder
> can build a 'noarch' package, so that architecture is essentially added
> by default and you don't need to do anything explicit for it.  Yay for
> documentation (or complete lack of it).
>
> However, this does expose a bug.  If an architecture is listed in
> optional_arches, it shouldn't actually be used by a package build unless
> the package was going to build for noarch already.
> > ----------</snip>------------
> > > What do the top bits of the specfile for WhoAmI look like?  i.e.,
> >
> > are
> > > there any of the following specfile tags, and what are their values?
> > >
> > > BuildArch
> > > ExclusiveArch
> > > ExcludeArch
> >
> > ======================== 
> > Excerpt from WhoAmI.spec 
> > ========================
> >
> > -----------<snip>------------ 
> > %define rversion 4.00 
> > %define rel      9 
> > %define pkgname  WhoAmI
> >
> > Summary         : %{__distribution_long} system information 
> > Name            : %{pkgname} 
> > Version         : %{rversion} 
> > Release         : %{rel}_%{__build_distribution}%{__build_release} 
> > License         : %__spec_internal_license 
> > Group           : System Environment/Base 
> > Url             : %__repository_url
> >
> > Packager        : %{packager}
> >
> > Source0         : %{pkgname}.tar.gz
> >
> > %if %__build_rhel 
> > Requires        : coreutils, bash, python, grep, rpm 
> > %endif
> >
> > %if %__build_suse 
> > Requires        : coreutils, bash, aaa_base, python, grep, rpm 
> > %endif
> >
> > BuildArch       : noarch
>
> I'm getting slightly confused here.  Plague 0.4.3 was building this
> package correctly as an i686 package?  Do you really _want_ this to
> build as an i686 package?  If so, I think the "BuildArch: noarch" is
> wrong then.  Typically, if you want a package to build noarch, you
> specify "BuildArch: noarch" and it will then only build on noarch.  What
> arch is this package supposed to be for, exactly?
>
> Cheers,
> Dan

[Dan Williams]

On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote:
> DW> and on the server, which has _no_ architecture at all, you'd have
> DW> in /etc/plague/server/targets:
> DW> 
> DW> fedora-devel-core.cfg
> DW> fedora-5-core.cfg
> DW> fedora-4-core.cfg
> DW> fedora-devel-extras.cfg
> DW> fedora-5-extras.cfg
> DW> fedora-4-extras.cfg
> DW> 
> DW> and on the server, if you care about depsolving (ie if you have
> the
> DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg)
> then
> DW> you actually do need mock target configs for everything too, 
> DW> because the server will use those to depsolve.
>
> Absolutely, we do care about depsolving..
>
> So, when you say "everything", does that also include (for example)  
> 'i686' now, too, regardless of the fact that i686 is still only a  
> sub-architecture?

No; you only need mock and builder configs for base architectures like
i386, x86-64, ppc, sparc, sparc64, ppc64, etc.
> DW> Do you really _want_ this to
> DW> build as an i686 package?
>
> No, of course not - this is a noarch only package, and always 
> has been.
>
> DW> If so, I think the "BuildArch: noarch" is
> DW> wrong then.  Typically, if you want a package to build noarch, you
> DW> specify "BuildArch: noarch" and it will then only build on
> noarch.  What
> DW> arch is this package supposed to be for, exactly?
>
> 'noarch' as is specified in the specfile.

Ok, can you apply the attached patch?  cd to /usr/share/plague/server
and use 'patch -p0 < patchfile'.  Enqueue the WhoAmI SRPM and let me
know what it prints out.  I tried running a noarch package through this
morning and it seemed to go through fine; plague decided that 'noarch'
was the only arch it was going to build for, even with i686 in the
optional_arches for that target.
> JT> > ====================================== 
> JT> > Excerpt from lnxaddons-100-install.cfg 
> JT> > ====================================== 
> JT> > 
> JT> > -----------<snip>------------   
> JT> > [Arches] 
> JT> > base_arches=i386 
> JT> > optional_arches=i686 noarch  
> JT> >
>
> GW> Ok; I think this is the cause of the noarch problem.  Since
> 'noarch' is
> GW> a base architecture in itself, you don't need it in
> optional_arches.  If
> GW> you remove it, I think things will work as you expect.
>
> Nope - no such luck; I have removed 'noarch' from 'optional_arches',  
> re-tested, and am still seeing that same error..

noarch shouldn't be in optional_arches, but if it's not working
correctly when you remove it there's some other issue.

Dan

[Joe Todaro]

Dan Williams  wrote on 07/19/2006 10:07:54 AM:
> On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote:
> > DW> and on the server, which has _no_ architecture at all, you'd have
> > DW> in /etc/plague/server/targets:
> > DW> 
> > DW> fedora-devel-core.cfg
> > DW> fedora-5-core.cfg
> > DW> fedora-4-core.cfg
> > DW> fedora-devel-extras.cfg
> > DW> fedora-5-extras.cfg
> > DW> fedora-4-extras.cfg
> > DW> 
> > DW> and on the server, if you care about depsolving (ie if you have
> > the
> > DW> "depsolve_jobs = yes" set in /etc/plague/server/plague-server.cfg)
> > then
> > DW> you actually do need mock target configs for everything too, 
> > DW> because the server will use those to depsolve.
> >
> > Absolutely, we do care about depsolving..
> >
> > So, when you say "everything", does that also include (for example) 
> > 'i686' now, too, regardless of the fact that i686 is still only a 
> > sub-architecture?
>
> No; you only need mock and builder configs for base architectures like
> i386, x86-64, ppc, sparc, sparc64, ppc64, etc.
> > DW> Do you really _want_ this to
> > DW> build as an i686 package?
> >
> > No, of course not - this is a noarch only package, and always 
> > has been.
> >
> > DW> If so, I think the "BuildArch: noarch" is
> > DW> wrong then.  Typically, if you want a package to build noarch, you
> > DW> specify "BuildArch: noarch" and it will then only build on
> > noarch.  What
> > DW> arch is this package supposed to be for, exactly?
> >
> > 'noarch' as is specified in the specfile.
>
> Ok, can you apply the attached patch?  cd to /usr/share/plague/server
> and use 'patch -p0 < patchfile'.  Enqueue the WhoAmI SRPM and let me
> know what it prints out.

Yes, I have applied the patch and re-enqueued the WhoAmI SRPM.
Below are the results.   Hope you find something - good luck.. 
And, thanks a lot for all the help you've been providing me,
which is very much appreciated.

=====================================
below from /var/log/plague-server.log
=====================================
Request to enqueue 'WhoAmI' tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for 
target 'ao100' (user '[e-mail]')
630 (WhoAmI): Starting tag 
'/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on target 
'lnxaddons-100-install'
630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude []
630 (WhoAmI): AH: addl_arches []
630 (WhoAmI): AH: base_arches ['i386', 'i686']
630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed: None
630 (WhoAmI): Requesting depsolve...
630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686'].
630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch i686.
Exception in thread PackageJob: 630/WhoAmI:
Traceback (most recent call last):
  File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap
    self.run()
  File "/usr/share/plague/server/PackageJob.py", line 86, in run
    self._pkg_job.process()
  File "/usr/share/plague/server/PackageJob.py", line 725, in process
    if func():
  File "/usr/share/plague/server/PackageJob.py", line 593, in 
_stage_depsolve
    if self._arch_deps_solved(arch) == False:
  File "/usr/share/plague/server/PackageJob.py", line 546, in 
_arch_deps_solved
    if base:
UnboundLocalError: local variable 'base' referenced before assignment

Re-activating builder 'https://lnxbuild1.abc.com:8888'.
=====================================
above from /var/log/plague-server.log
=====================================
> I tried running a noarch package through this
> morning and it seemed to go through fine; plague decided that 'noarch'
> was the only arch it was going to build for, even with i686 in the
> optional_arches for that target.
> > JT> > ====================================== 
> > JT> > Excerpt from lnxaddons-100-install.cfg 
> > JT> > ====================================== 
> > JT> > 
> > JT> > -----------<snip>------------ 
> > JT> > [Arches] 
> > JT> > base_arches=i386 
> > JT> > optional_arches=i686 noarch 
> > JT> >
> >
> > GW> Ok; I think this is the cause of the noarch problem.  Since
> > 'noarch' is
> > GW> a base architecture in itself, you don't need it in
> > optional_arches.  If
> > GW> you remove it, I think things will work as you expect.
> >
> > Nope - no such luck; I have removed 'noarch' from 'optional_arches', 
> > re-tested, and am still seeing that same error..
>
> noarch shouldn't be in optional_arches, but if it's not working
> correctly when you remove it there's some other issue.
>
> Dan
>
> [attachment "print-archhandling.patch" deleted by Joe 
> Todaro/Poughkeepsie/IBM]

[Dan Williams]

On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote:
> Dan Williams  wrote on 07/19/2006 10:07:54 AM:
> > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote:
> > > DW> and on the server, which has _no_ architecture at all, you'd
>
> have
> > > DW> in /etc/plague/server/targets:
> > > DW> 
> > > DW> fedora-devel-core.cfg
> > > DW> fedora-5-core.cfg
> > > DW> fedora-4-core.cfg
> > > DW> fedora-devel-extras.cfg
> > > DW> fedora-5-extras.cfg
> > > DW> fedora-4-extras.cfg
> > > DW> 
> > > DW> and on the server, if you care about depsolving (ie if you
>
> have
> > > the
> > > DW> "depsolve_jobs = yes" set
>
> in /etc/plague/server/plague-server.cfg)
> > > then
> > > DW> you actually do need mock target configs for everything too, 
> > > DW> because the server will use those to depsolve.
> > >
> > > Absolutely, we do care about depsolving..
> > >
> > > So, when you say "everything", does that also include (for
>
> example)
> > > 'i686' now, too, regardless of the fact that i686 is still only a
> > > sub-architecture?
> >
> > No; you only need mock and builder configs for base architectures
>
> like
> > i386, x86-64, ppc, sparc, sparc64, ppc64, etc.
> > > DW> Do you really _want_ this to
> > > DW> build as an i686 package?
> > >
> > > No, of course not - this is a noarch only package, and always 
> > > has been.
> > >
> > > DW> If so, I think the "BuildArch: noarch" is
> > > DW> wrong then.  Typically, if you want a package to build noarch,
>
> you
> > > DW> specify "BuildArch: noarch" and it will then only build on
> > > noarch.  What
> > > DW> arch is this package supposed to be for, exactly?
> > >
> > > 'noarch' as is specified in the specfile.
> >
> > Ok, can you apply the attached patch?  cd
>
> to /usr/share/plague/server
> > and use 'patch -p0 < patchfile'.  Enqueue the WhoAmI SRPM and let me
> > know what it prints out.
>
> Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. 
> Below are the results.   Hope you find something - good luck..   
> And, thanks a lot for all the help you've been providing me, 
> which is very much appreciated.
>
> ===================================== 
> below from /var/log/plague-server.log 
> ===================================== 
> Request to enqueue 'WhoAmI' tag
> '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for
> target 'ao100' (user '[e-mail]') 
> 630 (WhoAmI): Starting tag
> '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on
> target 'lnxaddons-100-install' 
> 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] 
> 630 (WhoAmI): AH: addl_arches [] 
> 630 (WhoAmI): AH: base_arches ['i386', 'i686'] 
> 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed:
> None

Ok; this tells me that the PackageJob class' arch_handling() function is
working as expected.  It also tells me that the depsolve stuff is
picking the wrong arches to depsolve.  So we're closer in some senses to
fixing the issue, but farther away in others :)

Dan
> 630 (WhoAmI): Requesting depsolve... 
> 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. 
> 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch
> i686. 
> Exception in thread PackageJob: 630/WhoAmI: 
> Traceback (most recent call last): 
>   File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap 
>     self.run() 
>   File "/usr/share/plague/server/PackageJob.py", line 86, in run 
>     self._pkg_job.process() 
>   File "/usr/share/plague/server/PackageJob.py", line 725, in process 
>     if func(): 
>   File "/usr/share/plague/server/PackageJob.py", line 593, in
> _stage_depsolve 
>     if self._arch_deps_solved(arch) == False: 
>   File "/usr/share/plague/server/PackageJob.py", line 546, in
> _arch_deps_solved 
>     if base: 
> UnboundLocalError: local variable 'base' referenced before assignment
> Re-activating builder 'https://lnxbuild1.abc.com:8888'.
> above from /var/log/plague-server.log
> > I tried running a noarch package through this
> > morning and it seemed to go through fine; plague decided that
>
> 'noarch'
> > was the only arch it was going to build for, even with i686 in the
> > optional_arches for that target.
> > > JT> > ====================================== 
> > > JT> > Excerpt from lnxaddons-100-install.cfg 
> > > JT> > ====================================== 
> > > JT> > 
> > > JT> > -----------<snip>------------   
> > > JT> > [Arches] 
> > > JT> > base_arches=i386 
> > > JT> > optional_arches=i686 noarch  
> > > JT> >
> > >
> > > GW> Ok; I think this is the cause of the noarch problem.  Since
> > > 'noarch' is
> > > GW> a base architecture in itself, you don't need it in
> > > optional_arches.  If
> > > GW> you remove it, I think things will work as you expect.
> > >
> > > Nope - no such luck; I have removed 'noarch' from
>
> 'optional_arches',
> > > re-tested, and am still seeing that same error..
> >
> > noarch shouldn't be in optional_arches, but if it's not working
> > correctly when you remove it there's some other issue.
> >
> > Dan
> >
> > [attachment "print-archhandling.patch" deleted by Joe 
> > Todaro/Poughkeepsie/IBM]

(FIXED) Re: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3)

[Joe Todaro]

Hey Dan,

Great news. We *fixed* the plague-0.5 "depsolve" problem with a minor
change to the PackageJob.py module.  See below...
> > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> >
> > Hi, 
> > I'm having a problem *not* being able to build 'i686' packages (i.e.
> > optional_arches=i686) anymore *after* having upgraded our plague
> > server/builder (Opteron x86_64) a couple of weeks ago from
> > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > continues to work flawlessly.
> Dan Williams  wrote on 07/19/2006 02:07:05 PM:
> Ok, can you apply the attached patch? 
> cd to /usr/share/plague/server
> and use 'patch -p0 < patchfile'.
> Enqueue the WhoAmI SRPM and let me
> know what it prints out.

------- snip - snip ------
> Ok; this tells me that the PackageJob class' arch_handling() function is
> working as expected.  It also tells me that the depsolve stuff is
> picking the wrong arches to depsolve.  So we're closer in some senses to
> fixing the issue, but farther away in others :)

In _write_yum_conf() we changed:

mock_config = self._target_cfg.mock_config_for_arch(arch)

to:

mock_config = 
self._target_cfg.mock_config_for_arch(ArchUtils.sub_arches[arch])

*that* did the trick! ... our new (plague-0.5 based) buildsys is 
up and running the same as it was *before* we upgraded from 
plague-0.4.

So, please _confirm_ the above change - that it's *really* what you 
intended to do?   If it is, I will be happy to re-test the code 
after you commit the changes to the CVS HEAD.

Well, thanks again for all your help, especially for clarifying some 
of the key Plague buildsystem design points, without which it would 
have been impossible to debug/resolve the above "depsolve" issue.

Also, this provided the opportunity to dig into some well-written
Python code and do some serious debugging. Thanks again!

Regards,
Joe Todaro

=============== HERE"S THE COMPLETE THREAD ==============
Dan Williams  wrote on 07/19/2006 02:07:05 PM:
> On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote:
> > Dan Williams  wrote on 07/19/2006 10:07:54 AM:
> > > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote:
> > > > DW> and on the server, which has _no_ architecture at all, you'd
> >
> > have
> > > > DW> in /etc/plague/server/targets:
> > > > DW> 
> > > > DW> fedora-devel-core.cfg
> > > > DW> fedora-5-core.cfg
> > > > DW> fedora-4-core.cfg
> > > > DW> fedora-devel-extras.cfg
> > > > DW> fedora-5-extras.cfg
> > > > DW> fedora-4-extras.cfg
> > > > DW> 
> > > > DW> and on the server, if you care about depsolving (ie if you
> >
> > have
> > > > the
> > > > DW> "depsolve_jobs = yes" set
> >
> > in /etc/plague/server/plague-server.cfg)
> > > > then
> > > > DW> you actually do need mock target configs for everything too, 
> > > > DW> because the server will use those to depsolve.
> > > >
> > > > Absolutely, we do care about depsolving..
> > > >
> > > > So, when you say "everything", does that also include (for
> >
> > example)
> > > > 'i686' now, too, regardless of the fact that i686 is still only a
> > > > sub-architecture?
> > >
> > > No; you only need mock and builder configs for base architectures
> >
> > like
> > > i386, x86-64, ppc, sparc, sparc64, ppc64, etc.
> > > > DW> Do you really _want_ this to
> > > > DW> build as an i686 package?
> > > >
> > > > No, of course not - this is a noarch only package, and always 
> > > > has been.
> > > >
> > > > DW> If so, I think the "BuildArch: noarch" is
> > > > DW> wrong then.  Typically, if you want a package to build noarch,
> >
> > you
> > > > DW> specify "BuildArch: noarch" and it will then only build on
> > > > noarch.  What
> > > > DW> arch is this package supposed to be for, exactly?
> > > >
> > > > 'noarch' as is specified in the specfile.
> > >
> > > Ok, can you apply the attached patch?  cd
> >
> > to /usr/share/plague/server
> > > and use 'patch -p0 < patchfile'.  Enqueue the WhoAmI SRPM and let me
> > > know what it prints out.
> >
> > Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. 
> > Below are the results.   Hope you find something - good luck.. 
> > And, thanks a lot for all the help you've been providing me, 
> > which is very much appreciated.
> >
> > ===================================== 
> > below from /var/log/plague-server.log 
> > ===================================== 
> > Request to enqueue 'WhoAmI' tag
> > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' for
> > target 'ao100' (user '[e-mail]') 
> > 630 (WhoAmI): Starting tag
> > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm' on
> > target 'lnxaddons-100-install' 
> > 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] 
> > 630 (WhoAmI): AH: addl_arches [] 
> > 630 (WhoAmI): AH: base_arches ['i386', 'i686'] 
> > 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None, allowed:
> > None
>
> Ok; this tells me that the PackageJob class' arch_handling() function is
> working as expected.  It also tells me that the depsolve stuff is
> picking the wrong arches to depsolve.  So we're closer in some senses to
> fixing the issue, but farther away in others :)
>
> Dan
> > 630 (WhoAmI): Requesting depsolve... 
> > 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. 
> > 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for arch
> > i686. 
> > Exception in thread PackageJob: 630/WhoAmI: 
> > Traceback (most recent call last): 
> >   File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap 
> >     self.run() 
> >   File "/usr/share/plague/server/PackageJob.py", line 86, in run 
> >     self._pkg_job.process() 
> >   File "/usr/share/plague/server/PackageJob.py", line 725, in process 
> >     if func(): 
> >   File "/usr/share/plague/server/PackageJob.py", line 593, in
> > _stage_depsolve 
> >     if self._arch_deps_solved(arch) == False: 
> >   File "/usr/share/plague/server/PackageJob.py", line 546, in
> > _arch_deps_solved 
> >     if base: 
> > UnboundLocalError: local variable 'base' referenced before assignment
> > Re-activating builder 'https://lnxbuild1.abc.com:8888'.
> > above from /var/log/plague-server.log
> > > I tried running a noarch package through this
> > > morning and it seemed to go through fine; plague decided that
> >
> > 'noarch'
> > > was the only arch it was going to build for, even with i686 in the
> > > optional_arches for that target.
> > > > JT> > ====================================== 
> > > > JT> > Excerpt from lnxaddons-100-install.cfg 
> > > > JT> > ====================================== 
> > > > JT> > 
> > > > JT> > -----------<snip>------------ 
> > > > JT> > [Arches] 
> > > > JT> > base_arches=i386 
> > > > JT> > optional_arches=i686 noarch 
> > > > JT> >
> > > >
> > > > GW> Ok; I think this is the cause of the noarch problem.  Since
> > > > 'noarch' is
> > > > GW> a base architecture in itself, you don't need it in
> > > > optional_arches.  If
> > > > GW> you remove it, I think things will work as you expect.
> > > >
> > > > Nope - no such luck; I have removed 'noarch' from
> >
> > 'optional_arches',
> > > > re-tested, and am still seeing that same error..
> > >
> > > noarch shouldn't be in optional_arches, but if it's not working
> > > correctly when you remove it there's some other issue.
> > >
> > > Dan
> > >
> > > [attachment "print-archhandling.patch" deleted by Joe 
> > > Todaro/Poughkeepsie/IBM]

(FIXED) Re: HELP: trouble building packages for optional_arches=i686 *after* upgrading to Plague-0.5.0 (from plague-0.4.3)

[Dan Williams]

On Fri, 2006-07-21 at 16:48 -0400, Joe Todaro wrote:
> Hey Dan,
>
> Great news. We *fixed* the plague-0.5 "depsolve" problem with a minor 
> change to the PackageJob.py module.  See below...
> > > On Tue, 2006-07-04 at 09:58 -0400, Joe Todaro wrote:
> > >
> > > Hi,  
> > > I'm having a problem *not* being able to build 'i686' packages
>
> (i.e.
> > > optional_arches=i686) anymore *after* having upgraded our plague
> > > server/builder (Opteron x86_64) a couple of weeks ago from
> > > plague-0.4.3 to *plague-0.5.0*. The problem happens only with i686
> > > (optional_arches=i686) -- not with i386 (base_arches=i386) which
> > > continues to work flawlessly.
> > Dan Williams  wrote on 07/19/2006 02:07:05 PM: 
> > Ok, can you apply the attached patch?   
> > cd to /usr/share/plague/server
> > and use 'patch -p0 < patchfile'. 
> > Enqueue the WhoAmI SRPM and let me
> > know what it prints out.
>
> ------- snip - snip ------
> > Ok; this tells me that the PackageJob class' arch_handling()
>
> function is
> > working as expected.  It also tells me that the depsolve stuff is
> > picking the wrong arches to depsolve.  So we're closer in some
>
> senses to
> > fixing the issue, but farther away in others :)
>
> In _write_yum_conf() we changed:
>
> mock_config = self._target_cfg.mock_config_for_arch(arch)
>
> to:
>
> mock_config =
> self._target_cfg.mock_config_for_arch(ArchUtils.sub_arches[arch])
>
> *that* did the trick! ... our new (plague-0.5 based) buildsys is  
> up and running the same as it was *before* we upgraded from  
> plague-0.4.
>
> So, please _confirm_ the above change - that it's *really* what you  
> intended to do?   If it is, I will be happy to re-test the code  
> after you commit the changes to the CVS HEAD.

Yes, that looks correct.  Usually we won't have a mock config file for
arches like i686 or em64t, since it's assumed that if you have a builder
for the sub-arch, you can build the whole list of arches, usually.
That's a somewhat erroneous assumption, but in reality, if you have an
i386-class machine, it's _not_ actually going to be an i386 from 1992 :)

I've committed a patch to HEAD that will try the specified arch (just in
case you do, in fact, have mock configs for the specific build arch) and
then if there's no mock config for that arch, will fall back to the base
architecture like your change above.  Thanks for the pointer.  Patch
attached.

Dan
> Well, thanks again for all your help, especially for clarifying some  
> of the key Plague buildsystem design points, without which it would  
> have been impossible to debug/resolve the above "depsolve" issue.
>
> Also, this provided the opportunity to dig into some well-written 
> Python code and do some serious debugging. Thanks again!
>
> Regards, 
> Joe Todaro
>
> =============== HERE"S THE COMPLETE THREAD ============== 
> Dan Williams  wrote on 07/19/2006 02:07:05 PM:
> > On Wed, 2006-07-19 at 13:09 -0400, Joe Todaro wrote:
> > > Dan Williams  wrote on 07/19/2006 10:07:54 AM:
> > > > On Mon, 2006-07-17 at 18:49 -0400, Joe Todaro wrote:
> > > > > DW> and on the server, which has _no_ architecture at all,
>
> you'd
> > > have
> > > > > DW> in /etc/plague/server/targets:
> > > > > DW> 
> > > > > DW> fedora-devel-core.cfg
> > > > > DW> fedora-5-core.cfg
> > > > > DW> fedora-4-core.cfg
> > > > > DW> fedora-devel-extras.cfg
> > > > > DW> fedora-5-extras.cfg
> > > > > DW> fedora-4-extras.cfg
> > > > > DW> 
> > > > > DW> and on the server, if you care about depsolving (ie if you
> > >
> > > have
> > > > > the
> > > > > DW> "depsolve_jobs = yes" set
> > >
> > > in /etc/plague/server/plague-server.cfg)
> > > > > then
> > > > > DW> you actually do need mock target configs for everything
>
> too,
> > > > > DW> because the server will use those to depsolve.
> > > > >
> > > > > Absolutely, we do care about depsolving..
> > > > >
> > > > > So, when you say "everything", does that also include (for
> > >
> > > example)
> > > > > 'i686' now, too, regardless of the fact that i686 is still
>
> only a
> > > > > sub-architecture?
> > > >
> > > > No; you only need mock and builder configs for base
>
> architectures
> > > like
> > > > i386, x86-64, ppc, sparc, sparc64, ppc64, etc.
> > > > > DW> Do you really _want_ this to
> > > > > DW> build as an i686 package?
> > > > >
> > > > > No, of course not - this is a noarch only package, and always 
> > > > > has been.
> > > > >
> > > > > DW> If so, I think the "BuildArch: noarch" is
> > > > > DW> wrong then.  Typically, if you want a package to build
>
> noarch,
> > > you
> > > > > DW> specify "BuildArch: noarch" and it will then only build on
> > > > > noarch.  What
> > > > > DW> arch is this package supposed to be for, exactly?
> > > > >
> > > > > 'noarch' as is specified in the specfile.
> > > >
> > > > Ok, can you apply the attached patch?  cd
> > >
> > > to /usr/share/plague/server
> > > > and use 'patch -p0 < patchfile'.  Enqueue the WhoAmI SRPM and
>
> let me
> > > > know what it prints out.
> > >
> > > Yes, I have applied the patch and re-enqueued the WhoAmI SRPM. 
> > > Below are the results.   Hope you find something - good luck..   
> > > And, thanks a lot for all the help you've been providing me, 
> > > which is very much appreciated.
> > >
> > > ===================================== 
> > > below from /var/log/plague-server.log 
> > > ===================================== 
> > > Request to enqueue 'WhoAmI' tag
> > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm'
>
> for
> > > target 'ao100' (user '[e-mail]') 
> > > 630 (WhoAmI): Starting tag
> > > '/afs/pok/projects/.../SRPMS/whoami/WhoAmI-4.00-9_rhel4.src.rpm'
>
> on
> > > target 'lnxaddons-100-install' 
> > > 630 (WhoAmI): AH: ba ['noarch'], exclusive [], exclude [] 
> > > 630 (WhoAmI): AH: addl_arches [] 
> > > 630 (WhoAmI): AH: base_arches ['i386', 'i686'] 
> > > 630 (WhoAmI): archjobs: {'noarch': None}, pkg_arches: None,
>
> allowed:
> > > None
> >
> > Ok; this tells me that the PackageJob class' arch_handling()
>
> function is
> > working as expected.  It also tells me that the depsolve stuff is
> > picking the wrong arches to depsolve.  So we're closer in some
>
> senses to
> > fixing the issue, but farther away in others :)
> >
> > Dan
> > > 630 (WhoAmI): Requesting depsolve... 
> > > 630 (WhoAmI): Starting depsolve for arches: ['i386', 'i686']. 
> > > 630 (WhoAmI/i686): Depsolve Error: WARNING: bad yum config for
>
> arch
> > > i686. 
> > > Exception in thread PackageJob: 630/WhoAmI: 
> > > Traceback (most recent call last): 
> > >   File "/usr/lib64/python2.3/threading.py", line 436, in
>
> __bootstrap
> > > self.run() 
> > >
> > > File "/usr/share/plague/server/PackageJob.py", line 86, in run 
> > >     self._pkg_job.process() 
> > >   File "/usr/share/plague/server/PackageJob.py", line 725, in
>
> process
> > > if func(): 
> > >
> > > File "/usr/share/plague/server/PackageJob.py", line 593, in
> > > _stage_depsolve 
> > >     if self._arch_deps_solved(arch) == False: 
> > >   File "/usr/share/plague/server/PackageJob.py", line 546, in
> > > _arch_deps_solved 
> > >     if base: 
> > > UnboundLocalError: local variable 'base' referenced before
>
> assignment
> > > Re-activating builder 'https://lnxbuild1.abc.com:8888'.
> > > above from /var/log/plague-server.log
> > > > I tried running a noarch package through this
> > > > morning and it seemed to go through fine; plague decided that
> > >
> > > 'noarch'
> > > > was the only arch it was going to build for, even with i686 in
>
> the
> > > > optional_arches for that target.
> > > > > JT> > ====================================== 
> > > > > JT> > Excerpt from lnxaddons-100-install.cfg 
> > > > > JT> > ====================================== 
> > > > > JT> > 
> > > > > JT> > -----------<snip>------------   
> > > > > JT> > [Arches] 
> > > > > JT> > base_arches=i386 
> > > > > JT> > optional_arches=i686 noarch  
> > > > > JT> >
> > > > >
> > > > > GW> Ok; I think this is the cause of the noarch problem.
>
> Since
> > > > > 'noarch' is
> > > > > GW> a base architecture in itself, you don't need it in
> > > > > optional_arches.  If
> > > > > GW> you remove it, I think things will work as you expect.
> > > > >
> > > > > Nope - no such luck; I have removed 'noarch' from
> > >
> > > 'optional_arches',
> > > > > re-tested, and am still seeing that same error..
> > > >
> > > > noarch shouldn't be in optional_arches, but if it's not working
> > > > correctly when you remove it there's some other issue.
> > > >
> > > > Dan
> > > >
> > > > [attachment "print-archhandling.patch" deleted by Joe 
> > > > Todaro/Poughkeepsie/IBM]