Critpath DNF/Python3 Conversion - Need Input
by Rob Marshall
I completed the conversion of the critpath.py script from YUM/Python2 to
Python2/3 compliance using DNF for dependency solving.
DNF requires an architecture name (arch) and a base architecture name
(basearch)
to resolve dependencies. The armhfp and i386 basearchs have a few issues
that
I worked around with what I consider a dirty hack.
Consider the below table dictionary of lists that DNF uses to validate
whether
or not an arch is valid:
Here's the code snippet:
76 _BASEARCH_MAP = _invert({
77 'aarch64': ('aarch64',),
78 'alpha': ('alpha', 'alphaev4', 'alphaev45', 'alphaev5',
'alphaev56',
79 'alphaev6', 'alphaev67', 'alphaev68', 'alphaev7',
'alphapca56'),
80 'arm': ('armv5tejl', 'armv5tel', 'armv5tl', 'armv6l',
'armv7l'),
81 'armhfp': ('armv6hl', 'armv7hl', 'armv7hnl',
'armv8l'),
82 'i386': ('i386', 'athlon', 'geode', 'i386', 'i486', 'i586',
'i686'),
83 'ia64': ('ia64',),
84 'mips': ('mips',),
85 'mipsel': ('mipsel',),
86 'mips64': ('mips64',),
87 'mips64el': ('mips64el',),
88 'noarch': ('noarch',),
89 'ppc': ('ppc',),
90 'ppc64': ('ppc64', 'ppc64iseries', 'ppc64p7',
'ppc64pseries'),
91 'ppc64le': ('ppc64le',),
92 'riscv32' : ('riscv32',),
93 'riscv64' : ('riscv64',),
94 'riscv128' : ('riscv128',),
95 's390': ('s390',),
96 's390x': ('s390x',),
97 'sh3': ('sh3',),
98 'sh4': ('sh4', 'sh4a'),
99 'sparc': ('sparc', 'sparc64', 'sparc64v', 'sparcv8',
'sparcv9',
100 'sparcv9v'),
101 'x86_64': ('x86_64', 'amd64',
'ia32e'),
102 })
If an arch does not exist in one of the lists associated with basearchs then
DNF will throw an error. Note that most of our basearchs have the basearch
name in the list of valid architectures.
The armhfp basearch does not contain its own name as a valid architecture
thus
DNF will error out.
Traceback (most recent call last):
File "./critpath.py", line 244, in
<module>
conf.arch = 'armhfp'
File "/usr/lib/python2.7/site-packages/dnf/conf/config.py", line 909,
in arch
raise dnf.exceptions.Error(msg.format("arch", val))
Studying the above dictionary of lists I concluded that I could probably
fix it
by submitting a PR to DNF that would add armhfp to the armhfp basearch list.
I tested the fix, and it worked except that it only came out with 3
packages in
the critical path from dependency solving. The i386 basearch had the same
number
which, under closer inspection, showed that both only came back with:
fedora-repos
fedora-release
setup
This seemed fishy; so I tried manually setting i686 as the architecture for
the i386 basearch and that yielded a more reasonable 1127 packages.
The dictionary of lists appears to put the lastest architecture name in the
last slot of the list so that led me to think that I could just send a PR
to DNF
that would add an API to return a list of architectures associated with a
given
basearch name and then use the last entry for my architecture.
Testing that theory it appears that armhfp basearch and armv8l arch also
only
yield the three default packages.
Thus I arrived at the code in scripts/critpath.py in my repo:
https://pagure.io/fork/kellin/releng/commits/update-critpath-intermediary
I looked in the fedora-comps and pungi-fedora repos; it seems to me that the
proper basearch is being associated to the packages. My instincts say this
is
more likely a metadata issue that one with DNF.
I'm OK with the code being sub-optimal as this is an intermediary step to a
more
elegant solution that wasn't going to be fully baked in time for the
September
4th Bodhi freeze.
My concerns are:
- this is a fragile hack as the architecture name that returns the right
packages is obviously subject to change given the behavior of i386
- I do not have certainty in my understanding of how packages get
associated
to architectures and base architectures and that I have all the right
information and pieces to determine a better longterm resolution
Any input and insight would be very much appreciated.
--Rob
6 years, 7 months
