Is there a specific yum or createrepo version I need to install on a CentOS 5.5 server so it can build packages for a child repository whose parent is Fedora 13?
mergerepos is exiting with an error: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error
I'm assuming this is because of the new sha256 checksums introduced a release or two back.
Jonathan
On Thu, 2010-06-17 at 21:31 +0300, Jonathan Dieter wrote:
Is there a specific yum or createrepo version I need to install on a CentOS 5.5 server so it can build packages for a child repository whose parent is Fedora 13?
mergerepos is exiting with an error: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error
I'm assuming this is because of the new sha256 checksums introduced a release or two back.
well if you're running createrepo -d
then chances are you need to check your version of yum-metadata-parser.
what do you see for versions of:
rpm -q createrepo yum yum-metadata-parser
-sv
On Thu, 2010-06-17 at 14:34 -0400, seth vidal wrote:
On Thu, 2010-06-17 at 21:31 +0300, Jonathan Dieter wrote:
Is there a specific yum or createrepo version I need to install on a CentOS 5.5 server so it can build packages for a child repository whose parent is Fedora 13?
<snip>
TypeError: Parsing primary.xml error: attributes construct error
I'm assuming this is because of the new sha256 checksums introduced a release or two back.
well if you're running createrepo -d
then chances are you need to check your version of yum-metadata-parser.
what do you see for versions of:
rpm -q createrepo yum yum-metadata-parser
-sv
$ rpm -q createrepo yum yum-metadata-parser createrepo-0.4.11-3.el5 yum-3.2.22-26.el5.centos yum-metadata-parser-1.1.2-3.el5.centos
On Thursday, June 17, 2010 01:31:54 pm Jonathan Dieter wrote:
Is there a specific yum or createrepo version I need to install on a CentOS 5.5 server so it can build packages for a child repository whose parent is Fedora 13?
mergerepos is exiting with an error: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error
I'm assuming this is because of the new sha256 checksums introduced a release or two back.
Jonathan
you need python-hashlib though koji should be pulling that in for you. you will need to get a rpm that has xz support to actually install the fedora-13 rpms there is one available inside of http://infrastructure.fedoraproject.org/builder-rpms/
Dennis
On Thu, 2010-06-17 at 14:06 -0500, Dennis Gilmore wrote:
you need python-hashlib though koji should be pulling that in for you. you will need to get a rpm that has xz support to actually install the fedora-13 rpms there is one available inside of http://infrastructure.fedoraproject.org/builder-rpms/
Ok, still getting the error message. I'm putting it, with slightly more context at the bottom.
$ rpm -q python-hashlib createrepo python-hashlib-20081119-4.el5.x86_64 createrepo-0.4.11-3.el5.noarch
I have updated rpm to the version available at the link above, but, as expected, it made no difference.
To give a bit more context, I've followed the instructions at http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap, using a local mirror, with updates having a priority of 1 and release having a priority of 2.
The error is happening on the newRepo task, which is run whenever I start kojira.
Jonathan
The last few lines of the error for x86_64, i386 is essentially the same: 16958/16961 - net6-1.3.9-3.fc12.x86_64 16959/16961 - junit-demo-3.8.2-6.4.fc12.x86_64 16960/16961 - perl-DateTime-Event-Recurrence-0.16-9.fc13.noarch 16961/16961 - perl-XML-XPath-1.13-10.fc13.noarch
Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Starting other db creation: Fri Jun 18 09:36:54 2010 Ending other db creation: Fri Jun 18 09:37:01 2010 Starting filelists db creation: Fri Jun 18 09:37:03 2010 Ending filelists db creation: Fri Jun 18 09:42:19 2010 Starting primary db creation: Fri Jun 18 09:42:22 2010 Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error
On Thu, 2010-06-17 at 21:31 +0300, Jonathan Dieter wrote:
Is there a specific yum or createrepo version I need to install on a CentOS 5.5 server so it can build packages for a child repository whose parent is Fedora 13?
If you are dealing with F-12+ repos. on RHEL5, install python-hashlib from EPEL. You may also want a newer createrepo, but the missing hashlib is the first thing to do.
..snip..
mergerepos is exiting with an error:
Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error
I'm assuming this is because of the new sha256 checksums introduced a release or two back.
This might be related or not, but a while ago I was grappling with the same error in mergerepos. The conclusion was that mergerepos couldn't handle certain unicode (or some random encoding. don't remember) which createrepo can handle. I manually ran the mergerepo command run by koji and looked at the the primary.xml file generated. Removing the packages that did unicode/random-encoding Provides/Requires solved the problem.
Jitesh
Jonathan
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On Fri, 2010-06-18 at 14:54 +0530, Jitesh Shah wrote:
..snip..
mergerepos is exiting with an error: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error I'm assuming this is because of the new sha256 checksums introduced a release or two back.This might be related or not, but a while ago I was grappling with the same error in mergerepos. The conclusion was that mergerepos couldn't handle certain unicode (or some random encoding. don't remember) which createrepo can handle. I manually ran the mergerepo command run by koji and looked at the the primary.xml file generated. Removing the packages that did unicode/random-encoding Provides/Requires solved the problem.
I suspect you're right, though it seems the bug is in createrepo. I've just upgraded to createrepo-0.9.8-5 (which meant rebuilding deltarpm for the updated for rpm-4.6.0, and then rebuilding yum-3.2.23-10 for el5), and the bug is gone, though now mergerepo ends with:
19133/16961 - junit-demo-3.8.2-6.4.fc12.x86_64 19134/16961 - rubygem-gettext_rails-doc-2.1.0-2.fc13.noarch 19135/16961 - perl-DateTime-Event-Recurrence-0.16-9.fc13.noarch 19136/16961 - bluez-compat-4.63-3.fc13.x86_64 19137/16961 - perl-XML-XPath-1.13-10.fc13.noarch
I guess it's the new way of doing math. :)
What version of createrepo and yum are on the builders?
Jonathan
On Fri, Jun 18, 2010 at 10:04 AM, Jonathan Dieter jdieter@lesbg.com wrote:
What version of createrepo and yum are on the builders?
[jstanley@x86-07 ~]$ rpm -q rpm yum createrepo rpm-4.6.0-4.0.notting.1.el5.x86_64 yum-3.2.22-26.el5.noarch createrepo-0.4.11-3.el5.noarch
On Fri, 2010-06-18 at 10:19 -0400, Jon Stanley wrote:
On Fri, Jun 18, 2010 at 10:04 AM, Jonathan Dieter jdieter@lesbg.com wrote:
What version of createrepo and yum are on the builders?
[jstanley@x86-07 ~]$ rpm -q rpm yum createrepo rpm-4.6.0-4.0.notting.1.el5.x86_64 yum-3.2.22-26.el5.noarch createrepo-0.4.11-3.el5.noarch
That's the builders - not the mashers/releng boxes.
rpm -qa | grep -E 'rpm|yum|createrepo' yum-metadata-parser-1.1.2-3.el5.i386 rpm-python-4.6.0-4.0.notting.1.el5.i386 rpm-4.6.0-4.0.notting.1.el5.i386 deltarpm-3.5-0.4.20090913git.el5.i386 yum-3.2.22-26.el5.noarch rpm-libs-4.6.0-4.0.notting.1.el5.i386 python-deltarpm-3.5-0.4.20090913git.el5.i386 createrepo-0.9.8-2.el5.noarch
-sv
On Fri, 2010-06-18 at 10:23 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 10:19 -0400, Jon Stanley wrote:
On Fri, Jun 18, 2010 at 10:04 AM, Jonathan Dieter jdieter@lesbg.com wrote:
What version of createrepo and yum are on the builders?
[jstanley@x86-07 ~]$ rpm -q rpm yum createrepo rpm-4.6.0-4.0.notting.1.el5.x86_64 yum-3.2.22-26.el5.noarch createrepo-0.4.11-3.el5.noarch
That's the builders - not the mashers/releng boxes.
rpm -qa | grep -E 'rpm|yum|createrepo' yum-metadata-parser-1.1.2-3.el5.i386 rpm-python-4.6.0-4.0.notting.1.el5.i386 rpm-4.6.0-4.0.notting.1.el5.i386 deltarpm-3.5-0.4.20090913git.el5.i386 yum-3.2.22-26.el5.noarch rpm-libs-4.6.0-4.0.notting.1.el5.i386 python-deltarpm-3.5-0.4.20090913git.el5.i386 createrepo-0.9.8-2.el5.noarch
Note that this seems to match enabling both:
http://infrastructure.fedoraproject.org/builder-rpms/i386/ http://infrastructure.fedoraproject.org/releng/i386/
On Fri, 2010-06-18 at 10:36 -0400, James Antill wrote:
On Fri, 2010-06-18 at 10:23 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 10:19 -0400, Jon Stanley wrote:
On Fri, Jun 18, 2010 at 10:04 AM, Jonathan Dieter jdieter@lesbg.com wrote:
What version of createrepo and yum are on the builders?
[jstanley@x86-07 ~]$ rpm -q rpm yum createrepo rpm-4.6.0-4.0.notting.1.el5.x86_64 yum-3.2.22-26.el5.noarch createrepo-0.4.11-3.el5.noarch
That's the builders - not the mashers/releng boxes.
rpm -qa | grep -E 'rpm|yum|createrepo' yum-metadata-parser-1.1.2-3.el5.i386 rpm-python-4.6.0-4.0.notting.1.el5.i386 rpm-4.6.0-4.0.notting.1.el5.i386 deltarpm-3.5-0.4.20090913git.el5.i386 yum-3.2.22-26.el5.noarch rpm-libs-4.6.0-4.0.notting.1.el5.i386 python-deltarpm-3.5-0.4.20090913git.el5.i386 createrepo-0.9.8-2.el5.noarch
Note that this seems to match enabling both:
http://infrastructure.fedoraproject.org/builder-rpms/i386/ http://infrastructure.fedoraproject.org/releng/i386/
I'd not recommend that.
And you'll need lzma/xz in rpm.
thats almost definitely the real trick.
-sv
On Fri, 2010-06-18 at 10:36 -0400, James Antill wrote:
Note that this seems to match enabling both:
http://infrastructure.fedoraproject.org/builder-rpms/i386/ http://infrastructure.fedoraproject.org/releng/i386/
That second link was the one that had the packages I wanted. Oh well, rebuilding wasn't that bad. :) Ok, I've finally managed to build my first package in my koji instance. Woo-hoo!
Now to track down why mash is deleting old deltarpms...
Jonathan
On Fri, 2010-06-18 at 10:23 -0400, seth vidal wrote:
rpm -qa | grep -E 'rpm|yum|createrepo'
<snip>
createrepo-0.9.8-2.el5.noarch
And that's the one I needed. Apparently there have been some large changes between 0.9.8-2 and 0.9.8-5 which have made the latter somewhat incompatible with mergerepo.
Thanks,
Jonathan
On Fri, 2010-06-18 at 17:46 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 10:23 -0400, seth vidal wrote:
rpm -qa | grep -E 'rpm|yum|createrepo'
<snip> > createrepo-0.9.8-2.el5.noarch
And that's the one I needed. Apparently there have been some large changes between 0.9.8-2 and 0.9.8-5 which have made the latter somewhat incompatible with mergerepo.
????
I'm just glancing through the git logs and I don't see anything serious or obviously incompatible.
some more info n the above claim would be helpful.
-sv
On Fri, 2010-06-18 at 10:52 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 17:46 +0300, Jonathan Dieter wrote:
Apparently there have been some large changes between 0.9.8-2 and 0.9.8-5 which have made the latter somewhat incompatible with mergerepo.
????
I'm just glancing through the git logs and I don't see anything serious or obviously incompatible.
some more info n the above claim would be helpful.
I may have been hasty in blaming createrepo. It may also be the difference between yum-3.2.22 and yum-3.2.23.
The newRepo task under createrepo-0.9.8-5 and yum-3.2.23 gave a total package count of 19137/16961 which seemed to indicate that it was a concatenation of release and updates rather than the latest packages.
More seriously (but probably yum rather than createrepo), I was getting
File "/usr/sbin/kojid", line 558, in getPackageList for h in ts.dbMatch(): TypeError: rpmdb open failed
at the end of a package build. Downgrading to yum-3.2.22 and createrepo-0.9.8-2 fixed both bugs.
Jonathan
On Fri, 2010-06-18 at 18:01 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 10:52 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 17:46 +0300, Jonathan Dieter wrote:
Apparently there have been some large changes between 0.9.8-2 and 0.9.8-5 which have made the latter somewhat incompatible with mergerepo.
????
I'm just glancing through the git logs and I don't see anything serious or obviously incompatible.
some more info n the above claim would be helpful.
I may have been hasty in blaming createrepo. It may also be the difference between yum-3.2.22 and yum-3.2.23.
The newRepo task under createrepo-0.9.8-5 and yum-3.2.23 gave a total package count of 19137/16961 which seemed to indicate that it was a concatenation of release and updates rather than the latest packages.
More seriously (but probably yum rather than createrepo), I was getting
File "/usr/sbin/kojid", line 558, in getPackageList for h in ts.dbMatch(): TypeError: rpmdb open failed
at the end of a package build. Downgrading to yum-3.2.22 and createrepo-0.9.8-2 fixed both bugs.
yum 3.2.22 from upstream? or yum 3.2.22 from rhel?
b/c 3.2.22 from rhel has A LOT of patches.
-sv
On Fri, 2010-06-18 at 11:12 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:01 +0300, Jonathan Dieter wrote:
<snip>
at the end of a package build. Downgrading to yum-3.2.22 and createrepo-0.9.8-2 fixed both bugs.
yum 3.2.22 from upstream? or yum 3.2.22 from rhel?
b/c 3.2.22 from rhel has A LOT of patches.
3.2.22 from rhel and 3.2.23 from Fedora (I think Fedora 12, got it from koji).
Jonathan
On Fri, 2010-06-18 at 18:16 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 11:12 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:01 +0300, Jonathan Dieter wrote:
<snip> > > at the end of a package build. Downgrading to yum-3.2.22 and > > createrepo-0.9.8-2 fixed both bugs. > > > > yum 3.2.22 from upstream? or yum 3.2.22 from rhel? > > b/c 3.2.22 from rhel has A LOT of patches.
3.2.22 from rhel and 3.2.23 from Fedora (I think Fedora 12, got it from koji).
3.2.22 from rhel is a lot closer to 3.2.25, really.
not _quite_ there - but much closer.
-sv
On Fri, 2010-06-18 at 11:20 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:16 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 11:12 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:01 +0300, Jonathan Dieter wrote:
<snip> > > at the end of a package build. Downgrading to yum-3.2.22 and > > createrepo-0.9.8-2 fixed both bugs. > > > > yum 3.2.22 from upstream? or yum 3.2.22 from rhel? > > b/c 3.2.22 from rhel has A LOT of patches.
3.2.22 from rhel and 3.2.23 from Fedora (I think Fedora 12, got it from koji).
3.2.22 from rhel is a lot closer to 3.2.25, really.
not _quite_ there - but much closer.
Ok, then that's probably what the main problem was. I had grabbed 3.2.23 because createrepo required it, but when I saw that you had rhel's 3.2.22 with createrepo 0.9.8, I figured you knew what you were doing and changed the requirements when I rebuilt createrepo-0.9.8-2 for el5.
Thanks again for the help. Should I update http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap to mention needing some packages from http://infrastructure.fedoraproject.org/releng?
Jonathan
On Fri, 2010-06-18 at 18:34 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 11:20 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:16 +0300, Jonathan Dieter wrote:
On Fri, 2010-06-18 at 11:12 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:01 +0300, Jonathan Dieter wrote:
<snip> > > at the end of a package build. Downgrading to yum-3.2.22 and > > createrepo-0.9.8-2 fixed both bugs. > > > > yum 3.2.22 from upstream? or yum 3.2.22 from rhel? > > b/c 3.2.22 from rhel has A LOT of patches.
3.2.22 from rhel and 3.2.23 from Fedora (I think Fedora 12, got it from koji).
3.2.22 from rhel is a lot closer to 3.2.25, really.
not _quite_ there - but much closer.
Ok, then that's probably what the main problem was. I had grabbed 3.2.23 because createrepo required it, but when I saw that you had rhel's 3.2.22 with createrepo 0.9.8, I figured you knew what you were doing and changed the requirements when I rebuilt createrepo-0.9.8-2 for el5.
Thanks again for the help. Should I update http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap to mention needing some packages from http://infrastructure.fedoraproject.org/releng?
The pkgs from releng are really only needed if you're going to be messing with pkgs made with rpm versions > than the ones in rhel5.
-sv
On Fri, 2010-06-18 at 11:39 -0400, seth vidal wrote:
On Fri, 2010-06-18 at 18:34 +0300, Jonathan Dieter wrote:
Thanks again for the help. Should I update http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap to mention needing some packages from http://infrastructure.fedoraproject.org/releng?
The pkgs from releng are really only needed if you're going to be messing with pkgs made with rpm versions > than the ones in rhel5.
Yeah, but that's any Fedora release >= 12, right? Anyhow, it's also based on the assumption that you're installing koji onto rhel5, so maybe it's not that relevant for that wiki page.
Jonathan
On Friday, June 18, 2010 09:04:51 am Jonathan Dieter wrote:
On Fri, 2010-06-18 at 14:54 +0530, Jitesh Shah wrote:
..snip..
mergerepos is exiting with an error: Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 229, in ? main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 224, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 205, in write_metadata mdgen.doRepoMetadata() File "/usr/lib/python2.4/site-packages/createrepo/__init__.py", line 789, in doRepoMetadata rp.getPrimary(complete_path, csum) File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary self.repoid)) TypeError: Parsing primary.xml error: attributes construct error I'm assuming this is because of the new sha256 checksums introduced a release or two back.This might be related or not, but a while ago I was grappling with the same error in mergerepos. The conclusion was that mergerepos couldn't handle certain unicode (or some random encoding. don't remember) which createrepo can handle. I manually ran the mergerepo command run by koji and looked at the the primary.xml file generated. Removing the packages that did unicode/random-encoding Provides/Requires solved the problem.
I suspect you're right, though it seems the bug is in createrepo. I've just upgraded to createrepo-0.9.8-5 (which meant rebuilding deltarpm for the updated for rpm-4.6.0, and then rebuilding yum-3.2.23-10 for el5), and the bug is gone, though now mergerepo ends with:
19133/16961 - junit-demo-3.8.2-6.4.fc12.x86_64 19134/16961 - rubygem-gettext_rails-doc-2.1.0-2.fc13.noarch 19135/16961 - perl-DateTime-Event-Recurrence-0.16-9.fc13.noarch 19136/16961 - bluez-compat-4.63-3.fc13.x86_64 19137/16961 - perl-XML-XPath-1.13-10.fc13.noarch
I guess it's the new way of doing math. :)
What version of createrepo and yum are on the builders?
Jonathan
yum-3.2.22-26.el5.noarch python-createrepo-0.9.8-1.el5.noarch createrepo-0.4.11-3.el5.noarch
the version of createrepo that ships in rhel. mergerepos imports createrepo for that we use python-createrepo from epel. note in fedora's koji the only thing we use external repos for is the rhel repos used to populate epel build roots
Dennis
buildsys@lists.fedoraproject.org
-
Dennis Gilmore -
James Antill -
Jitesh Shah -
Jon Stanley -
Jonathan Dieter -
Jonathan Dieter -
Seth Vidal