10:50 a.m.
Hello,
I'm trying to setup a mock chroot for usage with the epel-5-i386
target. I had considered mach, which I have already used in the past,
but since mock seems newer, better mantained an more integrated with
fedora and some other tools which look to me very useful and promising
( Cobbler, Koan, Koji ), I wanted to give it a shot.
But I seem to have an issue with it. I started with a fresh FC11
install, with the updates repo enabled, and installed mock from the
repos;
version 0.9.17 installed cleanly, and seems to be the latest version.
My issue seems related to different rpmdb versions inside and outside
the chroot which gets created. Take the following output:
It seems the rpmdb of the chroot has been created with an rpm
employing a different format ( I can assume it's the 'host' system rpm
), hence leading to a format mismatch which prevents from using rpmdb
from inside the chroot.
I have tinkered around for a little, but I couldn't find an easy
solution (beyond installing everything from outside using mock
--install, of course). Hence I've got some questions:
Is that an intended behaviour? Is the rpmdb supposed to be converted
back to the proper format later on in the deployment process, if using
mock for building packages which be used later on in a possibly
pre-build system?
Alan Franzoni
10:54 a.m.
I think I've forgot mock output :-/
[user@cobblerserver mock]$ mock --init -r epel-5-i386
INFO: mock.py version 0.9.17 starting...
State Changed: init plugins
State Changed: start
State Changed: lock buildroot
State Changed: clean
State Changed: init
State Changed: lock buildroot
Mock Version: 0.9.17
INFO: Mock Version: 0.9.17
INFO: enabled root cache
INFO: enabled yum cache
State Changed: cleaning yum metadata
INFO: enabled ccache
State Changed: running yum
State Changed: creating cache
[user@cobblerserver mock]$ mock --shell -r epel-5-i386
INFO: mock.py version 0.9.17 starting...
State Changed: init plugins
State Changed: start
State Changed: lock buildroot
mock-chroot> rpm -qa
rpmdb: /var/lib/rpm/Packages: unsupported hash version: 9
error: cannot open Packages index using db3 - Invalid argument (22)
error: cannot open Packages database in /var/lib/rpm
mock-chroot> rpm --rebuilddb
rpmdb: /var/lib/rpm/Packages: unsupported hash version: 9
error: cannot open Packages index using db3 - Invalid argument (22)
mock-chroot> exit
11:26 a.m.
On 09/14/2009 11:50 AM, Alan Franzoni (mailing) wrote:
It seems the rpmdb of the chroot has been created with an rpm
employing a different format ( I can assume it's the 'host' system rpm
), hence leading to a format mismatch which prevents from using rpmdb
from inside the chroot.
I have tinkered around for a little, but I couldn't find an easy
solution (beyond installing everything from outside using mock
--install, of course). Hence I've got some questions:
The chroot is created from outside the chroot, and hence uses that
version of rpm. Consider that when the process starts, there is no rpm
in the chroot to use.
Note that mock does make it easy to install things manually in the
chroot from the outside.
Is that an intended behaviour?
intended may be reaching, but known, expected, unavoidable -- yes
Is the rpmdb supposed to be converted
back to the proper format later on in the deployment process, if using
mock for building packages which be used later on in a possibly
pre-build system?
There is no back conversion. I'm not sure how this should affect any
deployment process for the resulting builds. The rpmdb in the chroot
should generally not be accessed by the build itself. Any such activity
in a spec file is unwise and questionable.
It is unfortunate that this incompatibility was introduced in rpm, but
it was for a good reason -- sha256 sums replaced semi-insecure md5sums.
One possibility, if you are only building for epel-5 and earlier, is to
use an older version of rpm on the host, perhaps just a stock epel-5
install. You'll be unable to build for Fedora 11+ on such a host, though.
12:13 p.m.
On Mon, 2009-09-14 at 12:26 -0400, Mike McLean wrote:
It is unfortunate that this incompatibility was introduced in rpm,
but
it was for a good reason -- sha256 sums replaced semi-insecure md5sums.
Note that you'll have rpmdb mismatches even when creating an EL5 chroot
on EL5, if you create an i386 chroot on an x86_64 host. Just that
difference is enough to cause rpmdb mismatches. A work around if you
must work within the chroot is to remove the db cache, rm
-f /var/lib/rpm/__db*
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
1:09 p.m.
2009/9/14 Mike McLean <mikem(a)redhat.com>:
The chroot is created from outside the chroot, and hence uses that
version
of rpm. Consider that when the process starts, there is no rpm in the chroot
to use.
Yes, I had noticed that.
There is no back conversion. I'm not sure how this should affect
any
deployment process for the resulting builds. The rpmdb in the chroot should
generally not be accessed by the build itself. Any such activity in a spec
file is unwise and questionable.
Sure it is, and such activity is not performed inside any of any SPEC.
I think that maybe I just asked the wrong question.
One of my targets is to build SRPMs inside an epel-5-i386 chroot; this
can be done pretty easily with mock, but what if I have a dependency
on an RPM which is not available in a yum repo? I'd like to test some
internally built rpms in a pseudo-production environment *before*
they're sent to our repo. So I just wanted to use "--copyin", then use
--shell to manually invoke rpm and manually install such packages.
Also, sometimes some packages have build dependencies that can't be
satisfied by repos and should be installed manually (again, dependency
on a test package).
If it's the quickest path I'll just setup an internal, testing only,
can-be-trashed-at-any-time repository, possibly on the vary same
machine hosting mock.
Then I'll also need to build "full" images for some machines to deploy
(e.g. a gzipped tar which holds the full operating system along with
some rpms) in our testing environment, but I suppose I can use
cobbler/koan for that, and I imagine they work differently and support
a working rpm inside?
--
Alan Franzoni
--
contact me at public(a)[mysurname].eu
1:29 p.m.
On 09/14/2009 12:13 PM, Jesse Keating wrote:
On Mon, 2009-09-14 at 12:26 -0400, Mike McLean wrote:
> It is unfortunate that this incompatibility was introduced in rpm, but
> it was for a good reason -- sha256 sums replaced semi-insecure md5sums.
Note that you'll have rpmdb mismatches even when creating an EL5 chroot
on EL5, if you create an i386 chroot on an x86_64 host. Just that
difference is enough to cause rpmdb mismatches. A work around if you
must work within the chroot is to remove the db cache, rm
-f /var/lib/rpm/__db*
With F11, I have found that I also had to remove /var/lib/rpm/Packages
when doing anything in an EL5 chroot that mucks with the rpmdb. e.g.
Building ISOs with Revisor. (My Makefile also does '/bin/rpm
--rebuilddb' before executing Revisor)
--
---
Paul B Schroeder <paul.schroeder "at" bluecoat "dot" com>
Blue Coat Systems, Inc.
1:38 p.m.
On 09/14/2009 02:09 PM, Alan Franzoni wrote:
> There is no back conversion. I'm not sure how this should
affect any
> deployment process for the resulting builds. The rpmdb in the chroot should
> generally not be accessed by the build itself. Any such activity in a spec
> file is unwise and questionable.
Sure it is, and such activity is not performed inside any of any SPEC.
Sorry, was just guessing. I have seen folks try to read the rpmdb at
build time before.
One of my targets is to build SRPMs inside an epel-5-i386 chroot;
this
can be done pretty easily with mock, but what if I have a dependency
on an RPM which is not available in a yum repo? I'd like to test some
internally built rpms in a pseudo-production environment *before*
they're sent to our repo. So I just wanted to use "--copyin", then use
--shell to manually invoke rpm and manually install such packages.
Also, sometimes some packages have build dependencies that can't be
satisfied by repos and should be installed manually (again, dependency
on a test package).
You could just use the rpm command line with --root to install from
outside the chroot. If you want to make sure all the mounts that mock
creates are in place, the easiest way is to use mock --shell in one
window, leave that idle, and run the rpm --root (or perhaps yum
--installroot $d localinstall $f) command in another window (outside the
chroot). Hackish, but, functional.
I suppose we should add a way to access the yum localinstall
functionality through mock.
If it's the quickest path I'll just setup an internal,
testing only,
can-be-trashed-at-any-time repository, possibly on the vary same
machine hosting mock.
That may be easier if you do this often. I'm sure you're aware, but just
for completeness note that you can include more than one repo in the
mock config, allowing this testing repo to be a small add-on (and hence
a very quick run of createrepo).
1:43 p.m.
On Monday 14 September 2009 01:09:26 pm Alan Franzoni wrote:
2009/9/14 Mike McLean <mikem(a)redhat.com>:
> The chroot is created from outside the chroot, and hence uses that
> version of rpm. Consider that when the process starts, there is no rpm in
> the chroot to use.
Yes, I had noticed that.
> There is no back conversion. I'm not sure how this should affect any
> deployment process for the resulting builds. The rpmdb in the chroot
> should generally not be accessed by the build itself. Any such activity
> in a spec file is unwise and questionable.
Sure it is, and such activity is not performed inside any of any SPEC.
I think that maybe I just asked the wrong question.
One of my targets is to build SRPMs inside an epel-5-i386 chroot; this
can be done pretty easily with mock, but what if I have a dependency
on an RPM which is not available in a yum repo? I'd like to test some
internally built rpms in a pseudo-production environment *before*
they're sent to our repo. So I just wanted to use "--copyin", then use
--shell to manually invoke rpm and manually install such packages.
Also, sometimes some packages have build dependencies that can't be
satisfied by repos and should be installed manually (again, dependency
on a test package).
build your srpm with --nodeps that way you dont need the
buildrequires
packages installed to make your srpm. this is how koji does it.
If it's the quickest path I'll just setup an internal, testing only,
can-be-trashed-at-any-time repository, possibly on the vary same
machine hosting mock.
setup a local repo. is the easiest way.
Then I'll also need to build "full" images for some
machines to deploy
(e.g. a gzipped tar which holds the full operating system along with
some rpms) in our testing environment, but I suppose I can use
cobbler/koan for that, and I imagine they work differently and support
a working rpm inside?
1:51 p.m.
On 09/14/2009 01:13 PM, Jesse Keating wrote:
Note that you'll have rpmdb mismatches even when creating an EL5
chroot
on EL5, if you create an i386 chroot on an x86_64 host. Just that
difference is enough to cause rpmdb mismatches. A work around if you
must work within the chroot is to remove the db cache, rm
-f /var/lib/rpm/__db*
Of course, that is a smaller level of incompatibility since those files
are so safely removable (as long as there is not a transaction running).
I've encountered many issues with one version not liking another
version's __db files over the years, but this is the first time since
RHL6.2 that I recall the main database files being incompatible like this.
4:13 p.m.
>>>> "MM" == Mike McLean
<mikem(a)redhat.com> writes:
MM> I suppose we should add a way to access the yum localinstall
MM> functionality through mock.
I have done this for years:
echo Installing built packages:
runmock -v --install $MOCKDIR/result/*{i386,x86_64,noarch}.rpm 2>&1
(dependent on the rest of the script, of course, but you get the idea).
- J<
9:31 a.m.
On 09/14/2009 05:13 PM, Jason L Tibbitts III wrote:
MM> I suppose we should add a way to access the yum localinstall
MM> functionality through mock.
I have done this for years:
echo Installing built packages:
runmock -v --install $MOCKDIR/result/*{i386,x86_64,noarch}.rpm 2>&1
(dependent on the rest of the script, of course, but you get the idea).
Ah, apparently yum now (and perhaps for a while) handles local rpms via
the install command as well as the localinstall command. Hence, you can
access this functionality via mock.
At one point, yum did not do this and you had to use localinstall for
local rpms.
Good to know. Thanks :)
9:35 a.m.
On Tue, 15 Sep 2009, Mike McLean wrote:
On 09/14/2009 05:13 PM, Jason L Tibbitts III wrote:
> MM> I suppose we should add a way to access the yum localinstall
> MM> functionality through mock.
>
> I have done this for years:
>
> echo Installing built packages:
> runmock -v --install $MOCKDIR/result/*{i386,x86_64,noarch}.rpm 2>&1
>
> (dependent on the rest of the script, of course, but you get the idea).
Ah, apparently yum now (and perhaps for a while) handles local rpms via the
install command as well as the localinstall command. Hence, you can access
this functionality via mock.
At one point, yum did not do this and you had to use localinstall for local
rpms.
Good to know. Thanks :)
localinstalls still care about gpgsignatures, though - so you make sure
you have the right keys or turn off gpg checking.
-sv
9:36 a.m.
2009/9/15 Mike McLean <mikem(a)redhat.com>:
[cut]
I think I got all the info I need, and I wish to thank you all for
your excellent and prompt support!
--
Alan Franzoni
--
contact me at public(a)[mysurname].eu
2:40 a.m.
On Mon, 14 Sep 2009, Mike McLean wrote:
On 09/14/2009 11:50 AM, Alan Franzoni (mailing) wrote:
> It seems the rpmdb of the chroot has been created with an rpm
> employing a different format ( I can assume it's the 'host' system rpm
> ), hence leading to a format mismatch which prevents from using rpmdb
> from inside the chroot.
>
> I have tinkered around for a little, but I couldn't find an easy
> solution (beyond installing everything from outside using mock
> --install, of course). Hence I've got some questions:
The chroot is created from outside the chroot, and hence uses that version of
rpm. Consider that when the process starts, there is no rpm in the chroot to
use.
Note that mock does make it easy to install things manually in the chroot
from the outside.
> Is that an intended behaviour?
intended may be reaching, but known, expected, unavoidable -- yes
> Is the rpmdb supposed to be converted
> back to the proper format later on in the deployment process, if using
> mock for building packages which be used later on in a possibly
> pre-build system?
There is no back conversion. I'm not sure how this should affect any
deployment process for the resulting builds. The rpmdb in the chroot should
generally not be accessed by the build itself. Any such activity in a spec
file is unwise and questionable.
It is unfortunate that this incompatibility was introduced in rpm, but it was
for a good reason -- sha256 sums replaced semi-insecure md5sums.
Mind you, the error here has nothing to do with sha256/md5:
rpmdb: /var/lib/rpm/Packages: unsupported hash version: 9
error: cannot open Packages index using db3 - Invalid argument (22)
error: cannot open Packages database in /var/lib/rpm
This is a much lower level thing, a Berkeley DB internal format
incompatibility: BDB can always read (up to a point) databases created
with older versions, but there's no guarantee the other way around. For
example BDB 4.3 and 4.5 were interchangably compatible but BDB 4.7 (which
is what RPM in Fedora 11 uses) is not.
The rpm-level view of the database hasn't really changed since the very
beginning - it's just a pile of header binary blobs and some associated
indexes. It is in fact possible to manually downgrade the database to and
older BDB format by using db_dump and db_load of appropriate BDB versions,
it's just not something that can be automatically done by RPM. So while
not exactly practical, you can install say RHEL 5 package set with Fedora
11 RPM, manually downgrade the BDB format to RHEL 5 RPM's level with BDB
tools, do rpm --rebuilddb with the RHEL 5 rpm to recreate indexes and
voila, you have fully "converted the db for later deployment".
And then there's the third level of compatibility: the data contained
within headers and it's interpretation. The md5 vs sha256 (or actually,
the algorithm is configurable) is just interpretation of tag values, the
data types of these are recognizable and accessible by pretty much any old
RPM you can dig up, but those older rpm versions wouldn't know what to do
with this tag even though they can load such a header. A more fundamental
issue is header data types unknown to older versions: RPM 4.6 introduced
64bit integer type, and headers containing such data cannot be even loaded
by older RPM versions as internal consistency checks cause failure on
unknown data types. Because this is such a grave incompatibility, RPM
avoids using 64bit types in headers when they're strictly not needed, ie
if 32bit integer is sufficient to contain the package/file sizes. To my
knowledge, there are no > 4GB packages in Fedora so this is not an issue
so far.
- Panu -
5329
days inactive
5331
days old
buildsys@lists.fedoraproject.org
13 comments
8 participants
participants (8)
-
Alan Franzoni -
Dennis Gilmore -
Jason L Tibbitts III -
Jesse Keating -
Mike McLean -
Panu Matilainen -
Paul B Schroeder -
Seth Vidal