On Wed, 2016-06-29 at 15:02 +0100, Daniel P. Berrange wrote:
On Wed, Jun 29, 2016 at 09:39:11AM -0400, Simo Sorce wrote:
> On Wed, 2016-06-29 at 11:34 +0100, Daniel P. Berrange wrote:
> > On Wed, Jun 29, 2016 at 12:15:02PM +0200, Florian Weimer wrote:
> > > On 06/29/2016 12:03 PM, Daniel P. Berrange wrote:
> > > > Debian handles this by having several packages [1]
> > > >
> > > > - qemu-user - the dynamic linked qemu user binaries
> > > > - qemu-binfmt - binfmt rules registering the dynamic linked
binaries
> > > > - qemu-user-static - the static linked qemu user binaries *and*
binfmt
> > > > rules to register them. The static binaries
all
> > > > have -static suffix on their name
> > >
> > > Debian builds static libraries and puts them into their -dev packages.
> > > Fedora does not do this systematically. Are all the static libraries you
> > > need actually available in Fedora?
> >
> > Yes, everything we need exists - as mentioned its only glibc-static,
> > glib2-static, zlib-static and pcre-static that we need for this.
> >
> > > An alternative would be to bind-mount the real file system into the
chroot
> > > and run the native dynamic linker with a --library-path command line
> > > argument that specifies the library directories in the bind mount.
(It's
> > > reasonable to specify --inhibit-cache as well.) This would need some
> > > adjustments in the binfmt wrapper.
> >
> > The binfmt registrations are global to the OS, so the same binfmt command
> > needs to work whether inside or outside the chroot. So a scheme which
> > requires us to pass special args to make the linker looks elsewhere when
> > inside the chroot is not so easy. We'd have to create a wrapper program
> > around the real qemu user binary that decides which args to pass to the
> > linker, and that wrapper would then have to be statically linked
>
> Just an idea:
>
> Would it make sense to build the qemu-user binary so that it looks in a
> different /lib directory (using rpath) like /lib/qemu-user-systemlib,
> and then symlink into that directory the libraries it needs from the
> host ?
>
> That way if you use chroot and bind mount in that directory in the right
> place the qemu-user binary uses different libraries from the emulated
> binaries yet you do not have to resort to static linking.
I think you would need multiple levels of symlinking
On the host root
symlink: /usr/lib/qemu-user-root -> /
dir: /usr/lib/qemu-user-host-lib
symlink: /usr/lib/qemu-user-host-lib/libc.so -> /usr/lib/qemu-user-root/lib/libc.so
And build qemu-$ARCH with rpath to /usr/lib/qemu-user-root/lib.
Then in the chroot you would need to bind mount the host root into
/usr/lib/qemu-user-root, to override the symlink that would otherwise
point back to /
Why do you need 2 symlinks ?
I was thinking you'd have
symlink: /usr/lib/qemu-user-lib-x86_64/ -> /usr/lib
In the chroot you just bind mount the host's /usr/lib
as /usr/lib/qemu-user-lib-x86_64
This assuming x86-64, you can bind mount in the appropriate arch-target
dir on other arches.
Even if you do all that, you've now prevented installation of
the
foreign arch's qemu-user RPM inside the chroot,
See above, I think my solution would not prevent that.
as that will clash
with the one you've setup from the host. Also the way you deal with
cross-arch chroots is now different (and more complex) on Fedora, vs
every other Linux distro which just provides a static qemu-$ARCH you
can copy without needing to care about bind mounting extra dirs.
Yes, bind-mount is an additional step, but doesn't look like a huge
price to pay. Could even be done transparently by providing a
qemu-user.sh script that "does the right thing"
So yes, it is probably possible, but it is not very appealing IMHO
Thought it may be neat, but if consistency with other distros is what
you are after then I would also lean to build a static binary.
It's just that linking in glibc statically is ... not great.
Simo.
--
Simo Sorce * Red Hat, Inc * New York