On Wed, Jun 29, 2016 at 09:39:11AM -0400, Simo Sorce wrote:
On Wed, 2016-06-29 at 11:34 +0100, Daniel P. Berrange wrote:
On Wed, Jun 29, 2016 at 12:15:02PM +0200, Florian Weimer wrote:
On 06/29/2016 12:03 PM, Daniel P. Berrange wrote:
Debian handles this by having several packages [1]
- qemu-user - the dynamic linked qemu user binaries
- qemu-binfmt - binfmt rules registering the dynamic linked binaries
- qemu-user-static - the static linked qemu user binaries *and* binfmt rules to register them. The static binaries all have -static suffix on their name
Debian builds static libraries and puts them into their -dev packages. Fedora does not do this systematically. Are all the static libraries you need actually available in Fedora?
Yes, everything we need exists - as mentioned its only glibc-static, glib2-static, zlib-static and pcre-static that we need for this.
An alternative would be to bind-mount the real file system into the chroot and run the native dynamic linker with a --library-path command line argument that specifies the library directories in the bind mount. (It's reasonable to specify --inhibit-cache as well.) This would need some adjustments in the binfmt wrapper.
The binfmt registrations are global to the OS, so the same binfmt command needs to work whether inside or outside the chroot. So a scheme which requires us to pass special args to make the linker looks elsewhere when inside the chroot is not so easy. We'd have to create a wrapper program around the real qemu user binary that decides which args to pass to the linker, and that wrapper would then have to be statically linked
Just an idea:
Would it make sense to build the qemu-user binary so that it looks in a different /lib directory (using rpath) like /lib/qemu-user-systemlib, and then symlink into that directory the libraries it needs from the host ?
That way if you use chroot and bind mount in that directory in the right place the qemu-user binary uses different libraries from the emulated binaries yet you do not have to resort to static linking.
I think you would need multiple levels of symlinking
On the host root
symlink: /usr/lib/qemu-user-root -> / dir: /usr/lib/qemu-user-host-lib symlink: /usr/lib/qemu-user-host-lib/libc.so -> /usr/lib/qemu-user-root/lib/libc.so
And build qemu-$ARCH with rpath to /usr/lib/qemu-user-root/lib.
Then in the chroot you would need to bind mount the host root into /usr/lib/qemu-user-root, to override the symlink that would otherwise point back to /
Even if you do all that, you've now prevented installation of the foreign arch's qemu-user RPM inside the chroot, as that will clash with the one you've setup from the host. Also the way you deal with cross-arch chroots is now different (and more complex) on Fedora, vs every other Linux distro which just provides a static qemu-$ARCH you can copy without needing to care about bind mounting extra dirs.
So yes, it is probably possible, but it is not very appealing IMHO
Regards, Daniel