Greetings fellow secondary arch folks,
As some of you might know, we are working to introduce a new architecture called ppc64le into Fedora. It's just as it sounds, ppc64 (64bit only) but little endian. We've been emulating the Fedora process as close as possible internally here and have quite a few packages built, including using things like mock and koji.
As with introducing any new architecture, one of the issues we are seeing is that current packages lack the libtool (config.guess|sub,m4) magic to recognize the architecture. While rpmbuild and mock do properly influence build and targets, the one area we still a high rate of failure with is when configure asks the linker if it supports shared libs. In our case, it actually asks the linker with the wrong architecture.
In an ideal situation, libtool would release a new version with the ppc64le content (which is already upstream with them) and ALL upstream maintainers would immediately add the new content and drop new releases. That's ideal but not realistic.
I also would prefer to NOT update every Fedora package/.spec with a patch to fix this and would prefer to have this be the rare exception rather than the rule.
So in the interest of beginning work in Fedora and not making a mess of things, I'd like to propose another alternative (and solicit more ideas as well). In extending the case where configure “asks” the linker if it supports shared libraries, it actually will pass an incorrect architecture. Using lcms2's configure as an example:
checking whether the gcc -std=gnu99 linker (/bin/ld -m elf64ppc) supports shared libraries... no
In our internal development here, we've patched the linker to always respond as if it was asked about the ppc64le architecture. This function is enabled as an environment variable (LD_FORCE_LE=1). We've been using this for several months and I have yet to encounter an instance where something bad happens.
So I'd like to propose that for initial Fedora ppc64le development, we enable this patch for the linker, at least for the first release. Meanwhile, we can identify and manage the list of packages that need updated libtool information on the side. We could then identify a good cutover point and disable the linker patch in a future release.
If this could be helpful for other new arches like aarch64, we could share the this patch as well.
I believe the Fedora PPC64 team is supportive of this (at least for the ppc64le arch) but we wanted to just solicit broader input.
Feedback appreciated,
Brent
Brent J. Baude Linux Technology Center 3605 Hwy 52N Rochester, MN 55901
http://www.ibm.com/linux/ltc/ 507.253-0708
On Wed, 23 Oct 2013 16:09:36 -0500 Brent Baude baude@us.ibm.com wrote:
Greetings fellow secondary arch folks,
As some of you might know, we are working to introduce a new architecture called ppc64le into Fedora. It's just as it sounds, ppc64 (64bit only) but little endian. We've been emulating the Fedora process as close as possible internally here and have quite a few packages built, including using things like mock and koji.
As with introducing any new architecture, one of the issues we are seeing is that current packages lack the libtool (config.guess| sub,m4) magic to recognize the architecture. While rpmbuild and mock do properly influence build and targets, the one area we still a high rate of failure with is when configure asks the linker if it supports shared libs. In our case, it actually asks the linker with the wrong architecture.
Do you need full updated libtool or having recent enough config.guess/config.sub will be sufficient? Currently redhat-rpm-config provides them and they are used to replace the project's upstream ones when the spec file calls the %configure macro. This was done in support for aarch64 instead of requiring running autoreconf or patching the autotools files.
Dan
In an ideal situation, libtool would release a new version with the ppc64le content (which is already upstream with them) and ALL upstream maintainers would immediately add the new content and drop new releases. That's ideal but not realistic.
I also would prefer to NOT update every Fedora package/.spec with a patch to fix this and would prefer to have this be the rare exception rather than the rule.
So in the interest of beginning work in Fedora and not making a mess of things, I'd like to propose another alternative (and solicit more ideas as well). In extending the case where configure “asks” the linker if it supports shared libraries, it actually will pass an incorrect architecture. Using lcms2's configure as an example:
checking whether the gcc -std=gnu99 linker (/bin/ld -m elf64ppc) supports shared libraries... no
In our internal development here, we've patched the linker to always respond as if it was asked about the ppc64le architecture. This function is enabled as an environment variable (LD_FORCE_LE=1). We've been using this for several months and I have yet to encounter an instance where something bad happens.
So I'd like to propose that for initial Fedora ppc64le development, we enable this patch for the linker, at least for the first release. Meanwhile, we can identify and manage the list of packages that need updated libtool information on the side. We could then identify a good cutover point and disable the linker patch in a future release.
If this could be helpful for other new arches like aarch64, we could share the this patch as well.
I believe the Fedora PPC64 team is supportive of this (at least for the ppc64le arch) but we wanted to just solicit broader input.
Feedback appreciated,
Brent
Brent J. Baude Linux Technology Center 3605 Hwy 52N Rochester, MN 55901
http://www.ibm.com/linux/ltc/ 507.253-0708
Hi Dan,
Indeed I do observe where that occurs but, as you note, it does depend on the .specs being a certain way. So in those cases, we don't need a work-around. But there are still many cases where we do; hence the proposal.
Brent J. Baude Linux Technology Center 3605 Hwy 52N Rochester, MN 55901
http://www.ibm.com/linux/ltc/ 507.253-0708
From: Dan Horák dan@danny.cz To: secondary@lists.fedoraproject.org, Date: 10/23/2013 04:25 PM Subject: Re: ppc64le architecture Sent by: secondary-bounces@lists.fedoraproject.org
On Wed, 23 Oct 2013 16:09:36 -0500 Brent Baude baude@us.ibm.com wrote:
Greetings fellow secondary arch folks,
As some of you might know, we are working to introduce a new architecture called ppc64le into Fedora. It's just as it sounds, ppc64 (64bit only) but little endian. We've been emulating the Fedora process as close as possible internally here and have quite a few packages built, including using things like mock and koji.
As with introducing any new architecture, one of the issues we are seeing is that current packages lack the libtool (config.guess| sub,m4) magic to recognize the architecture. While rpmbuild and mock do properly influence build and targets, the one area we still a high rate of failure with is when configure asks the linker if it supports shared libs. In our case, it actually asks the linker with the wrong architecture.
Do you need full updated libtool or having recent enough config.guess/config.sub will be sufficient? Currently redhat-rpm-config provides them and they are used to replace the project's upstream ones when the spec file calls the %configure macro. This was done in support for aarch64 instead of requiring running autoreconf or patching the autotools files.
Dan
In an ideal situation, libtool would release a new version with the ppc64le content (which is already upstream with them) and ALL upstream maintainers would immediately add the new content and drop new releases. That's ideal but not realistic.
I also would prefer to NOT update every Fedora package/.spec with a patch to fix this and would prefer to have this be the rare exception rather than the rule.
So in the interest of beginning work in Fedora and not making a mess of things, I'd like to propose another alternative (and solicit more ideas as well). In extending the case where configure “asks” the linker if it supports shared libraries, it actually will pass an incorrect architecture. Using lcms2's configure as an example:
checking whether the gcc -std=gnu99 linker (/bin/ld -m elf64ppc) supports shared libraries... no
In our internal development here, we've patched the linker to always respond as if it was asked about the ppc64le architecture. This function is enabled as an environment variable (LD_FORCE_LE=1). We've been using this for several months and I have yet to encounter an instance where something bad happens.
So I'd like to propose that for initial Fedora ppc64le development, we enable this patch for the linker, at least for the first release. Meanwhile, we can identify and manage the list of packages that need updated libtool information on the side. We could then identify a good cutover point and disable the linker patch in a future release.
If this could be helpful for other new arches like aarch64, we could share the this patch as well.
I believe the Fedora PPC64 team is supportive of this (at least for the ppc64le arch) but we wanted to just solicit broader input.
Feedback appreciated,
Brent
Brent J. Baude Linux Technology Center 3605 Hwy 52N Rochester, MN 55901
http://www.ibm.com/linux/ltc/ 507.253-0708
_______________________________________________ secondary mailing list secondary@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/secondary
On 10/23/2013 11:09 PM, Brent Baude wrote:
In an ideal situation, libtool would release a new version with the ppc64le content (which is already upstream with them) and ALL upstream maintainers would immediately add the new content and drop new releases. That's ideal but not realistic.
What we can do is to change the embedded copies of autotools code to check hard-coded system paths for newer versions and use them instead of the copies in the source tree. autoconf is based on the notion of an alien or downright hostile environment, and that just doesn't match reality anymore.
For config.guess and config.sub, this change is actually pretty easy, except for deciding on the paths (see the discussion on fedora-devel). The additional cost of extracting the versions and comparing them won't matter at all because they are run only few times during a build. For libtool, I'm not sure about that. I think we need to put the logic for that into the configure script, so that it's not executed every time gcc is run to compile a file.
Obviously, all this won't help you at all with ppc64le bootstrapping, but with a bit of luck, the changes will be in place for ppc128. :)
On 10/23/2013 11:09 PM, Brent Baude wrote:
Greetings fellow secondary arch folks,
As some of you might know, we are working to introduce a new architecture called ppc64le into Fedora. It's just as it sounds, ppc64 (64bit only) but little endian. We've been emulating the Fedora process as close as possible internally here and have quite a few packages built, including using things like mock and koji.
As with introducing any new architecture, one of the issues we are seeing is that current packages lack the libtool (config.guess|sub,m4) magic to recognize the architecture. While rpmbuild and mock do properly influence build and targets, the one area we still a high rate of failure with is when configure asks the linker if it supports shared libs. In our case, it actually asks the linker with the wrong architecture.
In an ideal situation, libtool would release a new version with the ppc64le content (which is already upstream with them) and ALL upstream maintainers would immediately add the new content and drop new releases. That's ideal but not realistic.
I also would prefer to NOT update every Fedora package/.spec with a patch to fix this and would prefer to have this be the rare exception rather than the rule.
So in the interest of beginning work in Fedora and not making a mess of things, I'd like to propose another alternative (and solicit more ideas as well). In extending the case where configure “asks” the linker if it supports shared libraries, it actually will pass an incorrect architecture. Using lcms2's configure as an example:
checking whether the gcc -std=gnu99 linker (/bin/ld -m elf64ppc) supports shared libraries... no
In our internal development here, we've patched the linker to always respond as if it was asked about the ppc64le architecture. This function is enabled as an environment variable (LD_FORCE_LE=1). We've been using this for several months and I have yet to encounter an instance where something bad happens.
So I'd like to propose that for initial Fedora ppc64le development, we enable this patch for the linker, at least for the first release. Meanwhile, we can identify and manage the list of packages that need updated libtool information on the side. We could then identify a good cutover point and disable the linker patch in a future release.
If this could be helpful for other new arches like aarch64, we could share the this patch as well.
I believe the Fedora PPC64 team is supportive of this (at least for the ppc64le arch) but we wanted to just solicit broader input.
Feedback appreciated,
Brent
Intriguing idea and certainly a very clever way to avoid having to either wait 1-2 years until a libtool upstream release has happened and the majority of packages have picked that up in newer releases or to go in into Fedora and fix iirc about 2000-3000 package manually that don't use %configure.
I think this could be very interesting for aarch64 as well as you mentioned as i can imagine they fight the same battle there, too.
Especially as it's easy to turn on and off we could set a certain criteria for turning it off in the future when e.g. only a small number of packages would still fail to build (less than 100 maybe?) and at which time we'd turn it off, probably sometime around the branching of a new release.
I'd be in favor of giving it a try and see how it goes. Especially as we don't have to do an official release in our first cycle for a bringup this seems to be the solution with the least amount of effort for the biggest benefit.
Thanks for bringing this up Brent!
Regards, Phil
secondary@lists.fedoraproject.org
-
Brent Baude -
Dan Horák -
Florian Weimer -
Phil Knirsch