I just spent a couple of days studying and attempting to fix ppc64 support in the ocaml package.
Background information ----------------------
OCaml is a nice, fast functional language. We carry ~ 70 OCaml- related packages in Fedora. Since OCaml compiles to native code, it needs a specific backend to be written for each target. Without this it falls back to using a bytecode interpreter which is ~10 times slower, so the backend is not required, but highly desirable.
Native code backends are not large nor especially hard to write. The ppc64 backend in Fedora has just under 2000 lines of code. Moreover a lot of that code is identical to the ppc (32 bit) backend.
486 asmrun/power64-elf.S 83 asmcomp/power64/arch.ml 989 asmcomp/power64/emit.mlp 241 asmcomp/power64/proc.ml 18 asmcomp/power64/reload.ml 65 asmcomp/power64/scheduling.ml 103 asmcomp/power64/selection.ml 1985 total
Upstream, there are backends for many different architectures. Of interest to this list, there is a backend for ppc (32 bit), and ppc64. *However* the upstream ppc64 backend ONLY works on Mac OS X, because it makes assumptions[1] about the architecture and the assembler which don't work with GNU as.
A long time ago, David Woodhouse wrote or modified a ppc64 backend, which we are now carrying as a separate, non-upstream patch in Fedora. It generally works, although it has several bugs, of which more later.
Upstream rejected this backend because they don't have the hardware required to maintain it.
What I did ----------
Because the ppc64 backend is so hard to maintain as a patch, I broke it out into a git repository. This also makes maintaining other patches easier (in particular my backport of the new ARM code generator). If you need access to this, please request it in the usual way via fedorahosted:
http://git.fedorahosted.org/git/?p=fedora-ocaml.git
After doing this, I rewrote parts of our ppc64 backend so that they more closely match the upstream ppc backend, ie. so that the differences in the following commands are minimized as far as possible:
diff -urb asmcomp/power64 asmcomp/power | less diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less
Note that I did NOT make any functional changes to the ppc64 backend when doing this.
I also compiled a number of packages using our ppc64 backend. I noticed that ocaml-camomile wouldn't compile. One module of this program contains a very large "entry" function. The "entry" function of a module is the name given to all the toplevel code in a module which runs when the module loads and isn't a part of any function definition in the module. By experimentation I found that when the entry function exceeds around 26 kbytes, it causes corruption of the minor heap. This is some sort of bug in the code generator, but I wasn't able to determine what, and would appreciate any help in finding it:
https://bugzilla.redhat.com/show_bug.cgi?id=826649
I also found a variety of other bugs which all seem to be caused by the smaller available default stack on ppc64 combined with the larger number of registers (therefore more stack spillage). I will report these upstream as I come across them, but the most serious is this one:
http://caml.inria.fr/mantis/view.php?id=5368
How I need help ---------------
I could do with help on #826649, particularly from ppc64 codegen experts. Also on compiling any remaining ocaml-* packages on ppc64 (most already done) and checking that they actually function. Also on identifying any remaining ocaml packages that have ExcludeArch: ppc64, removing that and if necessary fixing any bugs found.
Rich.
[1] I'm not precisely sure what assumptions it's making, and I could be wrong on this point, but I certainly couldn't get the upstream ppc64 backend to work on Fedora16/ppc64, and I also noticed the assembler syntax is somewhat different from gas and incompatible.
Hi Richard,
Thanks for the work you have done. I would be happy to help on whatever I can. I think that if you describe the steps to reproduce the issue on bugz #826649 it would be a good start point for me to jump in.
Also, do you hang out on freenode? I'm always on channel #fedora-ppc.
[]'s Gustavo
On 05/30/2012 02:19 PM, Richard W.M. Jones wrote:
I just spent a couple of days studying and attempting to fix ppc64 support in the ocaml package.
Background information
OCaml is a nice, fast functional language. We carry ~ 70 OCaml- related packages in Fedora. Since OCaml compiles to native code, it needs a specific backend to be written for each target. Without this it falls back to using a bytecode interpreter which is ~10 times slower, so the backend is not required, but highly desirable.
Native code backends are not large nor especially hard to write. The ppc64 backend in Fedora has just under 2000 lines of code. Moreover a lot of that code is identical to the ppc (32 bit) backend.
486 asmrun/power64-elf.S 83 asmcomp/power64/arch.ml 989 asmcomp/power64/emit.mlp 241 asmcomp/power64/proc.ml 18 asmcomp/power64/reload.ml 65 asmcomp/power64/scheduling.ml 103 asmcomp/power64/selection.ml 1985 total
Upstream, there are backends for many different architectures. Of interest to this list, there is a backend for ppc (32 bit), and ppc64. *However* the upstream ppc64 backend ONLY works on Mac OS X, because it makes assumptions[1] about the architecture and the assembler which don't work with GNU as.
A long time ago, David Woodhouse wrote or modified a ppc64 backend, which we are now carrying as a separate, non-upstream patch in Fedora. It generally works, although it has several bugs, of which more later.
Upstream rejected this backend because they don't have the hardware required to maintain it.
What I did
Because the ppc64 backend is so hard to maintain as a patch, I broke it out into a git repository. This also makes maintaining other patches easier (in particular my backport of the new ARM code generator). If you need access to this, please request it in the usual way via fedorahosted:
http://git.fedorahosted.org/git/?p=fedora-ocaml.git
After doing this, I rewrote parts of our ppc64 backend so that they more closely match the upstream ppc backend, ie. so that the differences in the following commands are minimized as far as possible:
diff -urb asmcomp/power64 asmcomp/power | less diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less
Note that I did NOT make any functional changes to the ppc64 backend when doing this.
I also compiled a number of packages using our ppc64 backend. I noticed that ocaml-camomile wouldn't compile. One module of this program contains a very large "entry" function. The "entry" function of a module is the name given to all the toplevel code in a module which runs when the module loads and isn't a part of any function definition in the module. By experimentation I found that when the entry function exceeds around 26 kbytes, it causes corruption of the minor heap. This is some sort of bug in the code generator, but I wasn't able to determine what, and would appreciate any help in finding it:
https://bugzilla.redhat.com/show_bug.cgi?id=826649
I also found a variety of other bugs which all seem to be caused by the smaller available default stack on ppc64 combined with the larger number of registers (therefore more stack spillage). I will report these upstream as I come across them, but the most serious is this one:
http://caml.inria.fr/mantis/view.php?id=5368
How I need help
I could do with help on #826649, particularly from ppc64 codegen experts. Also on compiling any remaining ocaml-* packages on ppc64 (most already done) and checking that they actually function. Also on identifying any remaining ocaml packages that have ExcludeArch: ppc64, removing that and if necessary fixing any bugs found.
Rich.
[1] I'm not precisely sure what assumptions it's making, and I could be wrong on this point, but I certainly couldn't get the upstream ppc64 backend to work on Fedora16/ppc64, and I also noticed the assembler syntax is somewhat different from gas and incompatible.
On Wed, May 30, 2012 at 02:58:53PM -0300, Gustavo Luiz Duarte wrote:
Hi Richard,
Thanks for the work you have done. I would be happy to help on whatever I can. I think that if you describe the steps to reproduce the issue on bugz #826649 it would be a good start point for me to jump in.
I've updated the bug with a rather massive reproducer:
https://bugzilla.redhat.com/show_bug.cgi?id=826649#c1
I'll try and get it down to something sensible in the morning.
Rich.
On Wed, 2012-05-30 at 18:19 +0100, Richard W.M. Jones wrote:
diff -urb asmcomp/power64 asmcomp/power | less diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less
Note that I did NOT make any functional changes to the ppc64 backend when doing this.
FWIW I did try to keep that delta minimal when I was doing the original port. I don't think the Darwin ppc64 port existed at the time, so I took a copy of the ppc32 port (which *is* portable to both OSX and Linux IIRC, so it's not that hard), and modified it to account for the differences between ppc32 and ppc64 ABIs.
My original code is in a git tree at git://, http://git.infradead.org/users/dwmw2/ocaml-ppc64.git
If upstream has a ppc64 port now, I'd definitely be looking at merging with it. Darwin *does* use ELF, so it's fairly much the back end you're looking for. I imagine you'd just need to port over the asm-agnostic features of the ppc32 back end (which are probably present in my ppc64 back end because I wouldn't have ripped them out).
That said, this was a number of years ago. Before I did it, I had completely forgotten everything they ever taught me in University about ML — and now I believe I have forgotten it all again... ;)
On Wed, 2012-05-30 at 19:33 +0100, David Woodhouse wrote:
If upstream has a ppc64 port now, I'd definitely be looking at merging with it. Darwin *does* use ELF, so it's fairly much the back end you're looking for. I imagine you'd just need to port over the asm-agnostic features of the ppc32 back end (which are probably present in my ppc64 back end because I wouldn't have ripped them out).
That's this kind of thing in emit.mlp...
+(* Output a "upper 16 bits" or "lower 16 bits" operator. *) + +let emit_upper emit_fun arg = + match Config.system with + | "elf" | "bsd" -> + emit_fun arg; emit_string "@ha" + | "rhapsody" -> + emit_string "ha16("; emit_fun arg; emit_string ")" + | _ -> assert false + +let emit_lower emit_fun arg = + match Config.system with + | "elf" | "bsd" -> + emit_fun arg; emit_string "@l" + | "rhapsody" -> + emit_string "lo16("; emit_fun arg; emit_string ")" + | _ -> assert false +
It'd definitely be interesting to see how far you can get by re-introducing this abstraction into the upstream ppc64 back end.
On Wed, 2012-05-30 at 19:37 +0100, David Woodhouse wrote:
It'd definitely be interesting to see how far you can get by re-introducing this abstraction into the upstream ppc64 back end.
Oh, wait. My memories of this are slowly returning. Isn't the Darwin/ppc64 ABI basically the same as the ELF/ppc32 ABI?
For Linux/ppc64 there are more differences in the ABI (most of which you can see mentioned in the changelog of my git tree, as I started with the ppc32 back end and made it cope with all the differences).
So no, it does make sense that Darwin/ppc64 uses the old power back end, while Linux/ppc64 gets its own.
On Wed, May 30, 2012 at 08:12:06PM +0100, David Woodhouse wrote:
On Wed, 2012-05-30 at 19:37 +0100, David Woodhouse wrote:
It'd definitely be interesting to see how far you can get by re-introducing this abstraction into the upstream ppc64 back end.
Oh, wait. My memories of this are slowly returning. Isn't the Darwin/ppc64 ABI basically the same as the ELF/ppc32 ABI?
It seems to use the 'd'-ouble variant (instead of 'w'-ord) instructions ...
For Linux/ppc64 there are more differences in the ABI (most of which you can see mentioned in the changelog of my git tree, as I started with the ppc32 back end and made it cope with all the differences).
So no, it does make sense that Darwin/ppc64 uses the old power back end, while Linux/ppc64 gets its own.
OK, guess I'll take a look at this tomorrow to understand the differences. Thanks.
Rich.
Upstream, there are backends for many different architectures. Of interest to this list, there is a backend for ppc (32 bit), and ppc64. *However* the upstream ppc64 backend ONLY works on Mac OS X, because it makes assumptions[1] about the architecture and the assembler which don't work with GNU as.
A long time ago, David Woodhouse wrote or modified a ppc64 backend, which we are now carrying as a separate, non-upstream patch in Fedora. It generally works, although it has several bugs, of which more later.
Upstream rejected this backend because they don't have the hardware required to maintain it.
What are their requirements ? Do they just need access to a ppc64 machine ? I think we might be able to help out with that, I'll talk with Brent if you can confirm that ssh access to a machine is sufficient.
Karsten
On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
Upstream rejected this backend because they don't have the hardware required to maintain it.
What are their requirements ? Do they just need access to a ppc64 machine ? I think we might be able to help out with that, I'll talk with Brent if you can confirm that ssh access to a machine is sufficient.
ISTR IBM even offered them a machine at the time, but they didn't want it.
On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
Upstream rejected this backend because they don't have the hardware required to maintain it.
What are their requirements ? Do they just need access to a ppc64 machine ? I think we might be able to help out with that, I'll talk with Brent if you can confirm that ssh access to a machine is sufficient.
ISTR IBM even offered them a machine at the time, but they didn't want it.
Yeah, I offered to buy them a machine too, but they just don't want to maintain a ppc64 backend (or apparently not a non-Rhapsody one). In any case I'm fine maintaining this.
On the subject, what is a good cheap ppc64 machine to buy these days? A second hand G5?
Rich.
Richard,
As Karsten mentioned, there are machines we can put you on (with way better perf than a G5 plus more memory) that are already publically available and running Fedora. Swing into #fedora-ppc and we can put you on one.
Brent J. Baude Linux Technology Center 3605 Hwy 52N Rochester, MN 55901
http://www.ibm.com/linux/ltc/ 507.253-0708
From: "Richard W.M. Jones" rjones@redhat.com To: David Woodhouse dwmw2@infradead.org, Cc: ppc@lists.fedoraproject.org Date: 05/30/2012 03:09 PM Subject: Re: OCaml ppc64 support Sent by: ppc-bounces@lists.fedoraproject.org
On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
Upstream rejected this backend because they don't have the hardware required to maintain it.
What are their requirements ? Do they just need access to a ppc64
machine ?
I think we might be able to help out with that, I'll talk with Brent if
you can confirm
that ssh access to a machine is sufficient.
ISTR IBM even offered them a machine at the time, but they didn't want it.
Yeah, I offered to buy them a machine too, but they just don't want to maintain a ppc64 backend (or apparently not a non-Rhapsody one). In any case I'm fine maintaining this.
On the subject, what is a good cheap ppc64 machine to buy these days? A second hand G5?
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v _______________________________________________ ppc mailing list ppc@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ppc
I bought a Xserve G5 for $100 off of Craigslist. It was still in the box and never turned on. Even came with OS X Tiger Unlimited Server. I was going to dual-boot gentoo/Fedora/OSX Leopard. I bought 6 Xserve G4's too, five parts machines and one machine i'm going to run Gentoo on.
Deals are around, I'd look for a Xserve if you can find one (theyre wide, but not tall and fit under a desk or endtable.
On Wed, May 30, 2012 at 4:08 PM, Richard W.M. Jones rjones@redhat.com wrote:
On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
Upstream rejected this backend because they don't have the hardware required to maintain it.
What are their requirements ? Do they just need access to a ppc64 machine ? I think we might be able to help out with that, I'll talk with Brent if you can confirm that ssh access to a machine is sufficient.
ISTR IBM even offered them a machine at the time, but they didn't want it.
Yeah, I offered to buy them a machine too, but they just don't want to maintain a ppc64 backend (or apparently not a non-Rhapsody one). In any case I'm fine maintaining this.
On the subject, what is a good cheap ppc64 machine to buy these days? A second hand G5?
Rich.
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into Xen guests. http://et.redhat.com/~rjones/virt-p2v _______________________________________________ ppc mailing list ppc@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ppc
On Wed, May 30, 2012 at 04:16:05PM -0400, Britt Dodd wrote:
I bought a Xserve G5 for $100 off of Craigslist. It was still in the box and never turned on. Even came with OS X Tiger Unlimited Server. I was going to dual-boot gentoo/Fedora/OSX Leopard. I bought 6 Xserve G4's too, five parts machines and one machine i'm going to run Gentoo on.
Deals are around, I'd look for a Xserve if you can find one (theyre wide, but not tall and fit under a desk or endtable.
Judging by prices on ebay, you should sell that Xserve G5 in the UK right now and make ~ $1500 in straight profit :-)
I just got an Apple G5 Tower 2 GHz dual core for a somewhat more reasonable £160 inc tax and delivery. Not as nice as the Xserve, but a fraction of the price ...
Thanks all,
Rich.
I finally managed to get a small(-ish) reproducer for the large entry function bug. Well, just over 1000 lines of code :-)
https://bugzilla.redhat.com/show_bug.cgi?id=826649#c5
Rich.
-
Brent Baude -
Britt Dodd -
David Woodhouse -
Gustavo Luiz Duarte -
Karsten Hopp -
Richard W.M. Jones