OCaml ppc64 support

[Richard W.M. Jones]

I just spent a couple of days studying and attempting to fix ppc64
support in the ocaml package.

Background information
----------------------

OCaml is a nice, fast functional language.  We carry ~ 70 OCaml-
related packages in Fedora.  Since OCaml compiles to native code, it
needs a specific backend to be written for each target.  Without this
it falls back to using a bytecode interpreter which is ~10 times
slower, so the backend is not required, but highly desirable.

Native code backends are not large nor especially hard to write.  The
ppc64 backend in Fedora has just under 2000 lines of code.  Moreover a
lot of that code is identical to the ppc (32 bit) backend.

  486 asmrun/power64-elf.S
   83 asmcomp/power64/arch.ml
  989 asmcomp/power64/emit.mlp
  241 asmcomp/power64/proc.ml
   18 asmcomp/power64/reload.ml
   65 asmcomp/power64/scheduling.ml
  103 asmcomp/power64/selection.ml
 1985 total

Upstream, there are backends for many different architectures.  Of
interest to this list, there is a backend for ppc (32 bit), and ppc64.
*However* the upstream ppc64 backend ONLY works on Mac OS X, because
it makes assumptions[1] about the architecture and the assembler which
don't work with GNU as.

A long time ago, David Woodhouse wrote or modified a ppc64 backend,
which we are now carrying as a separate, non-upstream patch in Fedora.
It generally works, although it has several bugs, of which more later.

Upstream rejected this backend because they don't have the hardware
required to maintain it.

What I did
----------

Because the ppc64 backend is so hard to maintain as a patch, I broke
it out into a git repository.  This also makes maintaining other
patches easier (in particular my backport of the new ARM code
generator).  If you need access to this, please request it in the
usual way via fedorahosted:

http://git.fedorahosted.org/git/?p=fedora-ocaml.git

After doing this, I rewrote parts of our ppc64 backend so that they
more closely match the upstream ppc backend, ie. so that the
differences in the following commands are minimized as far as
possible:

  diff -urb asmcomp/power64 asmcomp/power | less
  diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less

Note that I did NOT make any functional changes to the ppc64 backend
when doing this.

I also compiled a number of packages using our ppc64 backend.  I
noticed that ocaml-camomile wouldn't compile.  One module of this
program contains a very large "entry" function.  The "entry" function
of a module is the name given to all the toplevel code in a module
which runs when the module loads and isn't a part of any function
definition in the module.  By experimentation I found that when the
entry function exceeds around 26 kbytes, it causes corruption of the
minor heap.  This is some sort of bug in the code generator, but I
wasn't able to determine what, and would appreciate any help in
finding it:

  https://bugzilla.redhat.com/show_bug.cgi?id=826649

I also found a variety of other bugs which all seem to be caused by
the smaller available default stack on ppc64 combined with the larger
number of registers (therefore more stack spillage).  I will report
these upstream as I come across them, but the most serious is this
one:

  http://caml.inria.fr/mantis/view.php?id=5368

How I need help
---------------

I could do with help on #826649, particularly from ppc64 codegen
experts.  Also on compiling any remaining ocaml-* packages on ppc64
(most already done) and checking that they actually function.  Also on
identifying any remaining ocaml packages that have ExcludeArch: ppc64,
removing that and if necessary fixing any bugs found.

Rich.

[1] I'm not precisely sure what assumptions it's making, and I could
be wrong on this point, but I certainly couldn't get the upstream
ppc64 backend to work on Fedora16/ppc64, and I also noticed the
assembler syntax is somewhat different from gas and incompatible.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top

[Gustavo Luiz Duarte]

Hi Richard,

Thanks for the work you have done. I would be happy to help on whatever 
I can. I think that if you describe the steps to reproduce the issue on 
bugz #826649 it would be a good start point for me to jump in.

Also, do you hang out on freenode? I'm always on channel #fedora-ppc.

[]'s
Gustavo

On 05/30/2012 02:19 PM, Richard W.M. Jones wrote:
>  I just spent a couple of days studying and attempting to fix ppc64
>  support in the ocaml package.
>
>  Background information
>  ----------------------
>
>  OCaml is a nice, fast functional language.  We carry ~ 70 OCaml-
>  related packages in Fedora.  Since OCaml compiles to native code, it
>  needs a specific backend to be written for each target.  Without this
>  it falls back to using a bytecode interpreter which is ~10 times
>  slower, so the backend is not required, but highly desirable.
>
>  Native code backends are not large nor especially hard to write.  The
>  ppc64 backend in Fedora has just under 2000 lines of code.  Moreover a
>  lot of that code is identical to the ppc (32 bit) backend.
>
>     486 asmrun/power64-elf.S
>      83 asmcomp/power64/arch.ml
>     989 asmcomp/power64/emit.mlp
>     241 asmcomp/power64/proc.ml
>      18 asmcomp/power64/reload.ml
>      65 asmcomp/power64/scheduling.ml
>     103 asmcomp/power64/selection.ml
>    1985 total
>
>  Upstream, there are backends for many different architectures.  Of
>  interest to this list, there is a backend for ppc (32 bit), and ppc64.
>  *However* the upstream ppc64 backend ONLY works on Mac OS X, because
>  it makes assumptions[1] about the architecture and the assembler which
>  don't work with GNU as.
>
>  A long time ago, David Woodhouse wrote or modified a ppc64 backend,
>  which we are now carrying as a separate, non-upstream patch in Fedora.
>  It generally works, although it has several bugs, of which more later.
>
>  Upstream rejected this backend because they don't have the hardware
>  required to maintain it.
>
>  What I did
>  ----------
>
>  Because the ppc64 backend is so hard to maintain as a patch, I broke
>  it out into a git repository.  This also makes maintaining other
>  patches easier (in particular my backport of the new ARM code
>  generator).  If you need access to this, please request it in the
>  usual way via fedorahosted:
>
>  http://git.fedorahosted.org/git/?p=fedora-ocaml.git
>
>  After doing this, I rewrote parts of our ppc64 backend so that they
>  more closely match the upstream ppc backend, ie. so that the
>  differences in the following commands are minimized as far as
>  possible:
>
>     diff -urb asmcomp/power64 asmcomp/power | less
>     diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less
>
>  Note that I did NOT make any functional changes to the ppc64 backend
>  when doing this.
>
>  I also compiled a number of packages using our ppc64 backend.  I
>  noticed that ocaml-camomile wouldn't compile.  One module of this
>  program contains a very large "entry" function.  The "entry"
> function
>  of a module is the name given to all the toplevel code in a module
>  which runs when the module loads and isn't a part of any function
>  definition in the module.  By experimentation I found that when the
>  entry function exceeds around 26 kbytes, it causes corruption of the
>  minor heap.  This is some sort of bug in the code generator, but I
>  wasn't able to determine what, and would appreciate any help in
>  finding it:
>
>     https://bugzilla.redhat.com/show_bug.cgi?id=826649
>
>  I also found a variety of other bugs which all seem to be caused by
>  the smaller available default stack on ppc64 combined with the larger
>  number of registers (therefore more stack spillage).  I will report
>  these upstream as I come across them, but the most serious is this
>  one:
>
>     http://caml.inria.fr/mantis/view.php?id=5368
>
>  How I need help
>  ---------------
>
>  I could do with help on #826649, particularly from ppc64 codegen
>  experts.  Also on compiling any remaining ocaml-* packages on ppc64
>  (most already done) and checking that they actually function.  Also on
>  identifying any remaining ocaml packages that have ExcludeArch: ppc64,
>  removing that and if necessary fixing any bugs found.
>
>  Rich.
>
>  [1] I'm not precisely sure what assumptions it's making, and I could
>  be wrong on this point, but I certainly couldn't get the upstream
>  ppc64 backend to work on Fedora16/ppc64, and I also noticed the
>  assembler syntax is somewhat different from gas and incompatible.
>

[David Woodhouse]

On Wed, 2012-05-30 at 18:19 +0100, Richard W.M. Jones wrote:
>  
>    diff -urb asmcomp/power64 asmcomp/power | less
>    diff -urb asmrun/power64-elf.S asmrun/power-elf.S | less
>  
>  Note that I did NOT make any functional changes to the ppc64 backend
>  when doing this.  
FWIW I did try to keep that delta minimal when I was doing the original
port. I don't think the Darwin ppc64 port existed at the time, so I took
a copy of the ppc32 port (which *is* portable to both OSX and Linux
IIRC, so it's not that hard), and modified it to account for the
differences between ppc32 and ppc64 ABIs.

My original code is in a git tree at 
 git://, http://git.infradead.org/users/dwmw2/ocaml-ppc64.git

If upstream has a ppc64 port now, I'd definitely be looking at merging
with it. Darwin *does* use ELF, so it's fairly much the back end you're
looking for. I imagine you'd just need to port over the asm-agnostic
features of the ppc32 back end (which are probably present in my ppc64
back end because I wouldn't have ripped them out).

That said, this was a number of years ago. Before I did it, I had
completely forgotten everything they ever taught me in University about
ML — and now I believe I have forgotten it all again... ;)

-- 
dwmw2

[David Woodhouse]

On Wed, 2012-05-30 at 19:33 +0100, David Woodhouse wrote:
>  If upstream has a ppc64 port now, I'd definitely be looking at
> merging
>  with it. Darwin *does* use ELF, so it's fairly much the back end you're
>  looking for. I imagine you'd just need to port over the asm-agnostic
>  features of the ppc32 back end (which are probably present in my ppc64
>  back end because I wouldn't have ripped them out). 
That's this kind of thing in emit.mlp...

+(* Output a "upper 16 bits" or "lower 16 bits" operator. *)
+
+let emit_upper emit_fun arg =
+  match Config.system with
+  | "elf" | "bsd" ->
+      emit_fun arg; emit_string "@ha"
+  | "rhapsody" ->
+      emit_string "ha16("; emit_fun arg; emit_string ")"
+  | _ -> assert false
+
+let emit_lower emit_fun arg =
+  match Config.system with
+  | "elf" | "bsd" ->
+      emit_fun arg; emit_string "@l"
+  | "rhapsody" ->
+      emit_string "lo16("; emit_fun arg; emit_string ")"
+  | _ -> assert false
+

It'd definitely be interesting to see how far you can get by
re-introducing this abstraction into the upstream ppc64 back end.

-- 
dwmw2

[David Woodhouse]

On Wed, 2012-05-30 at 19:37 +0100, David Woodhouse wrote:
>  
>  It'd definitely be interesting to see how far you can get by
>  re-introducing this abstraction into the upstream ppc64 back end. 
Oh, wait. My memories of this are slowly returning. Isn't the
Darwin/ppc64 ABI basically the same as the ELF/ppc32 ABI?

For Linux/ppc64 there are more differences in the ABI (most of which you
can see mentioned in the changelog of my git tree, as I started with the
ppc32 back end and made it cope with all the differences).

So no, it does make sense that Darwin/ppc64 uses the old power back end,
while Linux/ppc64 gets its own.

-- 
dwmw2

[Richard W.M. Jones]

On Wed, May 30, 2012 at 08:12:06PM +0100, David Woodhouse wrote:
>  On Wed, 2012-05-30 at 19:37 +0100, David Woodhouse wrote:
>  > 
>  > It'd definitely be interesting to see how far you can get by
>  > re-introducing this abstraction into the upstream ppc64 back end.
>  
>  Oh, wait. My memories of this are slowly returning. Isn't the
>  Darwin/ppc64 ABI basically the same as the ELF/ppc32 ABI? 
It seems to use the 'd'-ouble variant (instead of 'w'-ord) instructions
...

>  For Linux/ppc64 there are more differences in the ABI (most of which
> you
>  can see mentioned in the changelog of my git tree, as I started with the
>  ppc32 back end and made it cope with all the differences).
>
>  So no, it does make sense that Darwin/ppc64 uses the old power back end,
>  while Linux/ppc64 gets its own. 
OK, guess I'll take a look at this tomorrow to understand the
differences.  Thanks.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

[David Woodhouse]

On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
>  > Upstream rejected this backend because they don't have the
> hardware
>  > required to maintain it.
>  >
>  
>  What are their requirements ? Do they just need access to a ppc64 machine ?
>  I think we might be able to help out with that, I'll talk with Brent if you can
> confirm 
>  that ssh access to a machine is sufficient. 
ISTR IBM even offered them a machine at the time, but they didn't want
it.

-- 
dwmw2

[Richard W.M. Jones]

On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
>  On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
>  > > Upstream rejected this backend because they don't have the hardware
>  > > required to maintain it.
>  > >
>  > 
>  > What are their requirements ? Do they just need access to a ppc64 machine ?
>  > I think we might be able to help out with that, I'll talk with Brent if you can
> confirm 
>  > that ssh access to a machine is sufficient.
>  
>  ISTR IBM even offered them a machine at the time, but they didn't want
>  it. 
Yeah, I offered to buy them a machine too, but they just don't want to
maintain a ppc64 backend (or apparently not a non-Rhapsody one).  In
any case I'm fine maintaining this.

On the subject, what is a good cheap ppc64 machine to buy these days?
A second hand G5?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

[Brent Baude]

Richard,

As Karsten mentioned, there are machines we can put you on (with way better
perf than a G5 plus more memory) that are already publically available and
running Fedora.  Swing into #fedora-ppc and we can put you on one.


Brent J. Baude
Linux Technology Center
3605 Hwy 52N
Rochester, MN  55901

http://www.ibm.com/linux/ltc/
507.253-0708




From:	"Richard W.M. Jones" <rjones(a)redhat.com&gt;
To:	David Woodhouse <dwmw2(a)infradead.org&gt;,
Cc:	ppc(a)lists.fedoraproject.org
Date:	05/30/2012 03:09 PM
Subject:	Re: OCaml ppc64 support
Sent by:	ppc-bounces(a)lists.fedoraproject.org



On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
>  On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
>  > > Upstream rejected this backend because they don't have the hardware
>  > > required to maintain it.
>  > >
>  >
>  > What are their requirements ? Do they just need access to a ppc64 
machine ?
>  > I think we might be able to help out with that, I'll talk
> with Brent if 
you can confirm
>  > that ssh access to a machine is sufficient.
>
>  ISTR IBM even offered them a machine at the time, but they didn't want
>  it. 
Yeah, I offered to buy them a machine too, but they just don't want to
maintain a ppc64 backend (or apparently not a non-Rhapsody one).  In
any case I'm fine maintaining this.

On the subject, what is a good cheap ppc64 machine to buy these days?
A second hand G5?

Rich.

--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v
_______________________________________________
ppc mailing list
ppc(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/ppc

[Britt Dodd]

I bought a Xserve G5 for $100 off of Craigslist. It was still in the
box and never turned on. Even came with OS X Tiger Unlimited Server. I
was going to dual-boot gentoo/Fedora/OSX Leopard. I bought 6 Xserve
G4's too, five parts machines and one machine i'm going to run Gentoo
on.

Deals are around, I'd look for a Xserve if you can find one (theyre
wide, but not tall and fit under a desk or endtable.

On Wed, May 30, 2012 at 4:08 PM, Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:
>  On Wed, May 30, 2012 at 09:04:49PM +0100, David Woodhouse wrote:
> > On Wed, 2012-05-30 at 23:03 +0200, Karsten Hopp wrote:
> > > > Upstream rejected this backend because they don't have the hardware
> > > > required to maintain it.
> > > >
> > >
> > > What are their requirements ? Do they just need access to a ppc64 machine ?
> > > I think we might be able to help out with that, I'll talk with Brent if you
> can confirm
> > > that ssh access to a machine is sufficient.
> >
> > ISTR IBM even offered them a machine at the time, but they didn't want
> > it.
>
>  Yeah, I offered to buy them a machine too, but they just don't want to
>  maintain a ppc64 backend (or apparently not a non-Rhapsody one).  In
>  any case I'm fine maintaining this.
>
>  On the subject, what is a good cheap ppc64 machine to buy these days?
>  A second hand G5?
>
>  Rich.
>
>  --
>  Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>  virt-p2v converts physical machines to virtual machines.  Boot with a
>  live CD or over the network (PXE) and turn machines into Xen guests.
>  http://et.redhat.com/~rjones/virt-p2v
>  _______________________________________________
>  ppc mailing list
>  ppc(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/ppc

[Karsten Hopp]

>  Upstream, there are backends for many different architectures.  Of
>  interest to this list, there is a backend for ppc (32 bit), and ppc64.
>  *However* the upstream ppc64 backend ONLY works on Mac OS X, because
>  it makes assumptions[1] about the architecture and the assembler which
>  don't work with GNU as.
>
>  A long time ago, David Woodhouse wrote or modified a ppc64 backend,
>  which we are now carrying as a separate, non-upstream patch in Fedora.
>  It generally works, although it has several bugs, of which more later.
>
>  Upstream rejected this backend because they don't have the hardware
>  required to maintain it.
>  
What are their requirements ? Do they just need access to a ppc64 machine ?
I think we might be able to help out with that, I'll talk with Brent if you can
confirm 
that ssh access to a machine is sufficient.

   Karsten

[Richard W.M. Jones]

On Wed, May 30, 2012 at 04:16:05PM -0400, Britt Dodd wrote:
>  I bought a Xserve G5 for $100 off of Craigslist. It was still in the
>  box and never turned on. Even came with OS X Tiger Unlimited Server. I
>  was going to dual-boot gentoo/Fedora/OSX Leopard. I bought 6 Xserve
>  G4's too, five parts machines and one machine i'm going to run Gentoo
>  on.
>  
>  Deals are around, I'd look for a Xserve if you can find one (theyre
>  wide, but not tall and fit under a desk or endtable. 
Judging by prices on ebay, you should sell that Xserve G5 in the UK
right now and make ~ $1500 in straight profit :-)

I just got an Apple G5 Tower 2 GHz dual core for a somewhat more
reasonable £160 inc tax and delivery.  Not as nice as the Xserve, but
a fraction of the price ...

Thanks all,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

[Richard W.M. Jones]

On Wed, May 30, 2012 at 02:58:53PM -0300, Gustavo Luiz Duarte wrote:
>  Hi Richard,
>  
>  Thanks for the work you have done. I would be happy to help on
>  whatever I can. I think that if you describe the steps to reproduce
>  the issue on bugz #826649 it would be a good start point for me to
>  jump in. 
I've updated the bug with a rather massive reproducer:

https://bugzilla.redhat.com/show_bug.cgi?id=826649#c1

I'll try and get it down to something sensible in the morning.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora

[Richard W.M. Jones]

I finally managed to get a small(-ish) reproducer for the large entry
function bug.  Well, just over 1000 lines of code :-)

https://bugzilla.redhat.com/show_bug.cgi?id=826649#c5

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v