v8 for aarch64

[Richard W.M. Jones]

Hello Tom and other v8 maintainers.

I was looking at porting the Fedora Rawhide v8 package to aarch64,
since it is a dependency of mongodb, and hence an indirect dependency
of OpenStack Ceilometer.

The bad news is that there is no aarch64 support in the current Fedora
package.  The good news is that upstream v8 from git has aarch64
support.  The not so good news is that it's not completely working.

Firstly at the moment you're building from SVN-generated tarballs (is
that right?).  V8 recently moved to using git.  How do you feel about
using git tags instead?  The latest git tag is 3.31.27 which was
released earlier today.  That tag has the aarch64 support.

There are two piece of not so good news:

(1) The newer version has removed scons support, which would
ordinarily be a good thing, scons being a huge steaming pile of crap.
However in this case it has been replaced with some kind of
home-brewed build system called 'gyp' which seems to be worse.

I've no idea how to package this.  At the moment the build downloads
gyp from svn(!?) and obviously that's not going to be acceptable for
Fedora.  I'm not clear if we can bundle gyp sources into the v8 srpm,
or need to have a separate package, or if there's some other way to do it.

(2) I built the git head on my aarch64 box and tested it.  It doesn't
fully pass the tests with over 600 (out of 22000) failing:

[21:42|% 100|+ 22384|- 636]: Done                                              

Compared to git head on x86-64 where only 4 tests fail:

[05:03|% 100|+ 21824|-   4]: Done                                              

Anyway I'll keep plugging away at this and see if I can fix the tests.

But I'm interested what your thoughts and plans are for v8 w.r.t
switching to git and about gyp.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

[T.C. Hollingsworth]

Replying by phone because holidays, sorry for any faux pas.

On Nov 28, 2014 4:15 AM, "Richard W.M. Jones" <rjones(a)redhat.com&gt; wrote:
>  Hello Tom and other v8 maintainers.
>
>  I was looking at porting the Fedora Rawhide v8 package to aarch64,
>  since it is a dependency of mongodb, and hence an indirect dependency
>  of OpenStack Ceilometer.
>
>  The bad news is that there is no aarch64 support in the current Fedora
>  package.  The good news is that upstream v8 from git has aarch64
>  support.  The not so good news is that it's not completely working.
>
>  Firstly at the moment you're building from SVN-generated tarballs (is
>  that right?).  V8 recently moved to using git.  How do you feel about
>  using git tags instead?  The latest git tag is 3.31.27 which was
>  released earlier today.  That tag has the aarch64 support. 
We don't typically track all v8 releases since they break API and ABI on a
monthly basis and are quickly no longer supported by Google. We do track v8
releases used by stable versions of node.js, since these are typically
maintained for years.

As of right now, it looks like the upcoming nodejs 0.12 release will use
the 3.30 branch (it currently ships 3.30.37). aarch64 has been "officially
supported" since 3.25.

If aarch64 support works in 3.30 just as well as 3.31, I'd suggest focusing
your polishing efforts there (along with fixing master as appropriate), so
it can last for awhile. If 3.31 is really better I may be able to push to
get node bumped upstream to make everyone's life easier.

>  There are two piece of not so good news:
>
>  (1) The newer version has removed scons support, which would
>  ordinarily be a good thing, scons being a huge steaming pile of crap.
>  However in this case it has been replaced with some kind of
>  home-brewed build system called 'gyp' which seems to be worse.
>
>  I've no idea how to package this.  At the moment the build downloads
>  gyp from svn(!?) and obviously that's not going to be acceptable for
>  Fedora.  I'm not clear if we can bundle gyp sources into the v8 srpm,
>  or need to have a separate package, or if there's some other way to do it. 
There's a gyp package in Fedora already. The maintainer is usually good
about updating it to the latest svn snapshot, if that is needed for aarch64
support.

>  (2) I built the git head on my aarch64 box and tested it.  It
> doesn't
>  fully pass the tests with over 600 (out of 22000) failing:
>
>  [21:42|% 100|+ 22384|- 636]: Done
>
>  Compared to git head on x86-64 where only 4 tests fail:
>
>  [05:03|% 100|+ 21824|-   4]: Done
>
>  Anyway I'll keep plugging away at this and see if I can fix the tests. 
There's also:

3.) MongoDB will need to be ported to work with newer v8. There have been
massive changes to the v8 API in recent versions (which is part of the
reason it has taken so long to get from node 0.10 to node 0.12). AFAICT no
porting effort has even been considered by mongodb upstream yet. [Actually,
they ship it with an even older version of v8 than we use it with in
Fedora. :-(  ]

As of this moment my current plan to maintain mongodb support in Fedora is
to introduce a compat-v8-314 package at the same time we bump v8 to 3.30.
Which will be no problem supporting for a couple more years at least, but
it's still basically on life support and that doesn't bode well for arm64
support.

You may want to talk to mongodb upstream first and foremost and inquire
about getting it to work with newer v8 versions.  I'd hate for you to do a
bunch of work fixing v8 on arm64 just for it to only end up benefitting
Chrome for Android users. ;-)

>  But I'm interested what your thoughts and plans are for v8 w.r.t
>  switching to git and about gyp 
-T.C.

[Richard W.M. Jones]

On Fri, Nov 28, 2014 at 05:49:39AM -0700, T.C. Hollingsworth wrote:
>  Replying by phone because holidays, sorry for any faux pas. 
No problem, thanks for the quick reply!

>  On Nov 28, 2014 4:15 AM, "Richard W.M. Jones"
> <rjones(a)redhat.com&gt; wrote:
>  >
>  > Hello Tom and other v8 maintainers.
>  >
>  > I was looking at porting the Fedora Rawhide v8 package to aarch64,
>  > since it is a dependency of mongodb, and hence an indirect dependency
>  > of OpenStack Ceilometer.
>  >
>  > The bad news is that there is no aarch64 support in the current Fedora
>  > package.  The good news is that upstream v8 from git has aarch64
>  > support.  The not so good news is that it's not completely working.
>  >
>  > Firstly at the moment you're building from SVN-generated tarballs (is
>  > that right?).  V8 recently moved to using git.  How do you feel about
>  > using git tags instead?  The latest git tag is 3.31.27 which was
>  > released earlier today.  That tag has the aarch64 support.
>  
>  We don't typically track all v8 releases since they break API and ABI on a
>  monthly basis and are quickly no longer supported by Google. We do track v8
>  releases used by stable versions of node.js, since these are typically
>  maintained for years.
>  
>  As of right now, it looks like the upcoming nodejs 0.12 release will use
>  the 3.30 branch (it currently ships 3.30.37). aarch64 has been "officially
>  supported" since 3.25.
>  
>  If aarch64 support works in 3.30 just as well as 3.31, I'd suggest focusing
>  your polishing efforts there (along with fixing master as appropriate), so
>  it can last for awhile. If 3.31 is really better I may be able to push to
>  get node bumped upstream to make everyone's life easier. 
I tested 3.30.37 and aarch64 support seems to be at the same level as
head.

The precise same set of 636 tests fail too.  However looking more
closely, (1) all the tests are in the same bit of code which is
something to do with math libraries AFAICT, and (2) the JIT makes
debugging it very difficult - there are no stack frames so gdb can't
make head nor tail of it.  I suggest we ignore the test failures unless
they actually affect mongodb/nodejs.

Anyway the bottom line is that 3.30.37 is fine.

>  3.) MongoDB will need to be ported to work with newer v8. There have
> been
>  massive changes to the v8 API in recent versions (which is part of the
>  reason it has taken so long to get from node 0.10 to node 0.12). AFAICT no
>  porting effort has even been considered by mongodb upstream yet. [Actually,
>  they ship it with an even older version of v8 than we use it with in
>  Fedora. :-(  ]
>  
>  As of this moment my current plan to maintain mongodb support in Fedora is
>  to introduce a compat-v8-314 package at the same time we bump v8 to 3.30.
>  Which will be no problem supporting for a couple more years at least, but
>  it's still basically on life support and that doesn't bode well for arm64
>  support.
>  
>  You may want to talk to mongodb upstream first and foremost and inquire
>  about getting it to work with newer v8 versions.  I'd hate for you to do a
>  bunch of work fixing v8 on arm64 just for it to only end up benefitting
>  Chrome for Android users. ;-) 
Bluuurrrrggggghhhhhhh.  Professional engineering practices FTW.

There's an upstream mongodb bug about this which doesn't look
encouraging:
https://jira.mongodb.org/browse/SERVER-10282

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

[Andrew Wafaa]

Hi Richard (et al),

On 28 November 2014 at 14:04, Richard W.M. Jones <rjones(a)redhat.com&gt; wrote:
>  On Fri, Nov 28, 2014 at 05:49:39AM -0700, T.C. Hollingsworth wrote:
> > Replying by phone because holidays, sorry for any faux pas.
>
>  No problem, thanks for the quick reply!
>
> > On Nov 28, 2014 4:15 AM, "Richard W.M. Jones" <rjones(a)redhat.com&gt;
> wrote:
> > >
> > > Hello Tom and other v8 maintainers.
> > >
> > > I was looking at porting the Fedora Rawhide v8 package to aarch64,
> > > since it is a dependency of mongodb, and hence an indirect dependency
> > > of OpenStack Ceilometer.
> > >
> > > The bad news is that there is no aarch64 support in the current Fedora
> > > package.  The good news is that upstream v8 from git has aarch64
> > > support.  The not so good news is that it's not completely working.
> > >
> > > Firstly at the moment you're building from SVN-generated tarballs (is
> > > that right?).  V8 recently moved to using git.  How do you feel about
> > > using git tags instead?  The latest git tag is 3.31.27 which was
> > > released earlier today.  That tag has the aarch64 support.
> >
> > We don't typically track all v8 releases since they break API and ABI on a
> > monthly basis and are quickly no longer supported by Google. We do track v8
> > releases used by stable versions of node.js, since these are typically
> > maintained for years.
> >
> > As of right now, it looks like the upcoming nodejs 0.12 release will use
> > the 3.30 branch (it currently ships 3.30.37). aarch64 has been "officially
> > supported" since 3.25.
> >
> > If aarch64 support works in 3.30 just as well as 3.31, I'd suggest focusing
> > your polishing efforts there (along with fixing master as appropriate), so
> > it can last for awhile. If 3.31 is really better I may be able to push to
> > get node bumped upstream to make everyone's life easier.
>
>  I tested 3.30.37 and aarch64 support seems to be at the same level as
>  head.
>
>  The precise same set of 636 tests fail too.  However looking more
>  closely, (1) all the tests are in the same bit of code which is
>  something to do with math libraries AFAICT, and (2) the JIT makes
>  debugging it very difficult - there are no stack frames so gdb can't
>  make head nor tail of it.  I suggest we ignore the test failures unless
>  they actually affect mongodb/nodejs.
>
>  Anyway the bottom line is that 3.30.37 is fine.
>
> > 3.) MongoDB will need to be ported to work with newer v8. There have been
> > massive changes to the v8 API in recent versions (which is part of the
> > reason it has taken so long to get from node 0.10 to node 0.12). AFAICT no
> > porting effort has even been considered by mongodb upstream yet. [Actually,
> > they ship it with an even older version of v8 than we use it with in
> > Fedora. :-(  ]
> >
> > As of this moment my current plan to maintain mongodb support in Fedora is
> > to introduce a compat-v8-314 package at the same time we bump v8 to 3.30.
> > Which will be no problem supporting for a couple more years at least, but
> > it's still basically on life support and that doesn't bode well for arm64
> > support.
> >
> > You may want to talk to mongodb upstream first and foremost and inquire
> > about getting it to work with newer v8 versions.  I'd hate for you to do a
> > bunch of work fixing v8 on arm64 just for it to only end up benefitting
> > Chrome for Android users. ;-)
>
>  Bluuurrrrggggghhhhhhh.  Professional engineering practices FTW.
>
>  There's an upstream mongodb bug about this which doesn't look
>  encouraging:
>  https://jira.mongodb.org/browse/SERVER-10282
>  
Funnily enough the whole embedded v8 issue cam up at work and I spoke
to one of the guys at MongoDB. In a nutshell, AArch64 is not a
priority for them at present :-( They don't have the resources to put
on the task, so it is a community effort only for now.

There are also another couple of bugs for reference:
https://jira.mongodb.org/browse/SERVER-1811
https://jira.mongodb.org/browse/SERVER-13828

One thing to note with MongoDB is that they have changed their storage
engine to WiredTiger (http://www.wiredtiger.com/), I haven't had a
chance to see if this will also pose any issue for ARM.

Regards,
Andy

>  Rich.
>