interpreter mystery

[Anthony Green]

gdb tells me that the azureus in x86 rawhide doesn't appear to be
running any of the Azureus or SWT .jar.so code.  It's all being
interpreted.

However, lsof tells me that both the Azureus and SWT .jar.so files are
being held open by the gij process.

This doesn't appear to be a system-wide issue, since I can see that
tomcat5 is not being interpreted (again, via gdb).

But looking at rssowl, I see the rssowl .jar.so _is_ being used, but
swt's .jar.so isn't (despite being held open by the process).

Any ideas or suggestions?

AG

[Tom Tromey]

> > > > > "Anthony" == Anthony Green  writes:

Anthony> But looking at rssowl, I see the rssowl .jar.so _is_ being used, but
Anthony> swt's .jar.so isn't (despite being held open by the process).

Anthony> Any ideas or suggestions?

It is possible for the .jar.so to be used by some classes from a given
.jar but not others.  In this situation, maybe some class that is used
rarely (say an initialization class) was used from the .jar.so and the
remaining classes are interpreted.  That could explain why you aren't
seeing it in gdb...

One way to check is to run gij with -verbose:class and look at the
output.  It will tell you which classes are defined "BC-compiled",
which are interpreted, etc.

I don't think there is a super-easy way to check the contents of a
.jar against a given .db -- but you could make a new .db using the SWT
.jar, then dump both the installed .db and the new one, and compare
class signatures using a script.  Maybe the installed .db is corrupted
or made from an old SWT or something.

Tom

[Anthony Green]

On Sun, 2006-08-27 at 15:02 -0600, Tom Tromey wrote:
> One way to check is to run gij with -verbose:class and look at the
> output.  It will tell you which classes are defined "BC-compiled",
> which are interpreted, etc.

This output looks very suspicious to me:
[Loaded (bytecode) org.bouncycastle.jce.provider.BouncyCastleProvider from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.ja\r <no certificates>)]
[Loaded (BC-compiled) org.bouncycastle.asn1.pkcs.PKCSObjectIdentifiers from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.j\ar <no certificates>)]
[Loaded (BC-compiled) org.bouncycastle.asn1.DERObjectIdentifier from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no \certificates>)]
[Loaded (BC-compiled) org.bouncycastle.asn1.DERObject from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no certificat\es>)]
[Loaded (BC-compiled) org.bouncycastle.asn1.ASN1Encodable from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no certif\icates>)]
[Loaded (pre-compiled) org.bouncycastle.asn1.DEREncodable from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no certif\icates>)]
[Loaded (pre-compiled) org.bouncycastle.asn1.DERTags from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no certificate\s>)]

This is all from the same .jar that was built to a .so using our
standard mechanism.

Similarly, it looks as though some Azureus and SWT classes are
"BC-compiled" and some are "bytecode".

BTW, I also noticed in an ant run that ant was executing natively, while
the Eclipse compiler it invokes is being interpreted.  Builds are
noticeable slower than on my FC5 machine.
> I don't think there is a super-easy way to check the contents of a
> .jar against a given .db -- but you could make a new .db using the SWT
> .jar, then dump both the installed .db and the new one, and compare
> class signatures using a script.  Maybe the installed .db is corrupted
> or made from an old SWT or something.

Gah.. this is the problem.  Somewhere along the line aot-compile-rpm is
creating partial classmap.db files.  For instance, bcprov-1.33.jar.db
only contains 331 entries but bcprov-1.33.jar has over 800 classes.

I'll file a bug (against java-1.4.2-gcj-compat-devel I suppose) and copy
relevant people.

AG

[Tom Tromey]

Anthony> This output looks very suspicious to me:

Me too.

Anthony> [Loaded (pre-compiled) org.bouncycastle.asn1.DERTags from (file:/usr/share/java/gcj-endorsed/bcprov-1.33.jar <no certificate\s>)]

"pre-compiled" means that this is compiled with the C++ ABI.  At
least, it is supposed to mean that -- it isn't clear to me whether the
test we use at runtime is 100% accurate.

In particular we do:

return c->otable_syms || c->atable_syms || c->itable_syms;

... but it seems like we could have a BC-compiled class where all of
these are NULL.  For instance a class which is nothing but a bunch of
'int' constants would probably fail this test.

So, without knowing more, it is hard to say whether this output means
there is a bug :(

Anthony> Similarly, it looks as though some Azureus and SWT classes are
Anthony> "BC-compiled" and some are "bytecode".

The 'bytecode' test is definitely accurate.

Anthony> BTW, I also noticed in an ant run that ant was executing
Anthony> natively, while the Eclipse compiler it invokes is being
Anthony> interpreted.  Builds are noticeable slower than on my FC5
Anthony> machine.

I've noticed this when running Eclipse as well.

Anthony> Gah.. this is the problem.  Somewhere along the line
Anthony> aot-compile-rpm is creating partial classmap.db files.  For
Anthony> instance, bcprov-1.33.jar.db only contains 331 entries but
Anthony> bcprov-1.33.jar has over 800 classes.

I looked at the .jar in the eclipse-ecj package and noticed this same
thing.  The .jar has 1346 .class files but the .db only has 180
entries.

Tom

[Gary Benson]

Anthony Green wrote:
> Gah.. this is the problem.  Somewhere along the line aot-compile-rpm
> is creating partial classmap.db files.  For instance,
> bcprov-1.33.jar.db only contains 331 entries but bcprov-1.33.jar has
> over 800 classes.

Ok, I see what happened.  When jarfiles are large enough to need
splitting only the classes from the first split are ending up in the
database and solib.  This affected all java-gcj-compat from 1.0.53 to
1.0.62 inclusive (so java-1.4.2-gcj-compat from 40jpp_93rh onwards).
ftp://sources.redhat.com/pub/rhug/java-gcj-compat-1.0.63.tar.gz has
the fix.

Affected rpms will need rebuilding; if you run the attached script on
binary rpms then it will tell you if they're broken or not.

Cheers,
Gary

[Anthony Green]

On Tue, 2006-08-29 at 12:49 +0100, Gary Benson wrote:
> Ok, I see what happened.  When jarfiles are large enough to need
> splitting only the classes from the first split are ending up in the
> database and solib.  This affected all java-gcj-compat from 1.0.53 to
> 1.0.62 inclusive (so java-1.4.2-gcj-compat from 40jpp_93rh onwards).
> ftp://sources.redhat.com/pub/rhug/java-gcj-compat-1.0.63.tar.gz has
> the fix.
>
> Affected rpms will need rebuilding; if you run the attached script on
> binary rpms then it will tell you if they're broken or not.

Thanks Gary.

Was Jesse planning another rebuild for FC6, or do we have to identify
all of the packages that need rebuilding?

AG

[Thomas Fitzsimmons]

Hi,

Gary Benson wrote:
> Anthony Green wrote:
> > Gah.. this is the problem.  Somewhere along the line aot-compile-rpm
> > is creating partial classmap.db files.  For instance,
> > bcprov-1.33.jar.db only contains 331 entries but bcprov-1.33.jar has
> > over 800 classes.
>
> Ok, I see what happened.  When jarfiles are large enough to need
> splitting only the classes from the first split are ending up in the
> database and solib.  This affected all java-gcj-compat from 1.0.53 to
> 1.0.62 inclusive (so java-1.4.2-gcj-compat from 40jpp_93rh onwards).
> ftp://sources.redhat.com/pub/rhug/java-gcj-compat-1.0.63.tar.gz has
> the fix.

Included in java-1.4.2-gcj-compat-1.4.2.0-40jpp.103 in Rawhide.

Tom

Regressions... Re: interpreter mystery

[Anthony Green]

On Tue, 2006-08-29 at 17:04 -0400, Thomas Fitzsimmons wrote:
> Included in java-1.4.2-gcj-compat-1.4.2.0-40jpp.103 in Rawhide.

This seems to be breaking every package I build with it.

RPM build errors:
    File not found by
glob: /var/tmp/azureus-2.5.0.0-1.4.fc6-root-mockbuild/usr/lib64/gcj/*

RPM build errors:
    File not
found: /var/tmp/rssowl-1.2.1-4.1.fc6-root-mockbuild/usr/lib/gcj/rssowl

Are the .jar.so files being installed somewhere new?

Even worse (and unrelated) there's compiler a compiler regression
preventing some packages from building:

iText on x86-64:

com/lowagie/text/pdf/BarcodePDF417.java: In class
'com.lowagie.text.pdf.BarcodePDF417':
com/lowagie/text/pdf/BarcodePDF417.java: In method
'com.lowagie.text.pdf.BarcodePDF417.<clinit>()':
com/lowagie/text/pdf/BarcodePDF417.java:0: internal compiler error:
Aborted
Please submit a full bug report,
with preprocessed source if appropriate.
See URL:http://bugzilla.redhat.com/bugzilla for instructions.
make: *** [itext-1.3.jar.1.o] Error 1
aot-compile-rpm: error: /usr/bin/make exited with code 2
error: Bad exit status from /var/tmp/rpm-tmp.89285 (%install)

kawa on ppc:

gnu/jemacs/lisp/simple.el: In class 'gnu.jemacs.lisp.simple':
gnu/jemacs/lisp/simple.el: In method
'gnu.jemacs.lisp.simple.apply2(gnu.expr.ModuleMethod,java.lang.Object,java.lang.Object)':
gnu/jemacs/lisp/simple.el:106: internal compiler error: Aborted
Please submit a full bug report,
with preprocessed source if appropriate.
See URL:http://bugzilla.redhat.com/bugzilla for instructions.

AG

Regressions... Re: interpreter mystery

[Thomas Fitzsimmons]

Hi,

Anthony Green wrote:
> On Tue, 2006-08-29 at 17:04 -0400, Thomas Fitzsimmons wrote:
> > Included in java-1.4.2-gcj-compat-1.4.2.0-40jpp.103 in Rawhide.
>
> This seems to be breaking every package I build with it.
>
> RPM build errors:
>     File not found by
> glob: /var/tmp/azureus-2.5.0.0-1.4.fc6-root-mockbuild/usr/lib64/gcj/*
>
> RPM build errors:
>     File not
> found: /var/tmp/rssowl-1.2.1-4.1.fc6-root-mockbuild/usr/lib/gcj/rssowl
>
> Are the .jar.so files being installed somewhere new?

java-1.4.2-gcj-compat-1.4.2.0-40jpp.103 had a bad bug where it would look in 
%{libdir}/%{name} instead of %{libdir}/gcj/%{name}. 
java-1.4.2-gcj-compat-1.4.2.0-40jpp.104 in Rawhide has the fix.
> Even worse (and unrelated) there's compiler a compiler regression
> preventing some packages from building:
>
> iText on x86-64:
>
> com/lowagie/text/pdf/BarcodePDF417.java: In class
> 'com.lowagie.text.pdf.BarcodePDF417':
> com/lowagie/text/pdf/BarcodePDF417.java: In method
> 'com.lowagie.text.pdf.BarcodePDF417.<clinit>()':
> com/lowagie/text/pdf/BarcodePDF417.java:0: internal compiler error:
> Aborted
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See URL:http://bugzilla.redhat.com/bugzilla for instructions.
> make: *** [itext-1.3.jar.1.o] Error 1
> aot-compile-rpm: error: /usr/bin/make exited with code 2
> error: Bad exit status from /var/tmp/rpm-tmp.89285 (%install)
>
> kawa on ppc:
>
> gnu/jemacs/lisp/simple.el: In class 'gnu.jemacs.lisp.simple':
> gnu/jemacs/lisp/simple.el: In method
> 'gnu.jemacs.lisp.simple.apply2(gnu.expr.ModuleMethod,java.lang.Object,java.lang.Object)':
> gnu/jemacs/lisp/simple.el:106: internal compiler error: Aborted
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See URL:http://bugzilla.redhat.com/bugzilla for instructions.

These could be caused by the latest backport.  Can you narrow them down?

Tom