Hi,
Here's what "gcj-dbtool --rebuild" needs to do:
Merge all db files in "$prefix/lib/gcj-$gcc_version/classmap.db.d/" into "$prefix/lib/gcj-$gcc_version/classmap.db".
Even better for FC5 would be eliminating the need for individual .db files altogether. We could have "gcj-dbtool --rebuild" compare the .sos in /usr/lib with the database contents and add/remove any new/missing entries to/from classmap.db. Shipping a .db file per package is one divergence from JPackage spec files that Fernando's proposal doesn't address but that an improved gcj-dbtool would.
Tom
Thomas Fitzsimmons wrote:
Here's what "gcj-dbtool --rebuild" needs to do:
Merge all db files in "$prefix/lib/gcj-$gcc_version/classmap.db.d/" into "$prefix/lib/gcj-$gcc_version/classmap.db".
Even better for FC5 would be eliminating the need for individual .db files altogether.
This would require the hashes to be stored in the .so, which I don't think they are at present. Is this something which is possible?
Also, the per-package databases are in $prefix/lib/gcj/$name these days.
Cheers, Gary
Gary Benson writes:
Thomas Fitzsimmons wrote:
Here's what "gcj-dbtool --rebuild" needs to do:
Merge all db files in "$prefix/lib/gcj-$gcc_version/classmap.db.d/" into "$prefix/lib/gcj-$gcc_version/classmap.db".
Even better for FC5 would be eliminating the need for individual .db files altogether.
This would require the hashes to be stored in the .so, which I don't think they are at present. Is this something which is possible?
Yes. It is part of the Grand Plan. But reading the hashes out of a .so is going to be far slower than out of a .db. It wouldn't be practical if you had thousands of them.
Andrew.
Andrew Haley wrote:
Gary Benson writes:
Thomas Fitzsimmons wrote:
Here's what "gcj-dbtool --rebuild" needs to do:
Merge all db files in "$prefix/lib/gcj-$gcc_version/classmap.db.d/" into "$prefix/lib/gcj-$gcc_version/classmap.db".
Even better for FC5 would be eliminating the need for individual .db files altogether.
This would require the hashes to be stored in the .so, which I don't think they are at present. Is this something which is possible?
Yes. It is part of the Grand Plan. But reading the hashes out of a .so is going to be far slower than out of a .db. It wouldn't be practical if you had thousands of them.
With hashes stored in the .so's, we'd still want to keep the .db as a sort of cache. The advantage would be that updating/rebuilding the db becomes much faster, as dbtool just needs to compare the hashes in each .so with what is in the database rather than reading all the bytecode and recomputing the hashes.
It would be too slow for the general case to have libgcj reading hashes from all the shared libraries at runtime, but it would allow things to work better in the case where the classes in a particular .so at runtime don't match the contents of the db - something that could happen during development or in case someone forgets to run dbtool. Currently we just quietly fall back to the interpreter in that case, resulting in (seemingly) mysterious performance problems.
Bryce
Bryce McKinlay writes:
Andrew Haley wrote:
Gary Benson writes:
Thomas Fitzsimmons wrote:
Here's what "gcj-dbtool --rebuild" needs to do:
Merge all db files in "$prefix/lib/gcj-$gcc_version/classmap.db.d/" into "$prefix/lib/gcj-$gcc_version/classmap.db".
Even better for FC5 would be eliminating the need for individual .db files altogether.
This would require the hashes to be stored in the .so, which I don't think they are at present. Is this something which is possible?
Yes. It is part of the Grand Plan. But reading the hashes out of a .so is going to be far slower than out of a .db. It wouldn't be practical if you had thousands of them.
With hashes stored in the .so's, we'd still want to keep the .db as a sort of cache. The advantage would be that updating/rebuilding the db becomes much faster, as dbtool just needs to compare the hashes in each .so with what is in the database rather than reading all the bytecode and recomputing the hashes.
Well, we only compute the hashes at package build time, so this won't affect installation time.
It would be too slow for the general case to have libgcj reading hashes from all the shared libraries at runtime, but it would allow things to work better in the case where the classes in a particular .so at runtime don't match the contents of the db - something that could happen during development or in case someone forgets to run dbtool. Currently we just quietly fall back to the interpreter in that case, resulting in (seemingly) mysterious performance problems.
Right. This is effectively "closing the loop", and it's one reason for putting the hashes into the DSOs. It's just one less weird thing that can go wrong.
Andrew.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Gary Benson wrote:
Also, the per-package databases are in $prefix/lib/gcj/$name these days.
Someone mentioned that files in /usr/lib are not necessarily writable, so that could break both the rebuild-gcj-db and rebuild-security providers. But these scripts are only (normally) called by rpm when it installs or uninstalls packages and /usr/lib must be writable at this point.
I don't know how I feel about the Makefile idea. In any case, it should never require user intervention. What besides sendmail is using this method anyhow? IMO, `make' should be touched by packagers only, certainly not by end users, and not even by system admins. Their work is supposed to be done by rpm out of the box must of the time. One of the good points about the existing scripts is that they are (a) completely managed by rpm (b) transparent to the user *unless they fail of course).
- -- Sincerely,
David Walluck david@zarb.org
David Walluck writes:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Gary Benson wrote:
Also, the per-package databases are in $prefix/lib/gcj/$name these days.
Someone mentioned that files in /usr/lib are not necessarily writable, so that could break both the rebuild-gcj-db and rebuild-security providers. But these scripts are only (normally) called by rpm when it installs or uninstalls packages and /usr/lib must be writable at this point.
Ah, but /usr might be *shared* between several computers, and each one needs to have its own master database. So that master database can't be in /usr.
I don't know how I feel about the Makefile idea. In any case, it should never require user intervention. What besides sendmail is using this method anyhow? IMO, `make' should be touched by packagers only, certainly not by end users, and not even by system admins.
They won't need to. The question here is how we fix our existing scripts so that they don't do unnecessary work. If we do that, then they can be called unconditionally. make is the right answer here, because it's a tool that already knows how to do the dependency analysis that's needed.
Their work is supposed to be done by rpm out of the box must of the time. One of the good points about the existing scripts is that they are (a) completely managed by rpm (b) transparent to the user *unless they fail of course).
And no-one wants to change that.
Andrew.
Andrew Haley wrote:
David Walluck writes:
I don't know how I feel about the Makefile idea. In any case, it should never require user intervention. What besides sendmail is using this method anyhow? IMO, `make' should be touched by packagers only, certainly not by end users, and not even by system admins.
They won't need to. The question here is how we fix our existing scripts so that they don't do unnecessary work. If we do that, then they can be called unconditionally. make is the right answer here, because it's a tool that already knows how to do the dependency analysis that's needed.
All it has to do is read every .db file under /usr/lib/gcj and write what it finds to a .db file in /var/lib/gcj. Where's the dependency analysis in that?
Cheers, Gary
Gary Benson writes:
Andrew Haley wrote:
David Walluck writes:
I don't know how I feel about the Makefile idea. In any case, it should never require user intervention. What besides sendmail is using this method anyhow? IMO, `make' should be touched by packagers only, certainly not by end users, and not even by system admins.
They won't need to. The question here is how we fix our existing scripts so that they don't do unnecessary work. If we do that, then they can be called unconditionally. make is the right answer here, because it's a tool that already knows how to do the dependency analysis that's needed.
All it has to do is read every .db file under /usr/lib/gcj and write what it finds to a .db file in /var/lib/gcj. Where's the dependency analysis in that?
We can already do that.
My suggestion was a way to only do that when we know we really have to. So, if you install a RPM that doesn't add or remove any .db files, nothing gets rebuilt because make doesn't see any changed dependencies.
This completely solves the problem of
That aside, having the database rebuilding as an alternative (so possibly a no-op) would also cause problems. Imagine this:
- User has GCJ as their JVM.
- a) User installs some Fedora packages. b) GCJ database is rebuilt into a consistent state.
- User switches to some other JVM.
- a) User installs some more Fedora packages. b) GCJ database is not rebuilt.
- User switches to GCJ as their JVM.
The user has ended up with a broken database. His applications will be slower, and in some (admittedly broken) cases will suddenly start to fail.
Rather than being an alternative, the rebuild command would need to rebuild all databases that could be installed on the system. This could be done by for example having all JVMs drop scripts in some database, and having 'aot-compile --rebuild' or whatever call them all in turn.
which was clipped from the mail.
Don't make database rebuilding an alternative. Use make.
Andrew.
java-devel@lists.fedoraproject.org
-
Andrew Haley -
Bryce McKinlay -
David Walluck -
Gary Benson -
Thomas Fitzsimmons