Re: [fedora-java] db merging script

[Tom Tromey]

> >>>> "Andrew" == Andrew Haley
> <aph(a)redhat.com&gt; writes: 
Andrew> The right way to fix this is to have gcj-dbtool read a list of files
Andrew> from stdin, which is what this patch does.
Andrew> (find . -name 'temp-*.db' | gcj-dbtool - -m $DB)

This needs a corresponding gcj.texi patch.

Tom

db merging script

[Andrew Haley]

Tom Tromey writes:
>  >>>>> "Andrew" == Andrew Haley
> <aph(a)redhat.com&gt; writes:
>  
>  Andrew> The right way to fix this is to have gcj-dbtool read a list of files
>  Andrew> from stdin, which is what this patch does.
>  Andrew> (find . -name 'temp-*.db' | gcj-dbtool - -m $DB)
>  
>  This needs a corresponding gcj.texi patch. 
Yes, but I'm waiting for feedback before I write it.

Andrew.

db merging script

[Tom Tromey]

> > This needs a corresponding gcj.texi patch. 
Andrew> Yes, but I'm waiting for feedback before I write it.

I think the idea is sound, and needed.

I think we should also support reading "find -0" output,
i.e. '\0'-separated strings.  That would let us support file names
with spaces in them, and other oddities.

Tom

db merging script

[Andrew Haley]

Tom Tromey writes:
>  >> This needs a corresponding gcj.texi patch.
>  
>  Andrew> Yes, but I'm waiting for feedback before I write it.
>  
>  I think the idea is sound, and needed.
>  
>  I think we should also support reading "find -0" output,
>  i.e. '\0'-separated strings.  That would let us support file names
>  with spaces in them, and other oddities. 
Okay.  This is why I like to wait for comments...

So we have an additional "-0" arg to dbtool?

Andrew.

db merging script

[Tom Tromey]

> >>>> "Andrew" == Andrew Haley
> <aph(a)redhat.com&gt; writes: 
Andrew> So we have an additional "-0" arg to dbtool?

Sounds good to me.

Tom

db merging script

[Andrew Haley]

Tom Tromey writes:
>  >> This needs a corresponding gcj.texi patch.
>  
>  Andrew> Yes, but I'm waiting for feedback before I write it.
>  
>  I think the idea is sound, and needed.
>  
>  I think we should also support reading "find -0" output,
>  i.e. '\0'-separated strings.  That would let us support file names
>  with spaces in them, and other oddities. 
2005-04-05  Andrew Haley  <aph(a)redhat.com&gt;

	* gnu/gcj/tools/gcj_dbtool/Main.java (Fileset): New class.
	(Tokenizer): New class.
	(getFiles): New method.
	(main): Add "-" argument.
	Add "-0" argument.
	Read list of files from stdin when merging map databases.
	(usage):  Add "-" argument.
	Add "-0" argument.
	Minor corrections.

Index: gnu/gcj/tools/gcj_dbtool/Main.java
===================================================================
RCS file: /cvs/gcc/gcc/libjava/gnu/gcj/tools/gcj_dbtool/Main.java,v
retrieving revision 1.9
diff -p -2 -c -r1.9 Main.java
*** gnu/gcj/tools/gcj_dbtool/Main.java	25 Mar 2005 00:38:43 -0000	1.9
--- gnu/gcj/tools/gcj_dbtool/Main.java	5 Apr 2005 14:10:33 -0000
*************** public class Main
*** 23,27 ****
--- 23,42 ----
    public static void main (String[] s)
    {
+     boolean fileListFromStdin = false;
+     char filenameSeparator = ' ';
+ 
      insist (s.length >= 1);
+ 
+     if (s[0].equals("-") ||
+ 	s[0].equals("-0"))
+       {
+ 	if (s[0].equals("-0"))
+ 	  filenameSeparator = (char)0;
+ 	fileListFromStdin = true;
+ 	String[] newArgs = new String[s.length - 1];
+ 	System.arraycopy(s, 1, newArgs, 0, s.length - 1);
+ 	s = newArgs;
+       }
+ 
      if (s[0].equals("-v") || s[0].equals("--version"))
        {
*************** public class Main
*** 146,150 ****
        {
  	// Merge databases.
! 	insist (s.length >= 3);
  	try
  	  {
--- 161,166 ----
        {
  	// Merge databases.
! 	insist (s.length >= 3
! 		|| fileListFromStdin && s.length == 2);
  	try
  	  {
*************** public class Main
*** 156,171 ****
  	    int newSize = 0;
  	    int newStringTableSize = 0;
! 	    PersistentByteMap[] sourceMaps = new PersistentByteMap[s.length - 2];
  	    // Scan all the input files, calculating worst case string
  	    // table and hash table use.
! 	    for (int i = 2; i < s.length; i++)
! 	      {
! 		PersistentByteMap b 
! 		  = new PersistentByteMap(new File(s[i]),
! 					  PersistentByteMap.AccessMode.READ_ONLY);
! 		newSize += b.size();
! 		newStringTableSize += b.stringTableSize();
! 		sourceMaps[i - 2] = b;
! 	      }
  	    
  	    newSize *= 1.5; // Scaling the new size by 1.5 results in
--- 172,195 ----
  	    int newSize = 0;
  	    int newStringTableSize = 0;
! 	    Fileset files = getFiles(s, 2, fileListFromStdin, 
! 				     filenameSeparator);
! 	    PersistentByteMap[] sourceMaps 
! 	      = new PersistentByteMap[files.size()];
! 
  	    // Scan all the input files, calculating worst case string
  	    // table and hash table use.
! 	    {
! 	      Iterator it = files.iterator();
! 	      int i = 0;
! 	      while (it.hasNext())
! 		{
! 		  PersistentByteMap b 
! 		    = new PersistentByteMap((File)it.next(),
! 					    PersistentByteMap.AccessMode.READ_ONLY);
! 		  newSize += b.size();
! 		  newStringTableSize += b.stringTableSize();
! 		  sourceMaps[i++] = b;
! 		}
! 	    }
  	    
  	    newSize *= 1.5; // Scaling the new size by 1.5 results in
*************** public class Main
*** 173,177 ****
  	    PersistentByteMap map 
  	      = PersistentByteMap.emptyPersistentByteMap
! 	        (temp, newSize, newStringTableSize);
  
  	    for (int i = 0; i < sourceMaps.length; i++)
--- 197,201 ----
  	    PersistentByteMap map 
  	      = PersistentByteMap.emptyPersistentByteMap
! 	      (temp, newSize, newStringTableSize);
  
  	    for (int i = 0; i < sourceMaps.length; i++)
*************** public class Main
*** 297,310 ****
         + "    gcj-dbtool -n file.gcjdb [size]     - Create a new gcj map
database\n"
         + "    gcj-dbtool -a file.gcjdb file.jar file.so\n"
!        + "            - Add the contents of file.jar to a new gcj map
database\n"
         + "    gcj-dbtool -f file.gcjdb file.jar file.so\n"
!        + "            - Add the contents of file.jar to a new gcj map
database\n"
         + "    gcj-dbtool -t file.gcjdb            - Test a gcj map
database\n"
         + "    gcj-dbtool -l file.gcjdb            - List a gcj map
database\n"
!        + "    gcj-dbtool -m dest.gcjdb [source.gcjdb]...\n"
!        + "             - Merge gcj map databases into dest\n"
!        + "               Replaces dest\n"
!        + "               To add to dest, include dest in the list of
sources\n"
!        + "    gcj-dbtool -p [LIBDIR]              - Print default database
name");
    }
  
--- 321,337 ----
         + "    gcj-dbtool -n file.gcjdb [size]     - Create a new gcj map
database\n"
         + "    gcj-dbtool -a file.gcjdb file.jar file.so\n"
!        + "            - Add the contents of file.jar to a gcj map database\n"
         + "    gcj-dbtool -f file.gcjdb file.jar file.so\n"
!        + "            - Add the contents of file.jar to a gcj map database\n"
         + "    gcj-dbtool -t file.gcjdb            - Test a gcj map
database\n"
         + "    gcj-dbtool -l file.gcjdb            - List a gcj map
database\n"
!        + "    gcj-dbtool [-][-0] -m dest.gcjdb [source.gcjdb]...\n"
!        + "            - Merge gcj map databases into dest\n"
!        + "              Replaces dest\n"
!        + "              To add to dest, include dest in the list of
sources\n"
!        + "              If the first arg is -, read the list from stdin\n"
!        + "              If the first arg is -0, filenames separated by nul\n"
!        + "    gcj-dbtool -p [LIBDIR]              - Print default database
name"
!        );
    }
  
*************** public class Main
*** 406,409 ****
      return hexBytes.toString();
    }
  }
-     
--- 433,535 ----
      return hexBytes.toString();
    }
+ 
+ 
+   // Return a Fileset, either from a String array or from System.in,
+   // depending on fileListFromStdin.
+   private static final Fileset getFiles(String[] s, int startPos,
+ 					boolean fileListFromStdin,
+ 					char separator)
+   {
+     if (fileListFromStdin)
+       return new Fileset(System.in, separator);
+     else
+       return new Fileset(s, startPos, s.length);
+   }
+ }
+ 
+ // Parse a stream into tokens.  The separator can be any char, and
+ // space is equivalent to any whitepace character.
+ class Tokenizer
+ {
+   final Reader r;
+   final char separator;
+ 
+   Tokenizer(Reader r, char separator)
+   {
+     this.r = r;
+     this.separator = separator;
+   }
+ 
+   boolean isSeparator(int c)
+   {
+     if (Character.isWhitespace(separator))
+       return Character.isWhitespace((char)c);
+     else
+       return c == separator;
+   }
+ 
+   // Parse a token from the input stream.  Return the empty string
+   // when the stream is exhausted.
+   String nextToken ()
+   {
+     StringBuffer buf = new StringBuffer();
+     int c;
+     try
+       {
+ 	while ((c = r.read()) != -1)
+ 	  {
+ 	    if (! isSeparator(c))
+ 	      {
+ 		buf.append((char)c);
+ 		break;
+ 	      }
+ 	  }
+ 	while ((c = r.read()) != -1)
+ 	  {
+ 	    if (isSeparator(c))
+ 	      break;
+ 	    else
+ 	      buf.append((char)c);
+ 	  }
+       }
+     catch (java.io.IOException e)
+       {
+       }
+     return buf.toString();
+   }
+ }
+ 
+ // A Fileset is a container for a set of files; it can be created
+ // either from a string array or from an input stream, given a
+ // separator character.
+ class Fileset
+ {
+   LinkedHashSet files = new LinkedHashSet();
+   
+   Fileset (String[] s, int start, int end)
+   {
+     for (int i = start; i < end; i++)
+       {
+ 	files.add(new File(s[i]));
+       }
+   }
+ 
+   Fileset (InputStream is, char separator)
+   {
+     Reader r = new BufferedReader(new InputStreamReader(is));
+     Tokenizer st = new Tokenizer(r, separator);
+     String name;
+     while (! "".equals(name = st.nextToken()))
+       files.add(new File(name));
+   }
+ 
+   Iterator iterator()
+   {
+     return files.iterator();
+   }
+ 
+   int size()
+   {
+     return files.size();
+   }
  }

db merging script

[Andrew Haley]

And the corresponding texinfo patch.

Andrew.


Index: gcj.texi
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/gcj.texi,v
retrieving revision 1.77
diff -u -r1.77 gcj.texi
--- gcj.texi	29 Mar 2005 21:46:49 -0000	1.77
+++ gcj.texi	5 Apr 2005 14:31:02 -0000
@@ -990,9 +990,9 @@
 @c man begin SYNOPSIS gcj-dbtool
 gcj-dbtool @option{OPTION} @var{DBFILE} [@option{MORE}] @dots{}
 
-gcj-dbtool [@option{-n}] [@option{-a}] [@option{-f}]
+gcj-dbtool [@option{-0}] [@option{-}] [@option{-n}] [@option{-a}] [@option{-f}]
   [@option{-t}] [@option{-l}] [@option{-p} [@var{LIBDIR}]]
-  [@option{-v}] [@option{--version}] [@option{--help}]
+  [@option{-v}] [@option{-m}] [@option{--version}] [@option{--help}]
 
 @c man end
 @c man begin SEEALSO gij
@@ -1037,6 +1037,18 @@
 that @var{LIB} exists before adding it to the database; @option{-f}
 skips this check.
 
+@item [@option{-}][@option{-0}] -m @var{DBFILE} @var{DBFILE},[@var{DBFILE}]
+Merge a number of databases.  The output database overwrites any
+existing database.  To add databases into an existing database,
+include the destination in the list of sources.
+
+If @option{-} or @option{-0} are used, the list of files to read is
+taken from standard input instead of the command line.  For
+@option{-0}, Input filenames are terminated by a null character
+instead of by whitespace.  Useful when arguments might contain white
+space.  The GNU find -print0 option produces input suitable for this
+mode.
+
 @item -t @var{DBFILE}
 Test a database.