[fedora-java] Eclipse Automated Tests

[Andrew Overholt]

Hi,

I let the tests run after leaving yesterday.  Here are results, etc.

http://people.redhat.com/overholt/eclipse/eclipse-tests/results/3.1.0_fc-2.ppc.20050713

Test results:
 ___________________________________________________________________________
/           Test                    | # tst | Fail | Err |   %    |  t (s)  \
|===================================|=======|======|=====|========|==========|
|org.eclipse.ant.core               |   84  |   3  |   0 |  96.43 |  129.894 |
|org.eclipse.ant.ui                 |  164  |  13  |  29 |  74.39 |  262.837 |
|org.eclipse.compare                |   23  |   0  |   0 | 100.00 |    0.298 |
|org.eclipse.core.expressions       |   31  |   0  |   0 | 100.00 |    0.070 |
|org.eclipse.core.filebuffer        |  218  |  14  |   0 |  93.58 |   68.946 |
|org.eclipse.help                   |   91  |  10  |   2 |  86.81 |  217.543 |
|org.eclipse.jdt.core.builder       |  104  |   0  |   0 | 100.00 |  260.146 |
|org.eclipse.jdt.core.compiler      | 5844  |   6  |   0 |  99.90 |  393.635 |
|org.eclipse.jdt.core.model         | 7031  |   6  |   5 |  99.84 | 1160.622 |
|org.eclipse.jdt.core.performance   |    1  |   0  |   0 | 100.00 |    0.041 |
|org.eclipse.jdt.debug              |  405  |   8  | 134 |  64.94 | 2728.929 |
|org.eclipse.jdt.text               |  327  |  10  |   0 |  96.94 |  226.457 |
|org.eclipse.jface.text             |  100  |   0  |   0 | 100.00 |   26.548 |
|org.eclipse.ltk.core.refactoring   |    2  |   0  |   0 | 100.00 |    1.723 |
|org.eclipse.ltk.ui.refactoring     |    1  |   0  |   0 | 100.00 |    0.011 |
|org.eclipse.pde.ui                 |    0  |   0  |   0 |   NaN  |    0.000 |
|org.eclipse.swt                    | 5115  |  20  | 147 |  96.74 |   98.027 |
|org.eclipse.text                   |  366  |   0  |   0 | 100.00 |    0.516 |
|org.eclipse.ui.editors             |   11  |   0  |   0 | 100.00 |    7.985 |
|org.eclipse.ui.rcp                 |   21  |   0  |   0 | 100.00 |   23.011 |
|org.eclipse.ui.workbench.texteditor|  16   |   0  |   5 |  68.75 |    1.840 |
\____________________________________________________________________________/

Did not complete (will investigate):

org.eclipse.jdt.ui.tests.refactoring_.html
org.eclipse.jdt.ui.tests_.html       
org.eclipse.ui.tests_.html   

No tests ran (will investigate):

org.eclipse.pde.ui.tests_.html

Dramatic size differences (from Aaron's previous run [1]):

For some reason, I was getting shorter tracebacks than a previous run that
Aaron did.  This caused the majority of the size differences between my
test run and his.

org.eclipse.ant.tests.core_.html                <-- smaller tracebacks (s.t)
org.eclipse.ant.tests.ui_.html                  <-- more failures with me, (s.t)
org.eclipse.core.filebuffers.tests_.html        <-- took way longer, (s.t)
org.eclipse.help.tests_.html                    <-- more tests ran w/me
org.eclipse.jdt.core.tests.builder_.html        <-- I ran perf. as well
org.eclipse.swt.tests_.html                     <-- 20 vs. 0 failures
org.eclipse.ui.workbench.texteditor.tests_.html <-- (s.t)

My times:

The tests that took forever are due to some sort of weirdness that happened
after the actual tests completed because the test times themselves aren't
incredibly different.  I marked ones that were questionable with an
asterisk.  Comments are in comparison to upstream times.  Not all tests
listed.

Jul 12 22:41 org.eclipse.jdt.core.tests.builder_.html            (5 min)
Jul 12 22:49 org.eclipse.jdt.core.tests.compiler_.html           (8 min)
Jul 12 23:09 org.eclipse.jdt.core.tests.model_.html   faster!    (20 min)     *
Jul 13 01:17 org.eclipse.ui.editors.tests_.html       !!!!       (2 h 8 min)  *
Jul 13 01:18 org.eclipse.ui.workbench.texteditor.tests_.html     (1 min)
Jul 13 01:19 org.eclipse.ui.tests.rcp_.html                      (1 min)
Jul 13 02:06 org.eclipse.jdt.debug.tests_.html    obviously!     (47 min)     *
Jul 13 06:21 org.eclipse.ltk.ui.refactoring.tests_.html   !!!!   (4 h 15 min) *
Jul 13 06:22 org.eclipse.ltk.core.refactoring.tests_.html        (1 min)
Jul 13 06:22 org.eclipse.text.tests_.html                        (< 1 min)
Jul 13 06:23 org.eclipse.jface.text.tests_.html                  (1 min)
Jul 13 06:25 org.eclipse.core.filebuffers.tests_.html            (2 min)
Jul 13 06:30 org.eclipse.ant.tests.ui_.html                      (5 min)
Jul 13 06:33 org.eclipse.swt.tests_.html                         (3 min)

Thoughts:

We're doing well overall.  The debug tests I would expect to fail since we
don't yet have JDWP.  I should have logged everything so that I could tell
what was taking so long between tests; I will do that the next time I run
them.  Note that these were run on my G5 box with our ppc RPMs from rawhide
(but not the latest gcc from rawhide).

Andrew

[1]
http://people.redhat.com/aluchko/eclipse-3.1.0_fc-2_tests_gcj/