Hi,
I was talking with Billy last night about Eclipse unit tests and I ran the UI ones this morning. There are 806 tests and we get 152 Errors and 40 Failures. 144 of the 152 Errors are from org.eclipse.ui.tests.dnd.DragTestSuite in which Billy said to expect errors (since this is 3.1M5a I'm testing).
I'd like to set things up to run these automatically (Aaron is looking into it) and get some hard data that we can work with.
The one thing that bothered me is that the tests took about 43 minutes to run on my P4 3 GHz with 1 GB RAM. Billy says the tests take only about 10 minutes normally. I'll try to spend some time figuring out where the time goes. This could have been my machine/setup but they took about the same amount of time to run on my laptop which is a much less beefier machine so I sort of doubt it.
This is how Billy taught me to run the tests (for future reference):
. start Eclipse with a new workspace eclipse -data testworkspace . switch to CVS perspective and add the Eclipse repo (dev.eclipse.org - /home/eclipse - anonymous - pserver) . expand the Eclipse repo and expand versions . expand org.eclipse.releng . expand I20050219-1500 (the tag for 3.1M5a) . expand maps . double-click on ui.map . note that plugin@org.eclipse.ui.tests=v20050218-1200 . close ui.map and expand org.eclipse.ui.tests . check out org.eclipse.ui.tests v20050218-1200 . switch to the Java perspective . once your workspace finishes building, there will be errors in a few things ... we only care about the errors in plugin.xml so edit it to comment out or remove the two offending lines . expand Eclipse UI Tests -> org.eclipse.ui.tests . right-click on UiTestSuite.java and click Run As -> JUnit Plug-in Test . beside Package Explorer and Hierarchy, you should get a JUnit tab; quickly select it and select the Hierarchy tab within before the tests start up . let the tests run without playing with your computer because some of the tests rely on focus
Andrew
On Tue, 2005-04-05 at 10:55 -0400, Andrew Overholt wrote:
Hi,
I was talking with Billy last night about Eclipse unit tests and I ran the UI ones this morning. There are 806 tests and we get 152 Errors and 40 Failures. 144 of the 152 Errors are from org.eclipse.ui.tests.dnd.DragTestSuite in which Billy said to expect errors (since this is 3.1M5a I'm testing).
I'd like to set things up to run these automatically (Aaron is looking into it) and get some hard data that we can work with.
I've been working on running them with a headless eclipse as outlined here, http://download.eclipse.org/eclipse/downloads/drops/S-3.1M5a-200502191500/au... and using 3.0 have been able to run them however using 3.1 M5 I've been unable to successfully run them due to a number of issues.
First the instructions on the page appear to be incomplete, first the runtests script requires a directory test-eclipse/eclipse which is not created by the script, as well the line unzip -qq -o -C eclipse-junit-tests-*.zip */plugins/org.eclipse.test* does not fully expand the tests.
To run them successfully on 3.0.1 I run unzip eclipse-SDK-*.zip unzip eclipse-junit-*.zip mkdir test-eclipse cp -r eclipse test-eclipse/ ./runtests -os linux -ws gtk -arch x86 -vm /usr/lib/jvm/java-1.4.2- ibm-1.4.2.1//jre/bin/java -noclean
which works, however of M5 using the identical steps (with zips changed to tar.gz) I get
Buildfile: test.xml
all:
jdttext:
runtests:
setup:
junit:
init:
BUILD FAILED /notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test.xml:299: The following error occurred while executing this line: /notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test.xml:155: The following error occurred while executing this line: /notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test.xml:84: The following error occurred while executing this line: /notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test.xml:65: The following error occurred while executing this line: /notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test- eclipse/eclipse/plugins/org.eclipse.jdt.text.tests_3.1.0/test.xml:17: org.eclipse.ant.core.AntSecurityException
Total time: 1 second
Aaron
On Tue, 2005-04-05 at 13:54 -0400, Aaron Luchko wrote:
/notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test- eclipse/eclipse/plugins/org.eclipse.jdt.text.tests_3.1.0/test.xml:17: org.eclipse.ant.core.AntSecurityException
I think this is bug in 3.1M5a. I encountered a similar problem when I was generating the tarballs for the CDT. Maybe you should try it again with 3.1M6 and see how it goes.
Cheers, Ben
On Tue, 2005-04-05 at 14:02 -0400, Ben Konrath wrote:
On Tue, 2005-04-05 at 13:54 -0400, Aaron Luchko wrote:
/notnfs/eclipse-tests/nightlies/eclipse-tests-devel/eclipse- testing/test- eclipse/eclipse/plugins/org.eclipse.jdt.text.tests_3.1.0/test.xml:17: org.eclipse.ant.core.AntSecurityException
I think this is bug in 3.1M5a. I encountered a similar problem when I was generating the tarballs for the CDT. Maybe you should try it again with 3.1M6 and see how it goes.
Yup that fixes it, thanks for the tip, I was hoping something would be fixed in M6 that would allow them to work but hadn't yet got around to testing it.
Well fixes it as far as here,
BUILD FAILED /notnfs/eclipse-tests/temp/eclipse-testing/test.xml:300: The following error occurred while executing this line: /notnfs/eclipse-tests/temp/eclipse-testing/test.xml:161: The following error occurred while executing this line: /notnfs/eclipse-tests/temp/eclipse-testing/test.xml:84: The following error occurred while executing this line: /notnfs/eclipse-tests/temp/eclipse-testing/test.xml:65: The following error occurred while executing this line: /notnfs/eclipse-tests/temp/eclipse-testing/test- eclipse/eclipse/plugins/org.eclipse.releng.tests_3.1.0/test.xml:31: Directory /builds/t creation was not successful for an unknown reason
these tests didn't work in M5 either, but removing the relEng test plugin and target in the test.xml fixes it, unfortunately this means that we won't be able to do automated tests until we change the rpms to M6.
Aaron
these tests didn't work in M5 either, but removing the relEng test plugin and target in the test.xml fixes it, unfortunately this means that we won't be able to do automated tests until we change the rpms to M6.
The Eclipse CDT is the blocker on M6 at the moment as Eclipse 3.0 CDT M5 will flat not work with Eclipse 3.1 M6. So with that in mind, I thought it best to hold off an additional week to to let the Eclipse CDT 3.0 M6 build be published, as the schedules are so close. This will ensure that our CDT users are not left stranded. One week, one way or the other, is not too much time. So soon, very soon... ;)
Regards
Phil Muldoon
So with that in mind, I thought it best to hold off an additional week to to let the Eclipse CDT 3.0 M6 build be published, as the schedules are so close. This will ensure that our CDT users are not left stranded. One week, one way or the other, is not too much time. So soon, very soon... ;)
There is a candidate integration build that the Eclipse CDT team has released for M6. As yet it is untagged in the eclipse repository, so I built a test CDT M6 build and put the rpm/srpm on:
http://people.redhat.com/pmuldoon/cdt
I did a quick smoke test, and it appears to work with Eclipse M6 ok. Feedback appreciated.
Regards
Phil
java-devel@lists.fedoraproject.org
-
Aaron Luchko -
Andrew Overholt -
Benjamin Konrath -
Phil Muldoon