Last Modified: June 13, 2007
+ Introduction
The testing framework is comprised of the org.eclipse.test plugin and the org.eclipse.ant.optional.junit + fragment.
These two projects are available from the dev.eclipse.org repository and are + included in the eclipse-test-framework-<buildId>.zip from the eclipse.org + downloads page.
Building and Installation
Since the org.eclipse.test plugin is stored in the repository in source-code + form, it needs to be compiled before it can be used. The org.eclipse.ant.optional.junit + fragment does not contain any source and can be used as is from the repository.
Setup
+Follow the steps given above to build and install all of the neccessary plugins + and fragments. Please note that the current version of the test framework is + not compatible with the PDE notion of self-hosting. If you want to run the tests, + you will need to setup a full target Eclipse so that the testing framework can + detect everything that is needed.
+If you are writing tests for one or more Eclipse plugins, then you should +create a separate plugin for tests. The test plugin will contain all of the +tests that will be run on your development plugins, as well as defining how +those tests get run.
+ +If you are not writing tests for an Eclipse plugin, then you should look into + using JUnit on its own to test your work. JUnit is designed for testing Java + code, and by default has no knowledge of Eclipse. There are separate mechanisms + for using JUnit on Java code in Eclipse. See the documentation provided here:
+
+JUnit Support in Eclipse
+
+ Using JUnit
Once a test plugin has been created, you must then create an Ant script that + will run all of the tests. Create a file called 'test.xml' in the root of your plug-in or bundle. + This is an Ant file that defines how each test is going to be run. The 'test.xml' + file is like a DOS batch file that scripts the entire testing process. Typically, + the 'test.xml' file will contain targets for setting up the test run, executing + the tests, cleaning up afterwards, and running the entire process.
+ +Converting existing Test Suites
+ +If you are converting an existing set of tests to use the new framework, the +actual tests that have been written should not need much change.
+If you have tests in multiple plugins, move these to a single test plug-in for your component.
+ +Make sure that the tests are defined in a plug-in. This is probably the most + common cause of confusion in the entire test process. Your tests need to be + in a plug-in so that Eclipse can find them when it tries to load them.
+ +Creating new Test Suites
+ +Creating new JUnit tests for an Eclipse plugin should be no more difficult + than writing standard JUnit tests. Since the framework allows tests to be run + inside of a working Eclipse, any tests that you write have available to them + any of the methods supplied by the Eclipse platform, provided that you add the + appropriate dependencies to your tests' manifest.
+ +Performance Issues
+You should keep in mind the number of times that Eclipse needs to be +started. Launching Eclipse has a substantial cost in terms of runtime. To +minimize the number of times the platform is started, you should consider +writing a TestSuite called AllTests for each of your test plugins. AllTests +should invoke each of the tests that you want to run for a particular plugin. +The 'test.xml' file can then run the AllTests class, which will run all of your +tests, but the platform will only ever be started once for each of your test +plugins.
Note: Sometimes tests involve shutting down, restarting, and testing the state + of metadata that was written to disk. These session tests will require Eclipse + to be launched several times in sequence, which will increase the runtime of + your tests. This cannot be avoided.
+ +
Running the Test Suite from the UI
Right click on the test.xml file and select 'Run Ant...' from the pull-down + menu. The Running Ant dialog box pops up. From the list of targets, select the + one that runs all of your tests. If you are using the example file provided + below, this target is called 'Run', and will be selected by default. Hit the + 'Finish' button to start the test process.
+ +Running the Test Suite from the command line
+When the test suites are invoked automatically, they are run from command +line. From the ${eclipse-home} directory, the command to use is:
+java -jar plugins\org.eclipse.equinox.launcher_<version>.jar -application org.eclipse.ant.core.antRunner -buildfile ${test-plugin-path}\test.xml
+-Declipse-home=${eclipse-home} -Dos=<operating system> -Dws=<windowing system> -Darch=<architecture>
+
+Individual tests can also be invoked directly. From the ${eclipse-home} +directory, use the command:
java -jar plugins\org.eclipse.equinox.launcher_<version>.jar
+ -application ${launcher} -os <operating system> -ws <windowing system> -arch <architecture> -dev bin -testpluginname ${plugin-name}
+-classname ${test-classname} formattter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,<path to output file with .xml extension>
+where ${launcher} is one of: org.eclipse.test.uitestapplication or org.eclipse.test.coretestapplication + depending on whether or not you want to run your tests within an active Workbench.
Output
By default, output from each test run is logged as XML. For each test that + is run, the output is logged into the file called ${classname}.xml. The individual + XML files are located in the ${eclipse-home} directory. When the test run is + finished, you can call the "collect" target in the library + file, which will collect the XML into a single file. See below for an example + of how to use this target correctly.
+ +Many plugin tests will not need the Workbench active in order to run. Indeed, + only the minimum number of plugins needed to run the plugin being tested need + to be present when testing in a target Eclipse. There are two different Ant + targets provided for running Eclipse plugin tests. One target starts the entire + Workbench. The other starts Eclipse with the minimum number of plugins needed. + It is up to you to decide which target is most appropriate. For examples, look + at the "ui-test" and "core-test" targets below.
+ +Some low-level tests for the Eclipse platform take actions that are not +normally possible inside of Eclipse. An example of this behaviour would be +disposing the display. While this action can be performed while running the UI, +it will also kill the UI for the copy of Eclipse that is running, and cause +errors when the Workbench tries to shutdown. If you need to test disposing the +display, or other similar actions, your tests should be running without a UI.
+ +It is very easy to forget to define your tests inside of a plugin. If your + tests will not load properly, make sure that a plug-in manifest exists in your + test project, and also that the plugin is being loaded by the platform. Make + sure that all of the dependencies are satisfied for your test plugin.
+ +Interface
The org.eclipse.test plugin defines many useful Ant tasks/targets to aid developers + in writing their test.xml scripts. Currently, there is only Ant targets defined, + which can be called using Ant's built-in <ant> task. To use these targets, + add the following line to the top of your script, and reference the ${library-file} + property when calling <ant>:
<property name="library-file"
+value="${eclipse-home}/fragments/org.eclipse.test/library.xml"/>
+
+The targets that are defined are:
For example, the following code will run the test
+org.eclipse.foo.bar.MyTest in the plugin org.eclipse.foo in a new Eclipse
+workbench. It passes the string "-Dbaz=true" to the VM. The Eclipse
+stores its metadata in the directory "data-folder".
+
+
+<ant target="ui-test" antfile="${library-file}" dir="${eclipse-home}">
+ <property name="data-dir" value="data-folder"/>
+ <property name="plugin-name" value="org.eclipse.foo"/>
+ <property name="classname" value="org.eclipse.foo.bar.MyTest"/>
+ <property name="vmargs" value="-Dbaz=true"/>
+</ant>
+
For example, the following code will run the test
+org.eclipse.foo.bar.MyTest in the plugin org.eclipse.foo in a headless Eclipse.
+It passes the string "-Dbaz=true" to the VM. The Eclipse stores its
+metadata in the directory "data-folder".
+
+
+<ant target="core-test" antfile="${library-file}" dir="${eclipse-home}">
+ <property name="data-dir" value="data-folder"/>
+ <property name="plugin-name" value="org.eclipse.foo"/>
+ <property name="classname" value="org.eclipse.foo.bar.MyTest"/>
+ <property name="vmargs" value="-Dbaz=true"/>
+</ant>
+
For example, the following code collects all of the files
+matching the pattern "org*.xml" in the directory ${eclipse-home},
+into the file named "logfile.xml".
+
+
+<ant target="collect" antfile="${library-file}" dir="${eclipse-home}">
+ <property name="includes" value="org*.xml"/>
+ <property name="output-file" value="logfile.xml"/>
+</ant>
+
Included is the 'test.xml' file from the org.eclipse.jdt.ui.tests.refactoring plugin. This +file controls all of the automated testing that is done for the org.eclipse.jdt.ui.tests.refactoring +plugin. It can be run from inside of Eclipse or from the command line. It is +intended to serve as a template file for testing any other plugin.
Notice that the structure of the file roughly mirrors that of a JUnit test. +Targets are defined for setting up the tests, defining what needs to be done, +cleaning up after the tests, and running everything in the right order.
+
+<?xml version="1.0" encoding="UTF-8"?>
+
+<project name="testsuite" default="run" basedir=".">
+ <!-- The property ${eclipse-home} should be passed into this script -->
+ <!-- Set a meaningful default value for when it is not. -->
+ <property name="eclipse-home" value="${basedir}\..\.."/>
+
+ <!-- sets the properties eclipse-home, and library-file -->
+ <property name="plugin-name" value="org.eclipse.jdt.ui.tests.refactoring"/>
+ <property name="library-file"
+ value="${eclipse-home}/plugins/org.eclipse.test/library.xml"/>
+
+ <!-- This target holds all initialization code that needs to be done for -->
+ <!-- all tests that are to be run. Initialization for individual tests -->
+ <!-- should be done within the body of the suite target. -->
+ <target name="init">
+ <tstamp/>
+ <delete>
+ <fileset dir="${eclipse-home}" includes="org*.xml"/>
+ </delete>
+ </target>
+
+ <!-- This target defines the tests that need to be run. -->
+ <target name="suite">
+ <property name="refactoring-folder"
+ value="${eclipse-home}/refactoring_folder"/>
+ <delete dir="${refactoring-folder}" quiet="true"/>
+ <ant target="ui-test" antfile="${library-file}" dir="${eclipse-home}">
+ <property name="data-dir" value="${refactoring-folder}"/>
+ <property name="plugin-name" value="${plugin-name}"/>
+ <property name="classname"
+ value="org.eclipse.jdt.ui.tests.refactoring.all.AllAllRefactoringTests"/>
+ </ant>
+ </target>
+
+ <!-- This target holds code to cleanup the testing environment after -->
+ <!-- after all of the tests have been run. You can use this target to -->
+ <!-- delete temporary files that have been created. -->
+ <target name="cleanup">
+ </target>
+
+ <!-- This target runs the test suite. Any actions that need to happen -->
+ <!-- after all the tests have been run should go here. -->
+ <target name="run" depends="init,suite,cleanup">
+ <ant target="collect" antfile="${library-file}" dir="${eclipse-home}">
+ <property name="includes" value="org*.xml"/>
+ <property name="output-file" value="${plugin-name}.xml"/>
+ </ant>
+ </target>
+ </project>
+Ant 1.3 has some issues when used with the <style> tag and absolute
+paths. Also, any tests that use the System.exit() call will not
+log their output properly when using Ant 1.3.
The test suites need to know where the root of the eclipse install is on the +file system (the ECLIPSE_HOME variable). However, this variable is only defined +in JDT. The ${eclipse-home} property can be set to a reasonable default inside +of the test.xml script. Then tests can be run from the standard Ant window, +without having to specify -Declipse-home=%ECLIPSE_HOME%. If a value for +${eclipse-home} does get passed in, the default (specified in test.xml) gets +overridden. The parameter is passed in by the build mechanism. For most cases, +the value "${basedir}/../.." is a reasonable default.
org.eclipse.test should provide Ant tasks, not template scripts.
When you run a TestSuite using the standard JUnit, it normally outputs a +series of dots to the console so that you can track the TestSuite's progress. +It is not possible to add this feature to the automated testing process at this +point in time.
Ant expects there to be a java executable on the system path. Furthermore, the executable
+must be a real file, not a symbolic link. If the test framework is throwing an exception
+java.io.IOException: java: not found, ensure that the java executable is on
+your system path.
The testing framework currently has no knowledge of PDE. In order to run the automated +you must be running a self hosting environment with a full development and target Eclipse.
+ + + +