What are the Self Tests?

Selenium RC has a mode where it can run Selenese tests. In "Self Test" mode, Selenium RC runs the Selenese tests checked-in as a part of Selenium Core.

You use RC to run the Self Tests, but the tests are really testing Selenium Core. It does have the added benefit of testing the browser launchers in Selenium RC, but that's not really the point of this type of test.

If the Self Tests pass on all browsers, then Selenium Core is pretty much ready to ship, except for a few quick minor manual tests you can run by hand in a few minutes. (RC needs additional testing to verify that it's ready to ship.)

Running the Self Tests

1. Download the latest selenium-server-standalone.jar.
   1. Go to http://maven.openqa.org/org/openqa/selenium/server/selenium-server/
   2. Click on the latest version (e.g. 0.9.3-SNAPSHOT)
   3. Scroll down to the bottom and get the last file that ends with "standalone.jar". (Don't get the tests or the sources or the .pom or the md5 or anything like that.)
2. Run the tests and store them in the directory "myOutputDirectory," like this: "java -jar selenium-server-standalone.jar -selfTest myOutputDirectory". This can quite a bit of time; we run at least some of the tests in "slow resources" (aka "slow web server") mode, to flush out any difficult timing issues.
3. Your output directory will now contain a .tar.bz2 file containing the output, as well as a bunch of .html files containing test results and logs. Look for test failures and examine the logs.

If you see a test failure in these self tests, it's definitely a bug. (Whether we can fix that bug is another story!)

Running a Single Self Test and Acquiring Debug Logs

We'll very likely need more information than the "info" logs provide; for that, you'll need to acquire "debug" logs.

1. Start the jar again, using this command: "java -jar selenium-server-standalone.jar -interactive"
2. When you're at the prompt, type a command to launch your browser: "cmd=launchOnly&1=*firefox". Instead of *firefox, you may specify *iexplore, *opera, *safari, *konqueror, *chrome, or *iehta.
3. Use check boxes to select the testing mode. To test in multiWindow mode, check the "AUT in separate window" check box. To test in slowResources mode, check the "Slow down web server" check box.
4. Scroll down to the test you want to run and click on it, but don't run the test just yet.
5. Click on "Show Log". The log window will open and the "info" level will be selected.
6. Click on the "Debug" radio button to select "Debug" logging.
7. Now run the test by clicking on the "Run the Selected Test" button (the second button from the left).
8. Copy the failing log and paste it into a bug or e-mail.