Developer's Guide

Abstract

Building Selenium Remote Control from the source repository can be a bit tricky. Selenium Remote Control is written in many languages; that means that in order to do a full build of Selenium Remote Control, you'll need to have interpreters installed for each supported language (Java, C#, Python, Ruby, and Perl).

Fortunately, there are ways around this that can make it easier for you if you only want to work on just one part of SRC.

Before you commit make sure that...

• you added and/or adapted unit/integration tests
• the code changes match a JIRA issue (create one if there is none!) and that you set the status appropriately
• you executed at least the minimum set of tests by a 'mvn install'.

These simple rules assure the robustness and reliability of Selenium and make it possible that we can provide release notes.

Of course, these rules also apply to patches provided by the community users!

IEDoc.js, IEDoc.xml and XSLT

Most of the code for the supported client drivers is generated using XSLT. The XML file is generated based on the current latest Selenium Core JavaScript files, so our client drivers never get out of date. This also allows us to embed API documentation into the generated Client Driver code, so we can use standard documentation generation tools like JavaDoc, NDoc, (E|Hap)Pydoc, RDoc or PerlDoc to present end users with documentation with they which will be familiar.

Specifically, "selenium-api.js" is a file in Selenium Core that contains a complete definition of every Selenium command. We've written a tool called "doc.js" that parses selenium-api.js and generates XML output from it (called iedoc.xml, for historical reasons).

For an example of IEDoc.xml, you can look inside the Selenium Server jar itself. (JAR files are just zip files, and can be extracted with any standard unzip tool.) "iedoc.xml" should be embedded inside selenium-server.jar in the /core directory. i.e. selenium-server.jar!/core/iedoc.xml.

You can always get the latest copy of iedoc.xml from the nightly builds of Selenium Core, available on release.openqa.org.

Developing the Java Selenium Server

(Note that If you're just developing one of the RC Client Drivers, you can safely skip this section; you don't have to build your own Selenium Server at all.)

The Java Selenium Server is officially built using Maven.

We're currently set up to use JDK 1.5.0_07 and Maven 2.0.7. To verify that you've set them up and installed them correctly, run the following commands from the command line:

> java -version
java version "1.5.0_07"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-b03)
Java HotSpot(TM) Client VM (build 1.5.0_07-b03, mixed mode)
> mvn -version
Maven version: 2.0.7

To build, start by pulling down the Selenium RC source from our Selenium RC source repository (http://svn.openqa.org/svn/selenium-rc/trunk). To commit to the repository (assuming you have developer access), you need to use the HTTPS URL. If your working copy was pulled via HTTP, you can switch to https:

svn switch --relocate http://svn.openqa.org/svn/selenium-rc/trunk https://svn.openqa.org/svn/selenium-rc/trunk

Selenium Server embeds a copy of Selenium Core in its standalone jar. By default, we'll look for a a pre-built copy of Selenium Core up on maven.openqa.org, but if you're modifying Selenium Core locally, don't forget to run mvn install in the Selenium Core build directory to make that JAR available for Selenium Server. (Otherwise, Maven will automatically use the version of Core that's available on maven.openqa.org, and ignore your valuable changes.) If you haven't already downloaded the source for Selenium Core, it can be found here (http://svn.openqa.org/svn/selenium-core/trunk/).

From the selenium-rc/trunk directory, you should be able to kick off a build of all the Java stuff from the command-line by typing

mvn install

If this is your first time running a Maven build, you may see an error message like "The plugin 'org.apache.maven.plugins:maven-site-plugin' does not exist or no valid version could be found". In this case, run

mvn install -U

If you want to build everything (including the dotnet, perl, php, python and ruby drivers, as well as the official downloadable distribution zip) you'll need to enable the rest of the Maven profiles, like this:

mvn -Pdotnet,perl,php,python,ruby,dist install

That's what we do on our official build system nightly.

Skipping the automated tests

In the course of running the build, we will also attempt to run the automated tests. On Windows, we'll automatically run tests of Firefox, IE and Opera, so make sure you've got a copy of all three of those browsers installed in the standard locations. (As of this writing July 5 2007, on other operating systems the only browser that we automatically test is Firefox.)

Assuming you have all of the relevant browsers installed, the automated tests should always pass; if they don't, it's a bug, and we need to look into that. However, if you want to just generate a binary without running the tests, you can do so by running:

mvn install -DskipTests

Running Non-Generated Tests

A selection of automated tests, found under selenium-rc/trunk/tests, are not run by the standard Maven build. To run the tests in the non-generated subfolder, simply issue

mvn test

from within that directory. To override the default browser selected for the test, use the selenium.defaultBrowser system property, e.g.

mvn test -Dselenium.defaultBrowser=*chrome

Developing the Java Selenium Server and Selenium Core Simultaneously

If you do a "build" of Selenium Core, by running mvn install in the Selenium Core directory, Maven will install your build into your personal local repository (in $HOME/.m2/repository), and Selenium RC will automatically use it. That may be good enough for you to work on both Selenium Core and Selenium Server.

But there are two problems with that. First, it can be somewhat inconvenient to have to cd to the selenium-core directory and build there, and then cd to the selenium-rc directory and build there. Second, if our official build machine finishes an official build after you've completed your build, Maven will pull down the official build and clobber your local build.

There is a better way. You can build Selenium RC using a special profile called core that will build Core together with RC. You can use the core profile like this:

mvn -Pcore install

However, out-of-the-box the core profile may not work for you, because it has to guess where you've pulled down the Selenium Core source code. By default, it will look in ../selenium-core; if that's incorrect, you'll need to pass in a corePath parameter, like this:

mvn -Pcore -DcorePath=/my/path/to/selenium-core install

That's pretty annoying to type frequently; you can workaround this by adding a new <profile> to your Maven settings.xml file in $HOME/.m2. Add it to the <profiles> section, make its <id> be core and include a <corePath> property pointing to your core location, like this:

<profiles>
  <!-- ... -->
  <profile>
    <id>core</id>
    <properties>
      <corePath>/my/path/to/selenium-core</corePath>
    </properties>
  </profile>
  <!-- ... -->
</profiles>

Then add that profile to your <activeProfiles> element, like this:

<activeProfiles>
  <!-- ... -->
  <activeProfile>core</activeProfile>
  <!-- ... -->
</activeProfiles>

Developing the Java RC Client Driver

If you follow the instructions in the Selenium Server section, you'll build both the Server and the RC Client Driver in one fell swoop. However, you don't have to build them both at once. You can go into the trunk/clients/java directory and run mvn install from there. If you don't have a locally built Selenium Server installed in your Maven local repository, we'll just fetch the latest one from maven.openqa.org and use that. (Note that if you are developing the Selenium Server or Selenium Core, you need to make sure you install it in the local repository [with "mvn install"] as you build it, or else the Java Client Driver will attempt to use the public versions of selenium-core.jar and selenium-server.jar from maven.openqa.org, instead of your own changes.)

To change Selenium.java or DefaultSelenium.java, you must make these changes in the XSLT file trunk/clients/java/iedoc2java.xml. These .java files are not checked in (and must not be checked in, or they'll get out of date).

Note that running mvn install will run the full automated test suite, including a set of tests that are automatically generated based on the HTML Selenese tests in Selenium Core. These tests are not generated using XSLT; instead, we use a generator written in Java which is built as a part of the Selenium Server. If you want to skip running these automated tests, you can use -DskipTests just like the Selenium Server.

Java Selenium RC Maven Projects

Here are the Maven projects in Selenium Remote Control:

• selenium-server-coreless (trunk/server-coreless): Builds selenium-server-coreless.jar; this includes all the code to launch browsers on various operating systems and the client-configured proxy, but doesn't include Selenium Core. (This can be used to test Selenium Core, and was created to avoid a circular dependency.)
• selenium-xlator (trunk/xlator): [EXPERIMENTAL] Builds a command-line tool that automatically translates HTML Selenese into Java using the JavaScript baked into Selenium IDE. As of this writing (July 5, 2007, Selenium RC 0.9.2) we don't actually use this xlator; we use our own pure-Java xlator in selenium-server-coreless.
• selenium-server (trunk/server): Builds selenium-server-standalone.jar; this includes selenium-server-coreless as well as selenium-core. You can use this to run your automated tests.
• selenium-client-drivers (trunk/clients): A parent POM that controls the build for the client drivers. By default, only the java client will build; to build the rest of the clients, you'll have to enable their profiles separately
• selenium-java-client-driver (trunk/clients/java): Runs the XSLT to generate DefaultSelenium.java and Selenium.java. Acts as a simple HTTP request system, no more complicated than any of the other drivers.
• other clients: The other client drivers (dotnet, perl, php, python, ruby), built as extractable zips. Again, these are only built if you explicitly enable their profiles.
• distrib (trunk/distrib): Builds the final zip that end users can download. Renames selenium-server-standalone.jar to selenium-server.jar, extracts documentation, etc. Only builds if the "distrib" profile is enabled.

selenium-server-coreless doesn't depend on any other Selenium project. selenium-server depends on selenium-server-coreless and selenium-core. We assume you're going to get selenium-core from maven.openqa.org; otherwise you'll need to install your own private selenium-core in your Maven local repository by running mvn install in the selenium-core root directory. selenium-server-coreless depends on Jetty and Ant (we use Ant as a library as well as a build tool), as well as Bouncy Castle, a pure Java SSL certificate library.

selenium-server uses the Maven assembly blugin maven-assembly-plugin to republish selenium-server's dependencies within its jar.

selenium-java-client-driver depends on selenium-server at build time and at test time (i.e. at the "test" scope) but not at runtime. Philosophically, of course, it depends on the Selenium Server in order to do its work at runtime, but it doesn't (necessarily) need the selenium-server.jar to be loaded up in the same classpath with it: the Selenium Server could easily be installed on another machine entirely and the Java Client Driver would still work. Note that this is not true of the unit tests for selenium-java-client-driver, which assume that the server is available in the classpath for running tests. (We could have written the unit tests without assuming that, but it makes the tests easier to write/run.)

Controlling the build with Maven profiles

The rest of the projects don't build by default when you run mvn install from the root of the build; instead, they only build if you're using Maven profiles. To build the dotnet client, for example, you'd need to enable its profile on the command line, like this: mvn -Pdotnet install. Alternately, you can always just cd to trunk/clients/dotnet and then run mvn install in there to build the dotnet client. (The idea is that we skip over the dotnet directory when the dotnet profile isn't enabled.)

There is one additional Maven profile called core. You can enable the core profile to build Selenium Core together with Selenium RC, all in one build. This is great for when you want to work on both Selenium Core and Selenium RC at once. However, if you simply run mvn -Pcore install, it probably won't do what you expect, because we have no way of guessing where your copy of the Selenium Core source is. By default, we look for it in "../selenium-core" relative to the root of the Selenium RC build. If you have installed it in another place, you'll need to specify a corePath property, to tell us where to look, like this:

mvn -Pcore -DcorePath=../../selenium-core/trunk install

Eclipse Projects

We have checked in Eclipse projects for our Maven projects, but they won't (can't) work until you've done a successful Maven build first (to pull down dependencies and autogenerate the code for the Java RC Client Driver).

Our Eclipse projects also depend directly on the "selenium-core" Eclipse project which is checked into the Selenium Core /selenium-core/trunk directory... you'll need to load that up into Eclipse before any of the Selenium RC Eclipse projects will work.

This requires a number of steps. If you haven't already, download the "selenium-core" source code from the subversion repository : (http://svn.openqa.org/svn/selenium-core/trunk/ ). Be sure to check out from the root of the repository tree to ensure that the Maven pom.xml file is checked out as well. Import this project into Eclipse and use maven to build it. At this point, you can then add the "selenium-core" dependency for the other projects.

Finally, Eclipse needs to know the path to the local maven repository. Therefore the classpath variable M2_REPO has to be set. Execute the following command:

mvn -Declipse.workspace=<path-to-eclipse-workspace> eclipse:add-maven-repo

You can also define a new classpath variable inside Eclipse: From the menu bar, select Window > Preferences. Select the Java > Build Path > Classpath Variables page, add "M2_REPO" and point to your Maven repository. Normally your M2_REPO is in your home directory, in a .m2/repository subdirectory. On Windows, you'll usually find the .m2 directory in the %USERPROFILE% directory, usually c:\Documents and Settings\MYUSERNAME.

Once you've built the code in Maven, added the Eclipse classpath variable, and imported the Selenium Core Eclipse project, you should be able to load up all of the Selenium RC Eclipse projects in Eclipse and build everything that way.

Don't forget to add the "selenium-core" jar that you built earlier to the project's Referenced Libraries. If you did a mvn install, this jar is in your maven repository and can be added  through Build Path -> Libraries -> Add Variable. Select the M2_REPO variable, click the Extend... button and point to the jar located in the org/seleniumhq/selenium/core/selenium-core/VERSION/selenium-core-VERSION.jar.

If you want to use the artifact built within Eclipse, you can added it through the Build Path -> Add JARs... and simply point to the selenium-core target/bin directory where the compiled jar is deployed to.

NOTE : The Java project libraries may be out of sync with the maven repositories (either because of newer versions or a rearrangement of the repository structure itself). To correct this, edit the Java Build Path for each affected project in Eclipse. Libraries which display the warning symbol usually have invalid paths to their resources. Edit these to point to the correct .jar in your .m2/repository directory. For example the most recent version of ant.jar is no longer found in the .m2/repository/ant/ant/version directory; it is now located in .m2/repository/org/apache/ant/ant/version. You must resolve all library reference inconsistencies before the build can succeed.

To run selenium-server within Eclipse, the source code provides a SeleniumServer.launch file that can be used to start the server. Edit this file to ensure that the PROGRAM_ARGUMENTS -log option points to a valid path on your system. To start the server, right click the launch file and choose the Run As... option; SeleniumServer should be an available option.

Developing the Other Client Drivers

If you're developing the other Client Drivers, (C#, Python, Ruby, Perl) you don't need to build your own Selenium Server jar; in fact, you don't need to do anything at all at the top level... you can do everything you need to do from the trunk/clients subdirectory.

You do however still need a working copy of Maven, which also acts as our XSLT runner. We'll also need to have some copy of Selenium Server around so the client driver build scripts can automatically extract iedoc.xml from it, as well as automatically start/stop the server in order to run their automated tests. If you don't provide your own, Maven will just use the latest version we can find off of maven.openqa.org automatically. (We'll automatically use any locally built version, if it has been installed with mvn install.)

Developing the .NET Client Driver

The .NET Client Driver is implemented in C#. A solution file is available with every build of Selenium RC that you can open up in Visual Studio .NET 2003, or you can use the NAnt script "solution.build" to kick off the build. (The Maven build runs the XSLT generator and launches the NAnt script.) At this time, the .NET Client Driver is only designed to work on Windows, but as far as we know it should work on Mono, too. If you're interested, I encourage you to take a look at SRC-41 (We should make a Mono-compatible .NET driver).

To automatically generate the sources you need, you can navigate to trunk/clients/dotnet and run:

mvn generate-sources

To change ISelenium.cs or DefaultSelenium.cs, you must make these changes in the XSLT file trunk/clients/dotnet/iedoc2csharp.xml. These .cs files are not checked in (and must not be checked in, or they'll get out of date).

To do a full build-and-test from the command line, you can run mvn install to do the build. This will automatically launch NDoc to generate documentation based on iedoc.xml.

Developing the Ruby Client Driver

To build the Ruby Driver, we expect Ruby, RDoc and Rake to appear on your PATH environment variable.

To automatically generate the sources you need, you can navigate to trunk/clients/ruby and run:

mvn generate-sources

To change selenium.rb, you must make these changes in the XSLT file trunk/clients/ruby/iedoc2ruby.xml. selenium.rb is not checked in (and must not be checked in, or it'll get out of date).

To do a full build-and-test from the command line, you can run mvn install to do the build; again, you'll need to specify the location of selenium-server.jar if you haven't built it locally with Maven. This will automatically launch RDoc to generate documentation based on iedoc.xml.

Developing the Python Client Driver

To build the Python Driver, we expect Python to appear on your PATH environment variable.

To automatically generate the sources you need, you can navigate to trunk/clients/python and run:

mvn generate-sources

To change selenium.py, you must make these changes in the XSLT file trunk/clients/python/iedoc2python.xml. selenium.py is not checked in (and must not be checked in, or it'll get out of date).

To do a full build-and-test from the command line, you can run mvn install to do the build; again, you'll need to specify the location of selenium-server.jar if you haven't built it locally with Maven.

iedoc2python.xml generates inline documentation as reStructuredText; by default, mvn install will automatically launch epydoc, which is checked into the trunk/clients/python/lib/epydoc directory, which in turn uses docutils, which we also have checked in. You can also build documentation using PyDoc (which comes with Python) if you like, but it doesn't look as beautiful.

Developing the Perl Client Driver

To build the Perl Driver, we expect Perl to appear on your PATH environment variable.

To automatically generate the sources you need, you can navigate to trunk/clients/perl and run:

mvn generate-sources

Unlike the other client drivers, the Perl driver is generated from XML using Perl, rather than using XSLT. You can find the driver generator in trunk/clients/perl/util/create_www_selenium.pl.

To run the tests for the Perl Driver, you can either run "mvn test" from the command line, or you can directly run "perl Makefile.PL". Running Perl on that Makefile will, in turn, autogenerate a single "Makefile" that you can use to build/run the tests. Note that this Makefile is incompatible with cygwin's GNU make; on Windows you should use nmake instead. (Note that nmake comes along with ActiveState Perl; it should already be available in your Perl/bin directory.) Once you've generated your Makefile, you can use "make test" (or "nmake test") to run the tests.

To change WWW/Selenium.pm, you must make these changes in the generator script trunk/clients/perl/create_www_selenium.pl. Selenium.pm is not checked in (and must not be checked in, or it'll get out of date).

To do a full build-and-test from the command line, you can run mvn install to do the build; again, you'll need to specify the location of selenium-server.jar if you haven't built it locally with Maven. This will automatically launch pod2html to generate documentation based on iedoc.xml.

Developing a Totally New Client Driver

Writing your own Client Driver is as easy as pie. All you need to automate tasks in the browser is to send HTTP commands to the Selenium Server, and to parse their responses.

Selenium Commands are sent to the server with HTTP Parameters, like this:

http://localhost:4444/selenium-server/driver/?cmd=getNewBrowserSession&1=*firefox&2=http%3A%2F%2Fgoogle.com

If the command is an "Accessor" (a command designed to return data), then the response will begin with "OK," followed by the data to be sent, or it will contain an error message. A normal response to this command might be:

OK,123125245345

You'll need to keep track of that string to use it again later, just like you did in the interactive tutorial.

Some commands can return more complicated values, like arrays of strings. Arrays of strings are encoded in a comma-delimited format... the values themselves are separated by "," characters. If one of the values itself contains a comma, it will prepend a backslash before the comma ("escaping" the comma), like this: "\,". If the value contains a backslash, the Server will escape the backslash, like this: "
".

Normally your workflow will be:

• Get a new browser session with "getNewBrowserSession"
• Send some commands using that session
• close the browser session with "testComplete"

We also have a detailed write-up of the Selenium Remote Control Client Driver Protocol (SRC-CDP) full of details, but that should be enough to get you started.

If you're familiar with at least one of the officially supported Client Drivers, you may benefit from viewing their source code as an example.

Auto-generating Your Client Driver Using XSLT

If you want to get fancy, you can automatically generate your Client Driver based on an XML representation of the Selenium API. All of our officially supported Client Drivers automatically generate themselves.

If you want to see examples of XSLT to generate our officially supported drivers, look here:

• Java
• C#
• Python
• Ruby
• Perl